Immutable solution version
Monitors as code with Terraform: resource, trio auth, explicit no-data
Complete solution
Manage Datadog monitors in Terraform with the datadog_monitor resource: provider needs api_key, app_key, and api_url, and every monitor sets notify_no_data explicitly.
Monitors as code with Terraform
Click-ops monitors rot: nobody knows which are current, thresholds drift, and the incident review finds the monitor that should have fired was edited by someone who left. Terraform fixes the audit trail, not just the creation.
Setup
Provider needs the trio: api_key, app_key, api_url (the api_url must match your site). In CI, feed them from env vars backed by the secret manager, never from committed files.
resource "datadog_monitor" "high_error_rate" {
name = "[team] checkout-api high error rate"
type = "metric alert"
query = "avg(last_5m):avg:trace.express.request.errors{service:checkout-api,env:prod} by {service} > 0.05"
message = "Error rate breached. Runbook: [link]. @slack-alerts"
monitor_thresholds {
critical = 0.05
critical_recovery = 0.01
}
notify_no_data = true
renotify_interval = 60
tags = ["team:checkout", "service:checkout-api"]
}Discipline
- notifynodata is a decision, not a default. Set it explicitly on every monitor. For critical monitors you usually want true (silent data loss is an incident).
- Name with the team first.
[team] service: whatmakes the alert list scannable at 3am. - Message carries the runbook. The alert message should link the runbook and tag the responder channel. A monitor without a next step is a notification, not an alert.
- Import existing monitors before writing new ones (the docs cover importing resources into Terraform) so apply does not duplicate or fight click-ops monitors.
- Ship apply events with dogwrap (
pip install datadog, then wrap the apply) so monitor changes show up in the event stream next to deploys.
Verify
terraform plan on a no-op change shows no diff (drift detection is the point). Trigger a test alert in staging and confirm the message, tags, and routing land where you expect.
Evidence
The Terraform integration docs show the datadogmonitor resource (name, type, query, monitorthresholds, notifynodata, renotify_interval) and dogwrap for shipping terraform apply events; the Getting Started with Terraform guide covers the monitor and monitor JSON resources plus the AWS integration with IAM roles. Verified against live docs 2026-09-26.
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_OHXN7Kku-wHk7x8drXCYEw" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Monitors as code with Terraform: resource, trio auth, explicit no-data <span>— Vectle</span></a>