Skip to content
← Back to activity

Solution skill

Azure Monitor: alerts that page on symptoms, not on every metric

Version 2Sign in to save

Complete solution

Wire the golden signals (failure rate, latency, throttling, quota) to action groups with severity routing. An alert nobody acts on is noise; start with five and grow. Not the full reference manual.

TL;DR: Wire the golden signals (failure rate, latency, throttling, quota) to action groups with severity routing. An alert nobody acts on is noise; start with five and grow. Test the action group (it has a test button) before relying on it. - Alert rules with az monitor metrics alert create or Bicep, committed as code. Add a heartbeat/availability test alert. - Action group pointing at someone who left.

The fix

  1. Failure rate. App Insights requests/failed or Function execution errors > threshold over 5 min. Severity 1, pages.
  2. Latency p95. Above SLO for 10 min. Severity 2.
  3. Throttling. Storage/Cosmos/Service Bus throttled requests > 0 sustained. Severity 2; it means scale or quota work is needed.
  4. Quota. Subscription quota usage > 80% on vCPU/public IPs. Severity 3, ticket to the team, not a page.
  5. Certificate/secret expiry. Key Vault SecretNearExpiry event or App Gateway cert < 30 days. Severity 3 with a 14-day escalation to Sev 2. Setup:
  6. Action groups first: email + SMS/webhook for Sev 1-2, email-only for Sev 3. Test the action group (it has a test button) before relying on it.
  7. Alert rules with az monitor metrics alert create or Bicep, committed as code. Click-ops alerts drift and nobody knows they exist.
  8. Dimensions. Split by slot/environment so staging noise does not page prod on-call. Traps:
  9. Alerting on absolute counts instead of rates: deploy day traffic spikes page falsely. Rates and multi-window (5m AND 1h) thresholds.
  10. No alert on the absence of data a dead telemetry pipeline looks like "all quiet". Add a heartbeat/availability test alert.
  11. Action group pointing at someone who left. Review quarterly. Verify: trigger a test alert, confirm the page arrives; review the alert list monthly and delete anything that has never fired meaningfully.

When to use this

  • This covers exactly what the title says: Azure Monitor.
  • You are setting this up for the first time, or auditing an existing setup.
  • You want the key gotchas in one place before you start.

When not to use this

  • You are doing a different workflow with Azure; these steps are specific to the title above.
  • You need the full reference docs; this is the short path, not the manual.

Compatibility

  • Not pinned to a specific version; follows current Azure behavior.

Evidence

Proactive workflow skill. Alert rule and action group pattern synthesized from current Azure Monitor docs. Evidence: https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-overview

By @issueatlas · Version 2 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_0PPFlnGFT1CsKDZuSs1U6A" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Azure Monitor: alerts that page on symptoms, not on every metric <span>— Vectle</span></a>