Baseline alert set for any service:
1. **Failure rate.** App Insights `requests/failed` or Function execution errors > threshold over 5 min. Severity 1, pages.
2. **Latency p95.** Above SLO for 10 min. Severity 2.
3. **Throttling.** Storage/Cosmos/Service Bus throttled requests > 0 sustained. Severity 2; it means scale or quota work is needed.
4. **Quota.** Subscription quota usage > 80% on vCPU/public IPs. Severity 3, ticket to the team, not a page.
5. **Certificate/secret expiry.** Key Vault `SecretNearExpiry` event or App Gateway cert < 30 days. Severity 3 with a 14-day escalation to Sev 2.
Setup:
- **Action groups** first: email + SMS/webhook for Sev 1-2, email-only for Sev 3. Test the action group (it has a test button) before relying on it.
- **Alert rules** with `az monitor metrics alert create` or Bicep, committed as code. Click-ops alerts drift and nobody knows they exist.
- **Dimensions.** Split by slot/environment so staging noise does not page prod on-call.
Traps:
- Alerting on absolute counts instead of rates: deploy day traffic spikes page falsely. Rates and multi-window (5m AND 1h) thresholds.
- No alert on the absence of data: a dead telemetry pipeline looks like "all quiet". Add a heartbeat/availability test alert.
- Action group pointing at someone who left. Review quarterly.
Verify: trigger a test alert, confirm the page arrives; review the alert list monthly and delete anything that has never fired meaningfully.