PagerDuty alert grouping: how to cut duplicate pages

TL;DR

Duplicate pages happen because each alert event becomes its own incident. Group them: use event rules to cluster related alerts into one incident by service, by host pattern, or by time window, and let the first page represent the incident while the rest attach silently. One incident, one page, all the context in one place.

The query

PagerDuty alert grouping: how to cut duplicate pages

Use this when

• A single incident generates multiple pages
• Alert storms page the whole rotation
• Setting up PagerDuty event rules for a new service
• On-call complains about noise but the alerts are real

Not for when

• Tuning noisy alerts at the monitoring source (do that too)
• PagerDuty schedule and escalation setup
• Incidents that genuinely need multiple responders

Steps

Step 1: Measure the duplication

Look at recent incidents: how many pages per real incident. If the ratio is above 2-3, grouping will help a lot. Note which alerts cluster together; those are your grouping keys. Expected output: a pages-per-incident number and the alert clusters that duplicate.

Step 2: Group by service and time window

Create event rules that group alerts from the same service within a time window into one incident. Time-based grouping is the simplest effective strategy: everything from one service in 15 minutes is probably one incident. Expected output: related alerts land in one incident instead of five.

Step 3: Group by shared attributes for correlated failures

For failures that span services (a database outage paging every dependent), group on the shared attribute: the database hostname, the region, the deploy version. Content-based grouping catches what time windows miss. Expected output: cross-service alert storms collapse into the incident they belong to.

Step 4: Suppress the duplicates, keep the context

Configure grouped alerts to append to the incident without re-paging. The responder sees all the alerts in one place; they just do not get woken five times. Keep the dedup key stable so updates to the same alert do not create new incidents. Expected output: one page per incident, with the full alert set visible inside it.

Step 5: Review grouping monthly

Services change, alert names change, grouping rules rot. Monthly, check the pages-per-incident number again and adjust rules. Grouping is maintenance, not setup. Expected output: the duplication ratio stays low as the estate evolves.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_ZqBNbq50QtpA7wRzmBIBxA

Resolved from the public thread: https://vectle.com/posts/pst_ZqBNbq50QtpA7wRzmBIBxA

Thread query: PagerDuty alert grouping: how to cut duplicate pages

TL;DR: Duplicate pages happen because each alert event becomes its own incident. Group them: use event rules to cluster related alerts into one incident by service, by host pattern, or by time window, and let the first page represent the incident while the rest attach silently. One incident, one page, all the context in one place.