agent escalated to the wrong on-call: stale rotation cache
This skill fixes a helpdesk agent that pages the wrong on-call engineer because it cached the rotation schedule. It shows how to fetch the current on-call at escalation time and handle the lookup failing. For agents that escalate incidents to human on-call rotations.
TL;DR
Never cache who is on call; fetch the current rotation holder at the moment of escalation. Query the scheduling system live, fall back to the team lead if the lookup fails, and log who was paged and why.
Steps
Remove any cached on-call mapping from the agent's state. Expected: The agent has no stored answer to go stale.
At escalation time, query the on-call schedule API for the current holder. Expected: The live holder is returned.
If the lookup fails, page the configured fallback and note the degraded path in the ticket. Expected: Escalation still reaches a human.
Log the paged person, the schedule source, and the timestamp. Expected: The audit shows exactly who was contacted.
When to use
- The agent pages humans for P1 or P2 incidents
- Rotations change weekly or on holidays
- A page went to someone no longer on the rotation
When not to use
- Escalation always goes to a fixed team queue
- A human reviews the escalation target before paging
- The schedule changes less than once a quarter
Why this happens
On-call rotations change on a schedule the agent does not observe, so any cached mapping decays. The failure is silent: the page goes through, just to the wrong person, and the real on-call never knows.
Edge cases
- Holiday overrides and shift swaps live outside the normal rotation; the live query must include them.
- If the scheduling API is down during an outage, the fallback path is the only thing standing.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_8-o3notPdqrxCJAW9QBRKw