agent froze mid-alert-dispatch: Slack webhook timed out

TL;DR

A synchronous send to a timed-out webhook freezes everything queued behind it, and the alerts most likely to matter are the ones stuck in that queue. Give each send a short timeout, isolate alerts so one failure cannot block others, record undelivered alerts durably, and fall back to a second channel for the highest severity after the first failure.

agent froze mid-alert-dispatch: Slack webhook timed out

Use this when

• Alert dispatch stalls and later alerts never arrive
• One webhook endpoint is slow or unresponsive
• The agent process hangs rather than erroring

Not for this skill when

• The messaging platform is down globally and no channel works
• Alerts are generated but the dispatch step is never called

Steps

1. Reproduce the freeze with a deliberately slow test endpoint and observe where the agent blocks. Verify: the blocking call and its missing timeout are confirmed
2. Add explicit send timeouts sized in seconds and treat timeout as a per-alert failure. Verify: a slow endpoint now costs seconds, not the dispatch run
3. Restructure dispatch so alerts send independently (queue or parallel with isolation) and failures persist for retry. Verify: one bad alert cannot starve the rest in a test with a poisoned first alert
4. Configure a fallback channel for critical alerts that triggers on primary failure. Verify: a critical test alert arrives via fallback when the primary is down
5. Add dispatch latency and failure metrics with an alert on the alerting path itself. Verify: the team learns about dispatch decay from metrics, not from missing alerts

Variant phrasings

Slack webhook timeout froze agent

A related phrasing for the same underlying problem.

alert dispatch blocked by slow webhook

A related phrasing for the same underlying problem.

per alert timeout and fallback

A related phrasing for the same underlying problem.

Compatibility: Compatibility: Slack or Teams incoming webhooks and similar notification endpoints called from agent runtimes, current versions.

Why it happens

Alerting code written as a simple function call inherits the worst behaviour of the network: unbounded waiting. Timeouts, isolation, and durable retry are what make an alert path trustworthy, and the fallback channel covers the platform-wide case a retry cannot.

Edge cases / pitfalls

• Webhook timeouts are ambiguous: the message may have posted. Dedupe retries with an alert id in the text to avoid duplicate storms.
• Fallback channels need the same severity routing as the primary, or critical alerts land in a channel nobody watches at night.
• Keep the dispatch path dependency-light; every library between the agent and the webhook is another way to hang.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstGRD61FRS0pGDMDXzjw0xQ