agent ignored 410 gone from webhook endpoint and kept retrying for a week

TL;DR

Map HTTP 410 to an immediate stop: archive the webhook registration, halt all retries, and send one alert with the endpoint and the action taken. Retries continue only for codes where they can help, like 429 and 5xx. Dead endpoints stop costing requests within seconds.

agent ignored 410 gone from webhook endpoint and kept retrying for a week

Use this when

• a webhook endpoint returns 410 and the agent keeps retrying
• retry logs show days of failures against the same URL
• you need a clear policy for which status codes are retried
• dead endpoints keep consuming queue workers

Not for this skill when

• the failures are 5xx or timeouts that may recover
• the endpoint moved and you need to update the URL
• you are designing the queue itself rather than the retry policy

Steps

1. Treat HTTP 410 as terminal: delete or archive the webhook registration and stop all retries immediately. Expected: zero further requests to a gone endpoint.
2. Map status codes to retry policy explicitly: retry 429 and 5xx with backoff; do not retry 4xx except 408 and 429. Expected: retries happen only where they can help.
3. On terminal codes, emit one alert with the endpoint, the last payload id, and the action taken. Expected: a human sees it once instead of a week of silent retries.
4. Keep a dead-endpoint registry so the agent never re-registers the same dead URL. Expected: re-registration is blocked with a clear reason.
5. Review the registry monthly and purge entries whose owners confirm a new endpoint. Expected: the registry stays small and accurate.

Variant phrasings

agent retried a dead webhook for days

Short version: 410 means gone; archive the registration and stop immediately.

retry loop ignored a terminal status code

Short version: map each status code to retry or stop, explicitly.

webhook endpoint is gone but retries continue

Short version: keep a dead-endpoint registry that blocks re-registration.

Compatibility: applies to any webhook delivery system in any language; the policy is a small table in the retry handler, no library pinned.

Why it happens

The retry logic treated every failure the same: failure means retry later. HTTP 410 exists precisely to say the resource is gone on purpose, but the agent never checked the code, only that the call failed. A week of retries was the loop doing exactly what it was told, on an endpoint that would never answer.

Edge cases / pitfalls

• Some APIs misuse 410 for temporary states; verify with the provider's docs before treating it as terminal for that integration.
• Archiving the registration should preserve the last payloads for debugging; delete the schedule, not the history.
• A single 410 can be a misconfigured path; confirm the URL is correct before declaring the endpoint dead.
• Do not retry harder when you see 429 alongside the 410; the 410 already ended the conversation.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_w2MP1HTO8Cb1zu8p64IV4A