# Fix the webhook retry storm from the translation agent polling status

## TL;DR

The agent polls aggressively and webhooks redeliver on top, so switch to webhook-driven updates with backoff polling as fallback. Polling every few seconds plus retried webhooks multiplies into a storm that can take down the deploy. Register the webhook, poll rarely as a safety net, and dedupe events by id.

## The error

```text
Deploy failed under load: 12,000 status polls in 10 minutes
translation agent polling plus webhook retries compounded
```

## Fix it

### Step 1: Confirm the storm in the logs

```bash
grep -c "polling translation status" logs/agent.log
```

Expected: The count is enormous.

### Step 2: Register a webhook for status changes

```bash
node -e "console.log('point the TMS webhook at your status endpoint, verify the signature')"
```

Expected: Updates arrive pushed instead of polled.

### Step 3: Drop polling to a rare safety net

```bash
node -e "console.log('poll every 5 minutes max, only for jobs with no webhook event in 10 minutes')"
```

Expected: Polling becomes the exception.

### Step 4: Dedupe events by id

```bash
node -e "console.log('pattern: keep a set of seen event ids, ignore repeats')"
```

Expected: Redeliveries stop multiplying work.

## When to use this

- A translation agent polls status aggressively
- Webhook retries plus polling cause a storm

## When NOT to use this

- Polling is gentle and webhooks work, leave it alone
- The deploy failed for other reasons, check those

## Tool and version compatibility

- TMS webhooks, status polling fallbacks
- Event idempotency by event id

## Variant phrasings

### no webhook support in the TMS

Then poll with exponential backoff and jitter, and cap the poll rate globally.

### storm during deploys specifically

Deploys restart the agent and it re-polls everything. Persist poll state across restarts.

## Why it happens

Polling is pull, webhooks are push, and running both at full rate double-counts every event. Retried webhooks add duplicates, the agent treats each as new work, and the load compounds until something falls over.

## Edge cases

- Webhooks need their own retry and dead-letter handling, do not lose events silently
- The safety-net poll interval should exceed the webhook retry window
- Load-test the status endpoint, it is now in the hot path

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_zGz1Abjq_ro4YSbtUx6Fjw
