support agent hit zendesk api 429 mid bulk reply: backoff resume

TL;DR

A bulk reply job that dies on a 429 without a checkpoint has to restart from zero, re-sending replies it already sent. Checkpoint every completed reply durably, back off with exponential delay honoring Retry-After, and resume from the checkpoint. Design the job so a 429 is a pause, not a failure.

The query

support agent hit zendesk api 429 mid bulk reply: backoff resume

Use this when

• Bulk reply jobs die on 429 halfway through
• Replies get duplicated after a retry
• You are designing bulk ticket operations
• Rate limits interrupt long jobs

Not for

• Authentication failures
• Reply copy or tone
• Which tickets to bulk-reply
• Ticket assignment logic

Steps

1. Checkpoint every completed reply durably

Write the ticket id and reply id to durable storage after each successful send, before moving to the next ticket. The checkpoint is what makes resume possible. In-memory progress dies with the job.

Expected output: per-reply checkpoint in durable storage.

2. Back off exponentially on 429, honoring Retry-After

On a 429, wait the Retry-After duration if present, then retry with exponential backoff and jitter. Do not retry immediately: immediate retries extend the throttle and risk a longer ban.

Expected output: retries spaced correctly, job continues after the throttle lifts.

3. Make replies idempotent before sending

Attach an idempotency key or check for an existing identical reply before sending, so a retried batch never double-posts. Checkpoint plus idempotency together make retries safe.

Expected output: retried sends never duplicate replies.

4. Resume from the checkpoint, never from zero

On restart, read the checkpoint and skip completed tickets. Verify the skip logic against a real partial run before trusting it. Restart-from-zero is how duplicates happen.

Expected output: resumed jobs skip completed work correctly.

5. Pace the job under the sustained limit

Size batches and sleeps so the job runs under the sustained rate limit, not the burst limit. A job that survives bursts but dies on sustained throttling was paced wrong. Monitor the 429 rate as the pacing signal.

Expected output: steady-state pacing with near-zero 429s.

Template: the checklist

Bulk reply 429 recovery:
[ ] Per-reply checkpoint in durable storage
[ ] Exponential backoff with jitter, Retry-After honored
[ ] Idempotent sends: no duplicate replies on retry
[ ] Resume-from-checkpoint verified on a partial run
[ ] Pacing under the sustained limit, 429 rate monitored

Variant phrasings

zendesk bulk update rate limited

Steps 2 and 5. Backoff plus pacing.

bulk reply duplicated after retry

Steps 1 and 3. Checkpoint plus idempotency.

resume failed bulk job zendesk

Steps 1 and 4. Checkpoint, then resume from it.

Why it happens

Bulk jobs concentrate normal-paced API usage into bursts that exceed sustained limits. Without checkpointing, the failure mode is catastrophic (restart from zero); with it, the failure mode is a pause. Most bulk-job code is written for the happy path and meets the 429 for the first time in production.

Edge cases

• Partial batch failures: some sends in a batch succeed before the 429. Checkpoint per item, not per batch.
• Long jobs crossing rate-limit windows: limits reset on timers; a job that pauses correctly rides through resets.
• Parallel workers: divide the ticket list explicitly so workers do not overlap after resume.
• Reply ordering: if reply order matters, single-thread the job. Parallel resume scrambles order.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstHapCCICtuUbnMGyqd-Nig