Survive OpenAI rate limits: read the headers, honor Retry-After, back off with jitter

TL;DR

Fixed-interval retries from many workers synchronize into thundering herds; jitter spreads the load.

When to use

You are seeing this: Read the headers on every response: x-ratelimit-limit-requests, x-ratelimit-remaining-requests, x-ratelimit-reset-requests, and the token equivalents. Use this skill when you run into "Survive OpenAI rate limits: read the headers, honor Retry-After, back off with jitter".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Rate limits as a protocol

The steps

1. Read the headers on every response: x-ratelimit-limit-requests, x-ratelimit-remaining-requests, x-ratelimit-reset-requests, and the token equivalents. These tell you exactly where you stand; guessing from error counts is worse.
2. Honor Retry-After when it is present. It is the minimum seconds to wait before retrying. Ignoring it and retrying anyway is how a 60-second slowdown becomes a ban-shaped problem.
3. Back off exponentially with jitter on 429s. Fixed-interval retries from many workers synchronize into thundering herds; jitter spreads the load.
4. Cap retries and fail visibly. A request that has retried five times should surface an error to the caller, not spin forever. Infinite retry loops are how one slow dependency takes down your whole app.
5. Separate bulk work from interactive work. Anything that can wait 24 hours belongs in the Batch API, which has its own higher rate limit pool and half the cost. Do not burn interactive quota on backfills.
6. Plan tier headroom before launch. Know your usage tier's limits, and load-test at 2x expected traffic. Finding the limit in production is the expensive way to learn it.

The trap

Retrying aggressively on 429 without reading Retry-After, or treating rate limits as someone else's problem. The headers are a contract; code that ignores them gets throttled harder.

Checklist

• Headers parsed on every response, remaining counts tracked.
• Retry-After honored whenever present.
• Exponential backoff with jitter, retries capped.
• Bulk work routed to Batch, not the interactive pool.
• Tier limits known and load-tested before launch.

OpenAI's rate limits guide documents the Retry-After and x-ratelimit-* headers and their meanings. Verified live at https://developers.openai.com/api/docs/guides/rate-limits.