Survive OpenAI rate limits: read the headers, honor Retry-After, back off with jitter

Rate limits as a protocol

The steps

1. Read the headers on every response: x-ratelimit-limit-requests, x-ratelimit-remaining-requests, x-ratelimit-reset-requests, and the token equivalents. These tell you exactly where you stand; guessing from error counts is worse.
2. Honor Retry-After when it is present. It is the minimum seconds to wait before retrying. Ignoring it and retrying anyway is how a 60-second slowdown becomes a ban-shaped problem.
3. Back off exponentially with jitter on 429s. Fixed-interval retries from many workers synchronize into thundering herds; jitter spreads the load.
4. Cap retries and fail visibly. A request that has retried five times should surface an error to the caller, not spin forever. Infinite retry loops are how one slow dependency takes down your whole app.
5. Separate bulk work from interactive work. Anything that can wait 24 hours belongs in the Batch API, which has its own higher rate limit pool and half the cost. Do not burn interactive quota on backfills.
6. Plan tier headroom before launch. Know your usage tier's limits, and load-test at 2x expected traffic. Finding the limit in production is the expensive way to learn it.

The trap

Retrying aggressively on 429 without reading Retry-After, or treating rate limits as someone else's problem. The headers are a contract; code that ignores them gets throttled harder.

Checklist

• Headers parsed on every response, remaining counts tracked.
• Retry-After honored whenever present.
• Exponential backoff with jitter, retries capped.
• Bulk work routed to Batch, not the interactive pool.
• Tier limits known and load-tested before launch.

OpenAI's rate limits guide documents the Retry-After and x-ratelimit-* headers and their meanings. Verified live at https://developers.openai.com/api/docs/guides/rate-limits.