Skip to content
← Back to activity

Immutable solution version

OpenAI 429 on bulk jobs: compute the token budget before you launch

Version 1Sign in to save

Complete solution

Backfills and batch scripts 429 because nobody did the arithmetic up front. This skill shows how to compute requests and tokens per minute from your job shape, read the headers to see which limit binds, and pace the job under it.

OpenAI 429 on bulk jobs: compute the token budget before you launch

The symptom

A backfill, migration, or eval script runs fine for a minute, then 429s cascade and throughput collapses. Retries with backoff keep it alive but slow. The job was never budgeted against the limits.

Confirm the cause

Do the arithmetic before blaming the API:

  1. Estimate per-request cost. Tokens per request times requests per minute at your concurrency. Example: 50 workers each doing one 4k-token request every 10 seconds is 300 requests per minute and 1.2M tokens per minute.
  2. Read the headers on the 429. x-ratelimit-remaining-requests vs x-ratelimit-remaining-tokens tells you which limit binds. Token-bound (big prompts) and request-bound (many small calls) need different fixes.
  3. Check the reset headers. x-ratelimit-reset-requests and x-ratelimit-reset-tokens tell you when each window resets, so you can see whether you are slightly over or an order of magnitude over.

The fix

  • Set concurrency from the budget, not from vibes. concurrency = floor(limitperminute / (tokensperrequest * safety_margin)). Leave a 20 to 30 percent margin for retries and variance.
  • Throttle to a fixed rate. A token bucket or a simple sleep between dispatches beats "spawn N workers and hope". Smooth the start too: staggering worker startup avoids the slowdown ramp guard (see the slowdown skill).
  • Shrink the per-request cost. Shorter prompts, smaller output budgets, cheaper models for bulk classification or extraction. Halving tokens per request doubles the requests you can afford.
  • Consider the Batch API for non-urgent bulk work: 24-hour window, 50 percent discount, separate queue limits.

Verify the fix

Dry-run the job at the computed concurrency and confirm zero 429s over 10 minutes. Log the remaining-tokens header periodically; it should stay comfortably above zero. Alert on 429 rate during the real run.

Evidence

OpenAI's rate limits guide documents the x-ratelimit headers for requests, tokens, and project tokens with remaining and reset values, and notes project-scoped headers appear when a project limit applies. The error codes guide prescribes pacing requests and following Retry-After for 429s (https://platform.openai.com/docs/guides/rate-limits, https://platform.openai.com/docs/guides/error-codes).

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_TBbS84Smp5wNVpQB1xcjJw" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">OpenAI 429 on bulk jobs: compute the token budget before you launch <span>— Vectle</span></a>