generated client broke on 429s - the agent didn't implement backoff because the docs never mentioned rate limits

TL;DR

Undocumented rate limits are still rate limits. Add exponential backoff with jitter to every request path, honor the Retry-After header when the provider sends one, and treat a 429 as "wait and retry," never as a fatal error. The docs' silence on limits does not mean the limits are absent - the 429s are the documentation.

The query

generated client broke on 429s  -  the agent didn't implement backoff because the docs never mentioned rate limits

Steps

1. Characterize the 429s you are actually getting

Run the client and log every 429: which endpoint, what request rate preceded it, and whether the response carries a Retry-After header or rate-limit headers. This is your empirical rate-limit documentation.

Expected: a rough picture of the real limits - requests per minute per endpoint, and whether Retry-After is provided.

2. Add exponential backoff with jitter to the request layer

In the generated client's HTTP layer, on a 429 (or 503 with a rate-limit flavor), wait and retry: start around one second, double each attempt, add random jitter so parallel workers do not retry in lockstep, and cap the wait at something sane like a minute. Retry a bounded number of times, then surface the error.

Expected: one retry wrapper every request path uses; 429s become pauses, not crashes.

3. Honor Retry-After when the provider sends it

If the 429 response includes a Retry-After header, wait exactly that long (plus a small buffer) instead of using the computed backoff. The provider is telling you when to come back; take the hint.

Expected: the client waits the provider-requested duration on 429s that carry the header.

4. Never treat a 429 as fatal to the whole job

A sync or crawl that aborts on the first 429 turns a momentary limit into a failed job. The backoff wrapper should let the job pause and continue. Log 429 encounters at warning level so they are visible without being fatal.

Expected: long jobs complete through 429 episodes with no manual restarts.

5. Discover and record the limits you found

Write the empirically observed limits into the client's config or docs: per-endpoint rates, burst tolerance, Retry-After behavior. The next agent that touches this client should not have to rediscover them.

Expected: a short rate-limit note in the codebase, sourced from observation, not from docs that never mentioned them.

Use this when

• The client crashes or aborts on 429 responses
• The generated code has no retry or backoff logic at all
• The docs never mentioned rate limits but the provider enforces them
• Parallel or paginated workloads hit 429s under normal operation

Not for this skill when

• The provider banned the key (a ban is not a rate limit; different response)
• 429s come from a bug making far too many requests (fix the request volume first)
• The failures are 400s or 500s (not rate limiting)
• The provider documents limits and the code ignores documented headers (still add backoff, but also read the docs)

Variant phrasings

docs never mentioned rate limits but the API 429s

The docs are incomplete; the 429s are the real spec. Step 1 turns them into usable numbers.

client had no retry logic and died on the first 429

Same fix. Step 2's wrapper is the missing piece the generator never wrote.

Why it happens

The agent generated against the docs, and the docs described a world without rate limits - so the generated client lives in that world. Rate limiting is an operational concern that quickstart docs routinely omit, and the agent had no failure to learn from during generation because test traffic never hit the limit. The first production-scale run discovered the limit the hard way, and with no backoff code, the discovery was fatal to the job.

Edge cases

• Provider 429s without Retry-After and without documented limits: pure exponential backoff (step 2) is the whole strategy. Start conservative.
• Different limits per endpoint: track backoff state per endpoint, not globally, or one hot endpoint throttles everything.
• Burst versus sustained limits: the client may pass a burst test and fail a sustained run. Step 1's characterization should cover the real workload shape.
• 429 during token refresh: the refresh call needs the same backoff. A 429 on refresh that aborts auth takes down every request behind it.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_6ai0E0uJElEDpTB8RIAD0A