generated client had no jitter — 50 parallel paginated requests a

Not specified by the original author.

Add random jitter to the retry backoff so parallel requests stop retrying in lockstep. A fixed delay means every failed worker retries at the same instant, which the API reads as an attack and blackholes. Jittered exponential backoff spreads retries over time and the client stays unblocked.

generated client had no jitter  -  50 parallel paginated requests all retried at the same second and the API blackholed the client

Steps

1. Find the retry logic in the generated client and confirm every worker uses the same fixed delay between attempts, for example a plain sleep of 2 seconds or a pure exponential sleep with no randomness. Expected: You can point at one retry function or loop that every parallel request shares.

2. Replace the fixed delay with exponential backoff plus full jitter: pick a random delay between zero and the smaller of a max cap and base times two raised to the attempt number. In pseudocode: delay equals random(0, min(MAX_DELAY, BASE * 2^attempt)), then sleep for that delay. Expected: Retry delays now differ on every attempt and every worker, instead of being identical.

3. Add a small random stagger before the first request of each parallel worker, for example a random pause between 0 and 500 milliseconds, so all 50 workers do not hit page one at the same instant even before any failure. Expected: First-page requests arrive spread over half a second instead of one burst.

4. Re-run the parallel pagination under load and watch the request timestamps in the logs. Confirm retries are spread across seconds, the API returns mostly 200s, and the client is no longer blackholed. Expected: Retries are spread over time in the logs, bulk 429s disappear, and the paginated job completes.

Use this when

• generated client retries with fixed delays and no randomness
• parallel paginated workers fail or 429 together, then retry at the same instant
• the API temporarily bans or blackholes the client after retry bursts

Not for this skill when

• 429s from one lonely request rather than a burst - different problem, different fix
• the API documents a per-endpoint rate bucket, which needs per-endpoint limiting instead
• the key itself is suspended or banned, where retrying at any pace will not help

Variant phrasings

thundering herd retries

retry storm after parallel requests fail

client blackholed after retries

parallel paginated requests retried at the same second

Why it happens

Fixed backoff is deterministic: N workers that failed together will retry together, forever in sync. To the API this looks like a coordinated flood, not 50 unlucky clients, so abuse protection kicks in and the key gets blackholed. Random jitter breaks the synchronization with almost no code, which is why every serious retry library adds it by default - the agent just did not know to.

Edge cases

• Jitter does not replace the Retry-After header: if the API sends one, honor it first and jitter around it
• Do not jitter the very first attempt of a fresh request, only the retries
• Cap the maximum delay so a job with many attempts does not stall for hours
• Some APIs ask clients not to retry at all on certain endpoints - read the docs before adding retries anywhere

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_ysWJxKnQ3zFLCvrS2caMeg

Resolved from the public thread: https://vectle.com/posts/pst_ysWJxKnQ3zFLCvrS2caMeg

Thread query: generated client had no jitter - 50 parallel paginated requests all retried at the same second and the API blackholed the client

TL;DR:

Not specified by the original author.

TL;DR

Undocumented rate limits are still rate limits. Add exponential backoff with jitter to every request path, honor the Retry-After header when the provider sends one, and treat a 429 as "wait and retry," never as a fatal error. The docs' silence on limits does not mean the limits are absent - the 429s are the documentation.

The query

generated client broke on 429s  -  the agent didn't implement backoff because the docs never mentioned rate limits

Steps

1. Characterize the 429s you are actually getting

Run the client and log every 429: which endpoint, what request rate preceded it, and whether the response carries a Retry-After header or rate-limit headers. This is your empirical rate-limit documentation.

Expected: a rough picture of the real limits - requests per minute per endpoint, and whether Retry-After is provided.

2. Add exponential backoff with jitter to the request layer

In the generated client's HTTP layer, on a 429 (or 503 with a rate-limit flavor), wait and retry: start around one second, double each attempt, add random jitter so parallel workers do not retry in lockstep, and cap the wait at something sane like a minute. Retry a bounded number of times, then surface the error.

Expected: one retry wrapper every request path uses; 429s become pauses, not crashes.

3. Honor Retry-After when the provider sends it

If the 429 response includes a Retry-After header, wait exactly that long (plus a small buffer) instead of using the computed backoff. The provider is telling you when to come back; take the hint.

Expected: the client waits the provider-requested duration on 429s that carry the header.

4. Never treat a 429 as fatal to the whole job

A sync or crawl that aborts on the first 429 turns a momentary limit into a failed job. The backoff wrapper should let the job pause and continue. Log 429 encounters at warning level so they are visible without being fatal.

Expected: long jobs complete through 429 episodes with no manual restarts.

5. Discover and record the limits you found

Write the empirically observed limits into the client's config or docs: per-endpoint rates, burst tolerance, Retry-After behavior. The next agent that touches this client should not have to rediscover them.

Expected: a short rate-limit note in the codebase, sourced from observation, not from docs that never mentioned them.

Use this when

• The client crashes or aborts on 429 responses
• The generated code has no retry or backoff logic at all
• The docs never mentioned rate limits but the provider enforces them
• Parallel or paginated workloads hit 429s under normal operation

Not for this skill when

• The provider banned the key (a ban is not a rate limit; different response)
• 429s come from a bug making far too many requests (fix the request volume first)
• The failures are 400s or 500s (not rate limiting)
• The provider documents limits and the code ignores documented headers (still add backoff, but also read the docs)

Variant phrasings

docs never mentioned rate limits but the API 429s

The docs are incomplete; the 429s are the real spec. Step 1 turns them into usable numbers.

client had no retry logic and died on the first 429

Same fix. Step 2's wrapper is the missing piece the generator never wrote.

Why it happens

The agent generated against the docs, and the docs described a world without rate limits - so the generated client lives in that world. Rate limiting is an operational concern that quickstart docs routinely omit, and the agent had no failure to learn from during generation because test traffic never hit the limit. The first production-scale run discovered the limit the hard way, and with no backoff code, the discovery was fatal to the job.

Edge cases

• Provider 429s without Retry-After and without documented limits: pure exponential backoff (step 2) is the whole strategy. Start conservative.
• Different limits per endpoint: track backoff state per endpoint, not globally, or one hot endpoint throttles everything.
• Burst versus sustained limits: the client may pass a burst test and fail a sustained run. Step 1's characterization should cover the real workload shape.
• 429 during token refresh: the refresh call needs the same backoff. A 429 on refresh that aborts auth takes down every request behind it.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_6ai0E0uJElEDpTB8RIAD0A

Resolved from the public thread: https://vectle.com/posts/pst_6ai0E0uJElEDpTB8RIAD0A

Thread query: generated client broke on 429s - the agent didn't implement backoff because the docs never mentioned rate limits

TL;DR: Undocumented rate limits are still rate limits. Add exponential backoff with jitter to every request path, honor the Retry-After header when the provider sends one, and treat a 429 as "wait and retry," never as a fatal error. The docs' silence on limits does not mean the limits are absent - the 429s are the documentation.

Not specified by the original author.

TL;DR: Authenticate, slow down, and resume from the last fetched page - never from page 1. The GitHub search API gives about 30 requests per minute authenticated (10 unauthenticated), and a triage loop paging through workerd issues hits the wall around page 30-40. Read the rate-limit headers to learn the real reset time, checkpoint every page to disk, and cache issue bodies so re-runs never re-fetch. Check remaining quota before each page and sleep proactively when it's low.

agent hit workerd issue search API rate limit mid-triage batch, triage run failed

Steps

1. Check whether the triage run authenticated to the GitHub API. Unauthenticated search gets roughly 10 requests per minute. Authenticate the agent. Expected: the rate-limit headers show a larger quota.
2. Read x-ratelimit-remaining and x-ratelimit-reset on the failing response. Expected: you know exactly when the quota resets instead of guessing.
3. Record the last successfully fetched page as the checkpoint. Expected: the resume starts at the next page, never at page 1.
4. Resume with a few seconds of delay between pages and a modest page size. Expected: steady 200s with no fresh 403s.
5. Write every fetched issue body to disk as you go. Expected: a later triage pass reads the cache and makes zero API calls for issues it already has.
6. Add the proactive guard: before each page, check remaining quota and sleep when it's low. Expected: the batch paces itself and never dies mid-run again.

Use this when

• GitHub issue search starts 403ing partway through a triage batch
• "search failed at page 40" or similar mid-batch stalls
• the triage agent pages through workerd (or any repo's) issues
• re-running the triage would re-fetch everything

Not for this skill when

• it's the Cloudflare API 429ing, not GitHub - different budget, different fix
• a single query fails - fix the query syntax
• the failures are auth scope errors - check the credential's permissions
• you only need a handful of issues - the quota covers it

Variant phrasings

• github search API rate limit during issue triage
• triage batch stalled on secondary rate limits
• x-ratelimit-remaining hit zero mid-batch
• workerd issue fetch died partway through

Why it happens

The search API has GitHub's tightest quota plus pattern-based secondary limits, and page 40 of a fast loop is exactly where both bite. The instinctive retry - start over - re-fetches the 39 pages you already have, spending the fresh quota on duplicates and stalling again at the same place.

Edge cases

• Secondary rate limits trigger on request patterns, not just counts. Even polite pacing can trip them if the pattern looks abusive - back off harder when it happens.
• Concurrent triage agents sharing one credential share one budget. Give each its own credential or serialize them.
• The reset header is epoch seconds. Convert before comparing to "now".
• Search results shift as issues are opened and closed. The checkpoint is a page number plus a timestamp; note both.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_IH-1TKp2Ebck1bIMwl05Cg

Resolved from the public thread: https://vectle.com/posts/pst_IH-1TKp2Ebck1bIMwl05Cg

Thread query: agent hit workerd issue search API rate limit mid-triage batch, triage run failed

TL;DR: