rate limit was per-endpoint not per-key - the agent applied one global backoff and throttled healthy endpoints too

Track rate limits separately per endpoint instead of one global backoff. When the API buckets limits by endpoint, a 429 on one route should pause only that route's requests while everything else keeps flowing. Split the limiter state by endpoint and the healthy calls stop paying for one hot endpoint's throttling.

rate limit was per-endpoint not per-key  -  the agent applied one global backoff and throttled healthy endpoints too

Steps

1. Read the API docs for the rate-limit scope and confirm which endpoints share a bucket. Look for phrases like per endpoint, per resource, or separate limits listed on each endpoint's page.

Expected: You can name the buckets: which endpoints share a limit and which stand alone.

1. Replace the single global limiter in the generated client with a map keyed by endpoint or route, where each entry keeps its own backoff state, retry count, and next-allowed time.

Expected: The client holds one limiter entry per endpoint instead of one for everything.

1. On a 429, back off only the limiter entry for the throttled endpoint. Requests to other endpoints continue on their normal schedule with no added delay.

Expected: A 429 on one route no longer adds delay to unrelated routes.

1. Test by hammering the throttled endpoint while other endpoints run normally. Confirm the throttled route backs off and recovers, and the healthy endpoints keep returning 200s at full speed.

Expected: 429s on one endpoint no longer slow unrelated endpoints, and each endpoint's rate recovers independently.

Use this when

• one endpoint's 429s slow down the entire generated client
• the docs show per-endpoint or per-resource rate limits
• the generated client has a single global rate limiter or one shared backoff timer

Not for this skill when

• the API documents a single global per-key limit - one limiter is correct there
• parallel workers retry in lockstep and need jitter, not bucketing
• the client aborts on 429 instead of backing off at all

Variant phrasings

per endpoint rate limiting

one slow endpoint throttles everything

rate limit buckets per route

global backoff throttling healthy endpoints

Why it happens

Agents default to the simplest limiter design: one counter, one timer, applied to every request. That matches APIs with a single per-key bucket, but many APIs bucket by endpoint or resource, so one hot route's 429s punish completely unrelated calls. The mismatch is invisible in testing because load tests rarely throttle exactly one endpoint while measuring the others.

Edge cases

• Some APIs bucket by resource ID inside one endpoint, not just by endpoint path - check the docs
• Retry-After on a 429 applies to that bucket only, so route it to the right limiter entry
• Keying too finely, for example per full URL with IDs in the path, fragments limiter state - key by route template
• A few providers change bucketing without notice, so log which bucket each 429 names when the header exists

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_kxP2ek63a6IrLztDZMt3Yg