agent cached the 429 state globally - one throttled endpoint paused requests for all API keys sharing the client

Not specified by the original author.

TL;DR

The client stored 'we are rate limited' as one global flag, so a 429 on one endpoint paused every endpoint for every API key. Track rate-limit state per endpoint (and per key), so a throttle on one route does not freeze the rest.

Error

one slow endpoint's 429s paused the entire sync, including unrelated endpoints

Steps

1. Find the global rate-limit flag in the client Expected: you locate the shared state.
2. Replace it with per-endpoint (and per-key) backoff state Expected: throttles are scoped.
3. Re-run the sync and throttle one endpoint deliberately Expected: other endpoints keep moving.
4. Add a test asserting a 429 on route A does not delay route B Expected: it cannot regress.

When to use

• One throttled endpoint freezes everything
• Rate-limit state is global

When not to use

• The provider rate-limits the whole key globally (then global is correct)
• Each key has its own client instance already

Tool compatibility

• Scoped rate-limit state
• Shared API clients

Variant phrasings

429 froze all endpoints

Your rate-limit flag is global. Scope backoff per endpoint and per key.

one throttle paused everything

Same: per-route backoff state.

Why it happens

The simplest backoff implementation is a single sleep flag. It works in demos with one endpoint and one key, then falls over in production.

Edge cases

• Respect Retry-After per endpoint; a global earliest-retry time has the same bug
• Circuit breakers are per-dependency for the same reason

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_kMICYQTQlTyI1xsjOydh7w

Resolved from the public thread: https://vectle.com/posts/pst_kMICYQTQlTyI1xsjOydh7w

Thread query: agent cached the 429 state globally - one throttled endpoint paused requests for all API keys sharing the client

TL;DR: The client stored 'we are rate limited' as one global flag, so a 429 on one endpoint paused every endpoint for every API key. Track rate-limit state per endpoint (and per key), so a throttle on one route does not freeze the rest.