agent hit wrangler API rate limit mid-batch, secret rotation failed halfway

TL;DR: Stop the per-secret loop and finish with one bulk operation. Each wrangler secret put is an API call against the shared account budget, so rotating many secrets in a tight loop 429s halfway and leaves the worker with a mix of old and new values. Figure out from the run log which secrets already rotated (values are write-only, the log is the only record), wait out the limit, then pipe the remaining secrets into wrangler secret bulk as JSON. Verify with a canary before discarding the old values.

agent hit wrangler API rate limit mid-batch, secret rotation failed halfway

Steps

1. Stop the loop. Build the rotated/not-rotated list from the run log. Expected: a definitive list. Secret values can't be read back, so the log is the only source of truth.
2. Wait out the rate-limit window - a few minutes - honoring any Retry-After header. Expected: the next API call returns a 200 instead of a 429.
3. Rotate the remaining secrets in one call: pipe a JSON object of the remaining secrets into wrangler secret bulk. Expected: one API call, all remaining secrets updated, no per-secret 429s.
4. Verify with a canary: exercise the code paths that use each rotated secret. Expected: the worker behaves correctly with the new values.
5. Keep the old secret values stored safely until the canary passes. Expected: a bad rotation can be rolled back in minutes.
6. Harden the routine: bulk by default, checkpoint per secret, one environment at a time, old values kept until verified. Expected: the next rotation can't half-finish.

Use this when

• rotating many Worker secrets and later ones fail with 429s
• "secret rotation failed halfway" with a mix of old and new values live
• the run log shows some secret put calls succeeding before the errors
• bulk secret updates of any size

Not for this skill when

• a single secret fails - check its name, value, and your permissions
• secrets fail validation rather than rate limiting - fix the name or value
• the failures are 401s - re-authenticate, don't back off
• you only have a handful of secrets - the loop is fine, the budget covers it

Variant phrasings

• wrangler secret put rate limited during rotation
• bulk secret rotation partially completed
• too many requests while rotating worker secrets
• half the secrets rotated before 429s started

Why it happens

Every secret update is an API call against the account's shared rate budget, and N secrets in a tight loop is N calls in seconds. Secret values are write-only, so without a checkpoint there's no way to ask the API which half rotated - the agent is left guessing, and guessing wrong means either skipping secrets or double-rotating.

Edge cases

• Secret names have format constraints. A name that fails validation fails in bulk too - validate names first.
• The bulk input must be valid JSON mapping names to values. A malformed file fails the whole batch.
• Rotating across environments doubles the call count. Finish one environment completely before starting the next.
• If the old and new values are both valid credentials upstream, the canary can't distinguish them. Verify behavior, not just absence of errors.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_3uGwr-LRaq5clQdCAUKNZw