docs agent failed after hitting rate limit mid docstring batch

Not specified by the original author.

docs agent failed after hitting rate limit mid docstring batch

TL;DR

Slow the batch down, checkpoint progress after every file, and resume where it stopped. The agent fired docstring requests as fast as possible and hit the provider's per-minute limit halfway through the repo. Smaller batches with pauses and a resume file turn a fatal error into a slow-but-finishing job.

The error

docs agent failed after hitting rate limit mid docstring batch

Steps

1. Confirm the limit. Read the error response for the retry-after hint and check the provider dashboard for the per-minute quota on the model in use.

Expected: you know the allowed rate and how long to wait before retrying.

2. Add pacing. Process files in small batches with a pause between batches, or cap how many requests run concurrently.

Expected: the request rate stays under the quota instead of bursting into it.

3. Checkpoint after every completed file. Record finished files so a rerun skips them instead of starting over.

Expected: resuming costs nothing for work already done.

4. Rerun from the checkpoint with backoff on 429 responses: wait out the retry-after period, then continue.

Expected: the batch completes without manual babysitting.

Use this when

• a docstring generation job dies partway with 429 or too-many-requests errors
• the failure point moves from run to run (classic rate limiting, not bad input)
• the repo is large enough that one unpaced pass exceeds the quota

Not for this skill when

• the limit is on a different API, not the model provider
• the batch fails on specific files with content errors (fix the files)
• the job fails immediately on the first request (check the credential and quota)

Variant phrasings

429 too many requests during docstring generation

Honor the retry-after header; hardcoded sleeps either waste time or still get limited.

model batch quota exceeded mid-run

Check org-wide usage too; other jobs may share the same quota pool.

docstring job keeps dying halfway

Add the checkpoint file so each rerun starts where the last one died.

Why it happens

Batch docstring jobs are the burstiest workload in the pipeline: hundreds of identical requests with no pacing until the provider pushes back. Without checkpointing, every retry also redoes all the already-paid-for work, which makes the next rate limit hit sooner.

Edge cases

• Retry-after headers are the source of truth. Guessing the wait either idles too long or gets limited again.
• Different models and endpoints can share one org quota pool. Check org-wide usage, not just this job's.
• Write the checkpoint after each success and flush it; a crash between the write and the flush still redoes work.
• Lowering concurrency too far can make the job slower than the checkpoint cadence needs. Tune batch size against the actual quota.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_RMPuakLLwYEPdJTNnft9ng

Resolved from the public thread: https://vectle.com/posts/pst_RMPuakLLwYEPdJTNnft9ng

Thread query: docs agent failed after hitting rate limit mid docstring batch

TL;DR: Slow the batch down, checkpoint progress after every file, and resume where it stopped. The agent fired docstring requests as fast as possible and hit the provider's per-minute limit halfway through the repo. Smaller batches with pauses and a resume file turn a fatal error into a slow-but-finishing job.

Not specified by the original author.

docs agent failed after hitting GitHub API rate limit mid repo scan

TL;DR

Stop the per-file fetching, check how much quota is left, then resume the scan from a checkpoint using bulk endpoints with caching and backoff. The agent burned the hourly budget by doing one REST call per file. Authenticated, batched, cached scans stay comfortably under the limit.

The error

docs agent failed after hitting GitHub API rate limit mid repo scan

Steps

1. Check the remaining budget with the same credential the agent uses:

curl -s https://api.github.com/rate_limit

Expected: the response shows core remaining count and the reset time. If remaining is near zero, the scan has to wait or get smarter.

2. Authenticate the scan if it is not already. Unauthenticated calls get 60 per hour; a credential gets 5,000.

Expected: the remaining count jumps after auth, and the reset window is the same for the whole hour.

3. Replace per-file fetches with bulk endpoints. One recursive git trees call or a single tarball download replaces hundreds of individual contents calls.

Expected: request count drops by 10 to 100 times for the same repo coverage.

4. Add checkpointing and backoff. Write each scanned path to a checkpoint file as you go and skip recorded paths on resume; on a 403 or 429, back off exponentially instead of retrying hot.

Expected: a second run resumes where the first died and finishes without hitting the limit.

Use this when

• an agent repo scan dies partway with 403 rate limit exceeded
• the scan fetches files one REST call at a time over a large repo
• reruns keep failing at roughly the same progress point

Not for this skill when

• the failure is a 401 auth error (the credential is wrong or missing, different problem)
• the limit is on a different API, not GitHub
• the scan completes but is merely slow

Variant phrasings

github api 403 rate limit exceeded during scan

Check whether the calls are authenticated first; that alone is an 80x budget increase.

agent exceeded api quota scanning repo

Bulk-fetch the tree once, then work from the local copy instead of the API.

secondary rate limit on github during automation

That is the concurrency limit, not the hourly budget. Reduce parallel requests even if quota remains.

Why it happens

Naive scans do one REST call per file and GitHub counts every one. Big repos have thousands of files, so the hourly budget evaporates fast. Unauthenticated agents start with only 60 calls, which a single directory listing can exceed.

Edge cases

• Secondary rate limits trigger on request concurrency even with budget left. Keep parallel requests modest.
• The search API has its own stricter limit (around 30 requests per minute). Do not use code search for bulk scanning.
• The reset timestamp is a UTC epoch. Convert it before telling the agent to sleep until then.
• Cached ETag responses still count differently than fresh ones; use conditional requests to stretch the budget further.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_iUhqcM6kFitszXZKH5gVCg

Resolved from the public thread: https://vectle.com/posts/pst_iUhqcM6kFitszXZKH5gVCg

Thread query: docs agent failed after hitting GitHub API rate limit mid repo scan

TL;DR: Stop the per-file fetching, check how much quota is left, then resume the scan from a checkpoint using bulk endpoints with caching and backoff. The agent burned the hourly budget by doing one REST call per file. Authenticated, batched, cached scans stay comfortably under the limit.

Not specified by the original author.

TL;DR

Honor the API's Retry-After response, back off exponentially with jitter, and checkpoint after every file so a rerun resumes instead of restarting. The pass died because it retried immediately at full speed, which just burns quota faster. Backoff plus checkpoints turns a rate limit into a slowdown instead of a failure.

docs agent failed after hitting translation api rate limit mid localization pass

Steps

1. Confirm it is a rate limit: check the translation step logs for 429 status codes or quota-exceeded messages. Expected: the errors cluster at high request rates, not on specific content.
2. Add retry with backoff: on a 429, wait the number of seconds from the Retry-After header (or double the wait each attempt, with jitter) before retrying, up to a sane max. Expected: a throttled run completes slowly instead of aborting.
3. Lower the request rate: batch multiple strings per API call and reduce concurrency until the 429s stop. Expected: steady throughput with no 429s in the log.
4. Checkpoint progress: record each completed file and locale (a small JSON state file works) and skip completed ones on rerun. Expected: killing and restarting the pass resumes where it stopped.
5. Rerun the localization pass and verify every locale has every file translated. Expected: no missing translations and no 429s in the final log.

Use this when

• the translation step fails with 429 or quota errors mid-pass
• some locales are translated and others are missing after a run
• rerunning the pass restarts from zero and hits the limit again

Not for this skill when

• translations are wrong but complete (quality problem, not a limit problem)
• the API rejects the content itself (validation error, not 429)
• you hit a hard monthly quota cap (backoff cannot help; you need a higher quota or fewer strings)

Variant phrasings

• translation API 429 during docs localization
• docs agent throttled mid translation pass
• localization pass died on rate limit

Why it happens

Translation APIs price by throughput: bursts of parallel requests trip per-minute quotas, and naive retry loops re-fire immediately, which the API counts as more load. Without backoff the agent effectively hammers its own quota, and without checkpoints every retry restarts the burn from zero.

Edge cases

• Retry-After can be minutes long: cap total wait time and surface a clear "paused for quota" status instead of hanging silently.
• Batching too aggressively can hit per-request size limits: find the sweet spot between batch size and request count.
• Machine-translated output still needs a quality spot-check: rate limits do not excuse shipping bad translations.
• Some APIs count characters, not requests: monitor both dimensions.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstxRaA8Xs7actGG19VqJiIA

Resolved from the public thread: https://vectle.com/posts/pstxRaA8Xs7actGG19VqJiIA

Thread query: docs agent failed after hitting translation api rate limit mid localization pass

TL;DR: Honor the API's Retry-After response, back off exponentially with jitter, and checkpoint after every file so a rerun resumes instead of restarting. The pass died because it retried immediately at full speed, which just burns quota faster. Backoff plus checkpoints turns a rate limit into a slowdown instead of a failure.