# docs agent failed after hitting GitHub API rate limit mid repo scan

## TL;DR
Stop the per-file fetching, check how much quota is left, then resume the scan from a checkpoint using bulk endpoints with caching and backoff. The agent burned the hourly budget by doing one REST call per file. Authenticated, batched, cached scans stay comfortably under the limit.

## The error

```text
docs agent failed after hitting GitHub API rate limit mid repo scan
```

## Steps

1. Check the remaining budget with the same credential the agent uses:

```text
curl -s https://api.github.com/rate_limit
```

Expected: the response shows core remaining count and the reset time. If remaining is near zero, the scan has to wait or get smarter.

2. Authenticate the scan if it is not already. Unauthenticated calls get 60 per hour; a credential gets 5,000.

Expected: the remaining count jumps after auth, and the reset window is the same for the whole hour.

3. Replace per-file fetches with bulk endpoints. One recursive git trees call or a single tarball download replaces hundreds of individual contents calls.

Expected: request count drops by 10 to 100 times for the same repo coverage.

4. Add checkpointing and backoff. Write each scanned path to a checkpoint file as you go and skip recorded paths on resume; on a 403 or 429, back off exponentially instead of retrying hot.

Expected: a second run resumes where the first died and finishes without hitting the limit.

## Use this when

- an agent repo scan dies partway with 403 rate limit exceeded
- the scan fetches files one REST call at a time over a large repo
- reruns keep failing at roughly the same progress point

## Not for this skill when

- the failure is a 401 auth error (the credential is wrong or missing, different problem)
- the limit is on a different API, not GitHub
- the scan completes but is merely slow

## Variant phrasings

### github api 403 rate limit exceeded during scan
Check whether the calls are authenticated first; that alone is an 80x budget increase.

### agent exceeded api quota scanning repo
Bulk-fetch the tree once, then work from the local copy instead of the API.

### secondary rate limit on github during automation
That is the concurrency limit, not the hourly budget. Reduce parallel requests even if quota remains.

## Why it happens
Naive scans do one REST call per file and GitHub counts every one. Big repos have thousands of files, so the hourly budget evaporates fast. Unauthenticated agents start with only 60 calls, which a single directory listing can exceed.

## Edge cases

- Secondary rate limits trigger on request concurrency even with budget left. Keep parallel requests modest.
- The search API has its own stricter limit (around 30 requests per minute). Do not use code search for bulk scanning.
- The reset timestamp is a UTC epoch. Convert it before telling the agent to sleep until then.
- Cached ETag responses still count differently than fresh ones; use conditional requests to stretch the budget further.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_iUhqcM6kFitszXZKH5gVCg