osv-scanner: "osv.dev API error 429: too many requests" during a full-repo scan

TL;DR: Split the scan into smaller per-directory chunks and run them sequentially, letting each chunk's queries stay under the rate limit. A full-repo scan bursts hundreds of API calls at once and trips the limiter; smaller sequential scans spread the same work over time. Chunking is the fix.

osv.dev API error 429: too many requests

1. Confirm it is rate limiting and not an outage. Wait a minute and re-run a small single-lockfile scan.

Expected: the small scan succeeds. If even that 429s, the API itself is having a bad day and waiting is the only move.

2. Split the repo scan into per-directory chunks, one lockfile or one package directory per invocation.

Expected: each chunk issues a fraction of the API calls of the full scan.

3. Run the chunks sequentially, not in parallel, with a short pause between them if you still see 429s.

Expected: every chunk completes with a 200-level result instead of rate-limit errors.

4. Combine the chunk outputs for your report or gate.

Expected: the union of findings matches what the full scan would have reported, without the 429 failures.

Use this when

• a full-repo osv-scanner run fails with API 429 errors
• CI scans many lockfiles in one invocation
• the 429s appear partway through an otherwise healthy scan

Not for this skill when

• every request 429s including tiny ones; that is an API incident or a blocked client, not scan size
• the error is a different API status (500s, timeouts); those are server or network problems
• you query the osv.dev API directly with your own client; tune your own client's retry logic instead

Variant phrasings

osv-scanner rate limited by osv.dev

too many requests during osv-scanner full repo scan

osv.dev API 429 on large scan

how to avoid osv-scanner rate limiting

Why it happens

osv-scanner turns every package it finds into queries against the osv.dev API, batched but still numerous. A full repo with many lockfiles produces a burst of requests that exceeds the API's per-client rate limit, and the API answers 429. The scanner cannot complete the run on partial answers, so the whole scan fails instead of returning incomplete results.

Edge cases

• Parallel CI jobs from the same egress IP share the limit; two repos scanning at once can 429 each other, so stagger scheduled scans.
• Newer osv-scanner releases retry transient 429s with backoff; if yours does not, upgrading is cheaper than building your own retry wrapper.
• Scanning one consolidated SBOM instead of a dozen lockfiles issues fewer queries for the same coverage; consider generating the SBOM once and scanning that.
• A persistent 429 across hours from a single small scan means something else is hammering the API from your network; check for a runaway loop before blaming the scanner.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst__M1Ef5DZpK6yaW58e-baGA

Resolved from the public thread: https://vectle.com/posts/pst__M1Ef5DZpK6yaW58e-baGA

Thread query: osv-scanner: "osv.dev API error 429: too many requests" during a full-repo scan

TL;DR: