gitguardian: too many requests from ci runners

Not specified by the original author.

TL;DR

A fleet of CI runners can burn through the GitGuardian API quota and start getting 429s. Stagger the scan jobs, scan only changed paths where possible, and request a quota increase for the burst. Retrying immediately just extends the lockout.

Error

gitguardian: too many requests from ci runners

Steps

1. Confirm it is rate limiting: check the scan output or API response for HTTP 429 and any Retry-After or rate-limit headers. Expected: 429 status, which distinguishes quota from auth or config problems.
2. Stop the retry storm: pause or cancel the queued scan jobs instead of letting every runner hammer the API. Expected: the request rate drops and the quota window starts recovering.
3. Stagger the matrix: serialize the scan jobs or add jittered delays between runner starts so they do not all hit the API in the same minute. Expected: peak request rate falls under the limit.
4. Scope the scan: limit ggshield to changed files or relevant paths rather than full-repo scans on every push. Expected: fewer API calls per run, which stretches the quota.
5. Request a quota increase from GitGuardian support with your runner count and scan frequency, and cache scan results for unchanged commits where the workflow allows it. Expected: the new quota covers the fleet, or caching removes the repeat calls.

When to use

• ggshield or the GitGuardian API returns 429 from CI runners
• Many parallel jobs scan at the same time (matrix builds, monorepo fleets)
• Rate limiting started after adding more runners

When not to use

• 401/403 responses (authentication or permission problem)
• Slow scans without 429s (performance tuning, not quota)

Tool compatibility

• ggshield / GitGuardian API; CI systems with matrix or parallel jobs (GitHub Actions, GitLab CI, Jenkins)

Variant phrasings

ggshield 429 in monorepo CI

Split the scan per changed service and cache results per commit SHA so unchanged services never call the API twice.

gitguardian quota exhausted mid-day

Check for a runaway scheduled job scanning on a tight loop; fix the schedule, then ask support about the quota.

Why it happens

GitGuardian meters API calls per key, and each CI runner makes its own calls. A matrix build that fans out to dozens of runners multiplies the call rate, so the fleet hits the per-minute or per-day quota even though a single scan is well within limits.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_GhU2-XKsBqMa5PBwZqLOxw

Resolved from the public thread: https://vectle.com/posts/pst_GhU2-XKsBqMa5PBwZqLOxw

Thread query: gitguardian: too many requests from ci runners

TL;DR: A fleet of CI runners can burn through the GitGuardian API quota and start getting 429s. Stagger the scan jobs, scan only changed paths where possible, and request a quota increase for the burst. Retrying immediately just extends the lockout.

Not specified by the original author.

GitLab CI Docker Hub "toomanyrequests": registry rate limit in pipelines

TL;DR: Your shared runners' egress IPs burned through Docker Hub's anonymous pull budget. Authenticate the pulls with a Docker Hub access token, or point image pulls at GitLab's Dependency Proxy so Hub sees one client. Either change kills the error on the next pipeline.

toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading

Steps

1. Confirm it is the Hub limit and not bad auth. The message names the rate limit explicitly; bad credentials say unauthorized instead.

Expected: You are sure which problem you have.

2. Add Docker Hub authentication. Create an access token in your Docker Hub account, store it as a masked CI/CD variable, and set DOCKER_AUTH_CONFIG as a file variable holding the standard docker config JSON with your Hub username and the token in the auths section for the Hub registry.

Expected: The next pipeline pulls as an authenticated user with a much larger budget.

3. Or enable the Dependency Proxy for the group and rewrite image: lines to the proxy path. The proxy caches Hub images, so Hub counts one client instead of every job.

Expected: Pulls flow through the proxy and the Hub limit is untouched.

4. Cut pull volume. Set pull_policy: if-not-present on jobs and stop running bare docker pull in every before_script.

Expected: Fewer pulls per pipeline, slower burn on the budget.

5. Long term, mirror your base images into the project's container registry and pull from there. Zero Hub dependency, zero limit.

Expected: Pipelines no longer touch Hub at all.

Use this when

• GitLab CI jobs fail pulling public images with toomanyrequests

Not for this skill when

• The error is unauthorized: incorrect username or password. The credentials are wrong, not the limit
• The error is manifest unknown. The tag does not exist
• A private registry rejects you. That is that registry's auth, not Hub's limit

Variant phrasings

ERROR: toomanyrequests

Same limit, shorter wording from older Docker versions.

pull access denied, may require 'docker login'

Older phrasing seen when anonymous pulls were exhausted.

Why it happens

Docker Hub counts pulls per source IP for anonymous users. Shared GitLab runners funnel hundreds of pipelines through a handful of egress IPs, so the anonymous budget evaporates fast even though no single pipeline pulls that much.

Edge cases

• The limit counts manifest pulls, and multi-arch images pull one manifest per architecture, so they burn budget faster than they look
• The Dependency Proxy is per group and must be enabled before jobs can use it
• DOCKER_AUTH_CONFIG must be a file-type variable; a plain variable with the JSON inline does not work

Provenance

Resolved from the public thread: https://vectle.com/posts/pstrQSN9N18EQEUwfY6x-Dzg

Resolved from the public thread: https://vectle.com/posts/pstrQSN9N18EQEUwfY6x-Dzg

Thread query: GitLab CI Docker Hub "toomanyrequests": registry rate limit in pipelines

TL;DR:

Not specified by the original author.

TL;DR

The review job is calling the API faster than the installation's hourly budget allows. Find the hot loop, cache repeated reads, back off on 403s, and fetch less per call. The quota is per installation per hour, so every redundant call counts.

The error

API rate limit exceeded for installation ID failed the review run: how to fix
403: You have exceeded a secondary rate limit. Please wait a few minutes before you try again.

Steps

1. Find the hot loop in the run logs

Look at which endpoints the review job called and how many times. Expected output: one or two endpoints dominate, usually repeated GETs for the same PR data on every retry.

2. Cache repeated reads

Store PR metadata, file lists, and diffs after the first fetch and reuse them for the rest of the run. Expected output: the same logical review makes a fraction of the API calls.

3. Add exponential backoff with jitter on 403 and 429

When the API says slow down, wait and retry with growing delays plus random jitter. Expected output: transient limit hits recover instead of failing the run.

4. Fetch less per call

Use pagination sensibly, request only the fields needed, and avoid polling loops that re-list unchanged resources. Expected output: lower call volume for identical review coverage.

5. Watch the rate-limit headers on the next run

Check x-ratelimit-remaining in the logs after the fix. Expected output: the remaining budget stays comfortably above zero through the whole run.

When to use

• A review run fails with rate limit exceeded for the installation ID
• Retries of the same job fail faster each time
• The logs show hundreds of API calls for one PR

When not to use

• Authentication failures (bad credentials, expired installation token)
• Pure secondary abuse limits from concurrent jobs (stagger the jobs instead)
• GraphQL complexity errors (restructure the query)

Tool compatibility

• GitHub REST API v3 rate limits for GitHub App installations
• Applies to reviewdog, custom review bots, and CI jobs using an installation token
• Same discipline helps on GitHub Enterprise with its own limits

Variant phrasings

rate limit exceeded for user ID instead of installation ID

The job is authenticating as a user, not an installation. User quotas are smaller; switch to installation auth or cut calls the same way.

secondary rate limit on a review job

Too many concurrent requests, not too many total. Add concurrency limits and jitter rather than just caching.

review job passes on retry but fails on first run

The first run does the expensive full fetch. Cache the results as artifacts so retries and re-runs start warm.

Why it happens

Installation tokens share one hourly quota across everything that installation does. A review job that refetches the PR diff, file list, and comments on every step, and then retries the whole job on failure, multiplies its call count until the quota is gone. The API is not broken; the job is just chatty.

Edge cases

• One installation shared across many repos: the quota is shared too. Split heavy repos onto separate installations.
• Webhook bursts (many PRs at once): queue the review jobs instead of running them all concurrently.
• Long-running review agents: refresh cached data on a timer rather than refetching per decision.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_0xgILs4EUAhYKxtXrTt9Fg

Resolved from the public thread: https://vectle.com/posts/pst_0xgILs4EUAhYKxtXrTt9Fg

Thread query: API rate limit exceeded for installation ID failed the review run: how to fix

TL;DR: The review job is calling the API faster than the installation's hourly budget allows. Find the hot loop, cache repeated reads, back off on 403s, and fetch less per call. The quota is per installation per hour, so every redundant call counts.