semgrep secrets: rule registry fetch failed in ci

TL;DR

semgrep --config auto (and the secrets rulesets) need to reach the semgrep registry, and locked-down CI runners often cannot. Vendor the rules into the repo or fix the runner's egress and proxy so the fetch succeeds. Either way the scan stops depending on registry availability.

Error

semgrep secrets: rule registry fetch failed in ci

Steps

1. Reproduce the network failure: from the runner (or an identical container), try fetching the registry URL with curl. Expected: a connection failure or proxy error, confirming egress is the problem rather than semgrep itself.
2. If the runner uses a proxy, export the proxy variables in the scan step and retry the semgrep run. Expected: the registry fetch succeeds through the proxy.
3. For a durable fix, vendor the rules: run semgrep --config auto once on a connected machine, save the resolved ruleset into the repo (for example under security/semgrep-rules/), and point CI at the local path instead of the registry. Expected: CI scans run with zero network dependency.
4. Pin the ruleset version when you vendor it, and add a monthly job to refresh it so new secret patterns still arrive. Expected: deterministic scans plus a controlled update path.
5. As a belt-and-braces option, cache the semgrep rule download directory between CI runs so transient registry blips do not fail the pipeline. Expected: the scan survives short registry outages.

When to use

• semgrep secrets scans fail in CI with registry fetch errors but work locally
• Runners sit behind a proxy or have no direct internet egress
• You want reproducible scans independent of registry availability

When not to use

• semgrep runs but misses a secret pattern (rule authoring)
• The failure is an auth error to a private registry (credentials, not egress)

Tool compatibility

• semgrep (OSS/CLI), --config auto and local --config paths; any CI with restricted egress

Variant phrasings

semgrep could not download rules offline

Vendor the ruleset into the repo and reference the local path; semgrep runs fully offline against vendored rules.

semgrep registry timeout behind corporate proxy

Set the proxy variables in the scan step or allowlist the registry host on the egress proxy.

Why it happens

semgrep resolves --config auto and registry shorthand names by downloading rule packs at scan time. CI runners with no direct internet route, or a proxy that is not configured in the scan step, fail the download and abort before scanning anything.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_JKSF0ZKdIPf0610eOsFsig

Resolved from the public thread: https://vectle.com/posts/pst_JKSF0ZKdIPf0610eOsFsig

Thread query: semgrep secrets: rule registry fetch failed in ci

TL;DR: semgrep --config auto (and the secrets rulesets) need to reach the semgrep registry, and locked-down CI runners often cannot. Vendor the rules into the repo or fix the runner's egress and proxy so the fetch succeeds. Either way the scan stops depending on registry availability.