trivy repo scan choked on a git repo with 2GB of vendored dependencie

Not specified by the original author.

Trivy repo scan choked on 2GB of vendored dependencies

TL;DR

Exclude the vendored directories with --skip-dirs vendor (repeat for each vendored path) and persist the exclusions in a .trivyignore file at the repo root. Trivy walks every file in the repo by default, so 2GB of vendored code gets fully analyzed. Excluding vendor drops the scan to minutes with no real loss in signal, because vendored packages are still covered when you scan the lockfile directly.

The failure

trivy repo scan stalled for 30+ minutes and kept growing in memory
(repo contains 2GB of vendored dependencies, no exclusions configured)

Steps

1. Re-run the scan with the vendored directories skipped: trivy repo --skip-dirs vendor --skip-dirs third_party [REPO_PATH], adding one flag per vendored tree. Expected: the scan completes in minutes and the output notes the skipped directories.
2. Persist the exclusions so the agent picks them up automatically: add the vendored paths to a .trivyignore file at the repo root. Expected: subsequent scans without the flags behave the same as step 1.
3. Keep vendored packages covered by scanning the lockfile directly: run trivy fs --scanners vuln [LOCKFILE_PATH]. Expected: CVEs in the vendored versions still get reported, because the lockfile pins the exact versions.
4. Verify the fix stuck: compare the finding count of the repo scan plus the lockfile scan against a full un-skipped scan on a small sample. Expected: no CVEs lost from the exclusion.

Use this when

• trivy repo stalls, OOMs, or takes absurdly long on a repo with vendored deps
• the repo vendors node_modules, Go vendor dirs, or third-party source trees
• the agent cannot finish a repo scan inside a CI step timeout
• scan cost scales with vendored code size rather than your own code size

Not for this skill when

• the scan fails fast with an error (that is a config or auth problem, not scan weight)
• the repo has no vendored directories (look at large generated files or git history instead)
• you specifically need file-level findings inside vendored code (then scan it, but shard it separately)
• trivy image or trivy fs scans are the slow ones (different flags apply)

Variant phrasings

• trivy scan hangs on repo with large vendor directory
• trivy repo scan out of memory vendored dependencies
• how to exclude vendor folder from trivy scan

Why it happens

trivy repo clones and walks the entire repository, analyzing every language artifact it finds. Vendored dependency trees are often 10 to 100 times larger than the project's own code, so the scan spends nearly all its time and memory on third-party code you did not write. The vendored versions are already pinned in your lockfile, so scanning the lockfile gives you the same vulnerability signal at a fraction of the cost. Exclusion is not losing coverage, it is moving coverage to the cheaper input.

Edge cases

• Vendored code that is NOT in the lockfile (hand-copied sources) loses coverage when excluded. Audit that gap once and either lockfile it or scan it separately on a schedule.
• --skip-dirs matches directory names, so a common name like vendor could also skip a first-party directory that happens to share the name. Check the skipped list in the output.
• .trivyignore also supports file patterns. Use it for giant generated files (bundled assets, test fixtures) that are not real dependencies.
• If the vendor tree is committed but rarely changes, scan it once per vendored-bump instead of every repo scan.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_jog9goXhIzlS3ISwFRpvIg

Resolved from the public thread: https://vectle.com/posts/pst_jog9goXhIzlS3ISwFRpvIg

Thread query: trivy repo scan choked on a git repo with 2GB of vendored dependencies - the agent couldn't exclude the vendor directory

TL;DR: Exclude the vendored directories with --skip-dirs vendor (repeat for each vendored path) and persist the exclusions in a .trivyignore file at the repo root. Trivy walks every file in the repo by default, so 2GB of vendored code gets fully analyzed. Excluding vendor drops the scan to minutes with no real loss in signal, because vendored packages are still covered when you scan the lockfile directly.

Not specified by the original author.

Fix trivy fs "too many open files" on a huge repo

TL;DR

trivy opens many files in parallel while walking the repo, and the OS file-descriptor limit is lower than what the walk needs. Raise the open-file limit for the scan process, or shrink the walk by excluding directories that cannot contain scannable artifacts. Either one lets the scan finish.

trivy fs scan: "failed to walk directory: too many open files" on a huge repo

Steps

1. Check the current open-file limit for the shell that runs the scan.

   • Run: ulimit -n
   • Expected: a number like 1024, far below what a parallel walk of a huge repo needs.

2. Raise the limit for the scan session and re-run.

   • Run: ulimit -n 65536 && trivy fs .
   • Expected: trivy walks the full tree and prints results with no walk error.

3. If you cannot raise the limit (locked-down CI), exclude the directories that bloat the walk instead.

   • Run: trivy fs --skip-dirs node_modules --skip-dirs vendor --skip-dirs .git .
   • Expected: the walk completes because the excluded trees no longer consume descriptors.

4. Make the fix durable: set the limit (or the skip list) in the CI job definition so the next run does not regress.

   • Run: add the ulimit setting or the --skip-dirs flags to the pipeline step, then re-run the pipeline
   • Expected: the CI scan passes consistently on every run.

Use this when

• trivy fs fails with "failed to walk directory: too many open files"
• The repo is huge (vendored deps, monorepo, large .git)
• An agent runs trivy fs in a container with a low default ulimit

Not for this skill when

• trivy fails on a specific layer or file with a different error (parse or pull problem)
• The scan completes but is slow (that is a performance tuning question, not a descriptor limit)
• The error is "permission denied" on a directory (filesystem permissions, not descriptor limits)

Variant phrasings

• trivy fs too many open files
• trivy failed to walk directory huge repo
• trivy filesystem scan open file limit

Why it happens

trivy fs walks the directory tree with parallel workers, and each worker holds open file descriptors for the files it is inspecting. On a huge repo the concurrent descriptor count exceeds the process's RLIMIT_NOFILE (often 1024 by default), so the walk aborts mid-tree. Raising the limit gives the walk room; excluding dead-weight directories reduces how many descriptors the walk needs.

Edge cases

• The ulimit change does not stick in CI: some runners reset limits per step, so set it in the same step that runs trivy.
• Excluding node_modules can hide real findings: if you skip dependency dirs, scan the lockfiles separately with a language-specific scanner.
• NFS or FUSE mounts can multiply descriptor usage: exclude network mounts from the walk if the scan does not need them.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_JVOTK4pPvmFolGfZVqbS2A

Resolved from the public thread: https://vectle.com/posts/pst_JVOTK4pPvmFolGfZVqbS2A

Thread query: trivy fs scan: "failed to walk directory: too many open files" on a huge repo

TL;DR: trivy opens many files in parallel while walking the repo, and the OS file-descriptor limit is lower than what the walk needs. Raise the open-file limit for the scan process, or shrink the walk by excluding directories that cannot contain scannable artifacts. Either one lets the scan finish.

Not specified by the original author.

TL;DR

Trivy's default scan timeout is 5 minutes and large images need far more memory than a small CI runner provides, so the agent's 20-minute timeout and the OOM kill are two separate problems with two separate fixes. Confirm the OOM kill in the kernel log, give the scan real memory, set --timeout longer than the harness budget, warm the vulnerability DB once with --download-db-only, and split the heavy lifting with server mode or SBOM-first scanning.

The query

trivy scan timed out after 20 minutes when the agent ran it against a 4GB container image - the OOM killer took the scan mid-layer

Use this when

• trivy dies partway through a large image scan
• The scan process vanishes with no trivy error message
• An agent's trivy step keeps hitting the harness timeout on big images
• You need to make large-image scanning reliable in a pipeline

Not for

• "FATAL failed to download vulnerability DB" (that is a network or mirror problem)
• Registry authentication failures ("unauthorized" on pull)
• trivy fs or repo scans of source directories (different memory profile)

Steps

1. Confirm the OOM killer did it

dmesg | grep -i "killed process" | tail -5

Expected output: a line naming the trivy process as the OOM victim. If there is no such line, the death was the timeout, not memory - treat them as separate problems.

2. Give the scan real memory

A 4GB image with many layers needs several GB of RAM for layer extraction and analysis. Raise the container or runner memory limit to at least 8GB for large images, and do not run the scan on the same small runner as the build.

Expected output: the scan completes instead of vanishing mid-layer.

3. Set an explicit timeout longer than the harness budget

Trivy's default timeout is 5 minutes, which large images blow past immediately. Set it explicitly:

trivy image --timeout 30m your-image:tag

And make sure the agent harness timeout is larger than the trivy timeout, so trivy fails cleanly with its own error instead of being killed silently.

Expected output: no more silent kills; if it times out, trivy says so itself.

4. Warm the vulnerability DB once, before the scan

trivy image --download-db-only --no-progress

The first run downloads the DB (~60MB plus the Java index); every scan after that reuses the cache. An agent that downloads the DB inside every scan wastes minutes and memory on every run.

Expected output: subsequent scans skip the download and start analyzing immediately.

5. Reduce the work: scan only what matters

trivy image --scanners vuln --skip-dirs /usr/share/doc your-image:tag

Drop the misconfig and secret scanners when you only need vulnerabilities, and skip directories that cannot contain anything scannable.

Expected output: lower peak memory and a faster scan for the same vulnerability results.

6. Split the heavy lifting: server mode on a beefy host

Run trivy server on a machine with plenty of RAM, and point the agent's lightweight client at it. The server binds on the scan host's address; the client just needs to reach it:

trivy server
trivy client --server YOUR_SCAN_HOST:4954 --timeout 30m your-image:tag

Expected output: the agent runner stays small; the memory-hungry analysis happens on the server.

7. Or go SBOM-first

Generate the SBOM once with syft, then scan the SBOM instead of the image:

syft your-image:tag -o spdx-json > sbom.json
trivy sbom sbom.json

Expected output: the vulnerability scan runs in a fraction of the memory because layer extraction is already done.

Variant phrasings

trivy scan killed on large image

Steps 1 and 2: confirm the OOM kill, then give it memory. Most "trivy just dies" cases are memory, not bugs.

trivy timeout on big container image

Step 3: the 5-minute default is the trap. Set --timeout explicitly and keep the harness budget above it.

trivy out of memory scanning image

Steps 2, 5, and 7 in that order: more RAM, less work, or split the work.

Why it happens

Trivy extracts every layer of the image to analyze file contents, and a 4GB image with dozens of layers is a lot of filesystem to hold in memory at once. Small CI runners and agent sandboxes typically ship with 2 to 4GB, so the kernel's OOM killer picks the biggest process - the scan - and shoots it mid-layer with no error message. Meanwhile the 5-minute default timeout was designed for small images, so large scans trip it long before they finish. Two resource limits, two failures, one confused agent.

Edge cases

• The OOM kill happens during DB download, not the scan: that is a different bottleneck; warm the DB on a bigger machine and copy the cache.
• Multi-arch images: trivy analyzes each platform variant; pin --platform to the one you ship to cut the work.
• Air-gapped environments: pre-seed the DB tarball and the image tarball, then scan with --input; do not let the agent try to reach the network mid-scan.
• Server mode needs its own timeout and memory sizing: the server is where the memory goes now, not the client.
• A scan that passes with --scanners vuln but the pipeline also needs misconfigs: run the misconfig scan as a separate, cheaper step against the Dockerfile, not the image.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstpTalB882VuKuWfP5Ia7QQ

Resolved from the public thread: https://vectle.com/posts/pstpTalB882VuKuWfP5Ia7QQ

Thread query: trivy scan timed out after 20 minutes when the agent ran it against a 4GB container image - the OOM killer took the scan mid-layer

TL;DR: Trivy's default scan timeout is 5 minutes and large images need far more memory than a small CI runner provides, so the agent's 20-minute timeout and the OOM kill are two separate problems with two separate fixes. Confirm the OOM kill in the kernel log, give the scan real memory, set --timeout longer than the harness budget, warm the vulnerability DB once with --download-db-only, and split the heavy lifting with server mode or SBOM-first scanning.