trivy fs scan: "failed to walk directory: too many open files" on a huge repo

Fix trivy fs "too many open files" on a huge repo

TL;DR

trivy opens many files in parallel while walking the repo, and the OS file-descriptor limit is lower than what the walk needs. Raise the open-file limit for the scan process, or shrink the walk by excluding directories that cannot contain scannable artifacts. Either one lets the scan finish.

trivy fs scan: "failed to walk directory: too many open files" on a huge repo

Steps

1. Check the current open-file limit for the shell that runs the scan.

   • Run: ulimit -n
   • Expected: a number like 1024, far below what a parallel walk of a huge repo needs.

2. Raise the limit for the scan session and re-run.

   • Run: ulimit -n 65536 && trivy fs .
   • Expected: trivy walks the full tree and prints results with no walk error.

3. If you cannot raise the limit (locked-down CI), exclude the directories that bloat the walk instead.

   • Run: trivy fs --skip-dirs node_modules --skip-dirs vendor --skip-dirs .git .
   • Expected: the walk completes because the excluded trees no longer consume descriptors.

4. Make the fix durable: set the limit (or the skip list) in the CI job definition so the next run does not regress.

   • Run: add the ulimit setting or the --skip-dirs flags to the pipeline step, then re-run the pipeline
   • Expected: the CI scan passes consistently on every run.

Use this when

• trivy fs fails with "failed to walk directory: too many open files"
• The repo is huge (vendored deps, monorepo, large .git)
• An agent runs trivy fs in a container with a low default ulimit

Not for this skill when

• trivy fails on a specific layer or file with a different error (parse or pull problem)
• The scan completes but is slow (that is a performance tuning question, not a descriptor limit)
• The error is "permission denied" on a directory (filesystem permissions, not descriptor limits)

Variant phrasings

• trivy fs too many open files
• trivy failed to walk directory huge repo
• trivy filesystem scan open file limit

Why it happens

trivy fs walks the directory tree with parallel workers, and each worker holds open file descriptors for the files it is inspecting. On a huge repo the concurrent descriptor count exceeds the process's RLIMIT_NOFILE (often 1024 by default), so the walk aborts mid-tree. Raising the limit gives the walk room; excluding dead-weight directories reduces how many descriptors the walk needs.

Edge cases

• The ulimit change does not stick in CI: some runners reset limits per step, so set it in the same step that runs trivy.
• Excluding node_modules can hide real findings: if you skip dependency dirs, scan the lockfiles separately with a language-specific scanner.
• NFS or FUSE mounts can multiply descriptor usage: exclude network mounts from the walk if the scan does not need them.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_JVOTK4pPvmFolGfZVqbS2A