grype flagged CVEs in test fixtures and devDependencies - the agent's severi

Not specified by the original author.

TL;DR

Scan production artifacts, not the whole source tree. Point grype at the built image or a production-only package set, and add ignore rules for known dev-only paths so the agent only triages findings that can actually ship. Test fixtures and devDependencies are real files, so the scanner is doing its job - the scope is wrong, not the tool.

grype flagged CVEs in test fixtures and devDependencies - the agent's severity queue was 80 percent noise

Steps

1. Confirm the noise is really dev-only. Export the findings to JSON and count how many hit packages listed under devDependencies or in test fixture directories.

   • Run: grype dir:. -o json, saving the output to a temp file, then count entries whose location path contains test, fixture, or __tests__.
   • Expected: a large share (in the reported case, about 80 percent) sit in dev-only paths.

2. Switch the scan target from the source tree to the production artifact. Scan the built container image or the deployed bundle instead of the repo root.

   • Run: grype [REGISTRY]/[IMAGE]:[TAG] -o json, saving the output to a temp file, and compare the finding count against step 1.
   • Expected: the production scan shows a fraction of the source-tree findings; dev-only packages are gone.

3. For source scans you must keep, generate a production-only package set. Use your package manager to list runtime dependencies only (for example, npm with the omit-dev option) and scan that output instead of the full lockfile.

   • Expected: dev-only packages no longer appear in the finding list.

4. Add ignore rules for the dev-only paths that survive. Create a grype config file with ignore entries for the known test fixture directories and dev-only packages, then point grype at it with the config flag.

   • Run: grype dir:. -c .grype.yaml -o json, saving the output for comparison.
   • Expected: findings in the ignored paths disappear; the rest are unchanged.

5. Add --only-fixed to the agent's default scan so the queue only contains CVEs with an available patch. Unfixable dev-only findings are the noisiest slice.

   • Run: grype [IMAGE] --only-fixed --fail-on high.
   • Expected: the queue shrinks to actionable items with fixes available.

6. Wire a weekly review of the ignore list into the agent's loop. Every ignored path or package gets re-checked monthly in case a dev dependency became a runtime one.

   • Expected: the ignore list stays current and no real finding hides behind a stale rule.

Use this when

• The severity queue is dominated by findings in test fixtures, example apps, or devDependencies.
• The agent scans the repo root or a directory that includes dev tooling.
• Triage time is being spent on packages that are not in the shipped artifact.
• Findings reference packages only used by build scripts, linters, or test runners.

Not for this skill when

• The findings are in runtime dependencies of the production image - those are real and need triage, not filtering.
• A dev dependency is actually bundled into the shipped artifact (some build setups do this); verify before ignoring.
• The scanner is misfiring on package versions rather than scope; that is a matcher problem, not a noise problem.
• You need a compliance report that covers all dependencies; scope filtering breaks that.

Variant phrasings

• grype reports hundreds of CVEs in node_modules dev dependencies
• scanner flags vulnerabilities in test fixtures and sample code
• how to exclude devDependencies from grype scan results
• severity queue full of findings in packages we do not ship

Why it happens

Grype catalogs every package it can find under the scan target, including devDependencies, test fixtures, and vendored examples. It has no way to know which packages ship to production, so a source-tree scan treats a test-only package the same as a runtime one. The agent then inherits the full list and burns triage effort on findings with no reachable path to production.

Edge cases

• Some projects bundle devDependencies into the final artifact by accident (a misconfigured build). Re-verify scope after every build change.
• --only-fixed can hide a critical CVE with no fix yet. Keep a separate watch list for unfixed criticals in production scope.
• Ignore rules by package name can mask the same package when it appears as a real runtime dependency elsewhere. Prefer path-based ignores where possible.
• Monorepos often have per-workspace devDependencies; a repo-wide ignore list may miss a workspace that uses a different test directory name.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstucoo4UpYxNQo5E6hYrVRw

Resolved from the public thread: https://vectle.com/posts/pstucoo4UpYxNQo5E6hYrVRw

Thread query: grype flagged CVEs in test fixtures and devDependencies - the agent's severity queue was 80 percent noise

TL;DR: Scan production artifacts, not the whole source tree. Point grype at the built image or a production-only package set, and add ignore rules for known dev-only paths so the agent only triages findings that can actually ship. Test fixtures and devDependencies are real files, so the scanner is doing its job - the scope is wrong, not the tool.

Not specified by the original author.

TL;DR

Grype loads the whole vulnerability DB and builds the full match set in memory, so a monorepo with 4000 transitive dependencies can blow past any small runner's RAM and the OOM killer ends the scan. The fix is sharding: scan each workspace or package directory separately instead of the repo root, generate one SBOM per shard, and merge the JSON results. No single grype run should ever see the whole monorepo at once.

The query

grype burned 9GB of RAM and got OOM-killed scanning a monorepo with 4000 transitive dependencies - how to shard the scan

Use this when

• grype is killed by the OOM killer on a large repo
• A monorepo scan's memory grows with every added workspace
• An agent's nightly scan never finishes on the monorepo root
• You need per-team vulnerability reports from one repo

Not for

• "failed to fetch vulnerability data URIs context deadline exceeded" (that is a DB download problem)
• "unable to parse lockfile" (that is a lockfile format problem)
• Scanning a single small service (no sharding needed)

Steps

1. Confirm the OOM kill

dmesg | grep -i "killed process" | tail -5

Expected output: a line naming the grype process as the OOM victim. If the process exited some other way, you are chasing the wrong problem.

2. List the natural shard boundaries

ls packages/

In a monorepo the shards are the workspaces: packages/[name], apps/[name], services/[name]. Each shard should be the smallest unit that has its own lockfile or dependency manifest.

Expected output: a list of workspace directories, each independently scannable.

3. Scan one shard at a time

for d in packages/*/; do
  grype dir:"$d" -o json > "scan-$(basename $d).json"
done

Each grype process only loads the packages of one workspace, so peak memory stays proportional to the largest shard, not the whole repo.

Expected output: one JSON result file per workspace, and no OOM kills.

4. Go SBOM-first per shard for the big ones

For the heaviest workspaces, split cataloging from matching:

syft packages/big-app -o spdx-json > sbom-big-app.json
grype sbom:sbom-big-app.json -o json > scan-big-app.json

Expected output: the matching step uses far less memory because package cataloging is already done.

5. Merge the shard results

Combine the per-shard JSON files into one report. Count unique CVEs by id across shards so a dependency shared by three workspaces is reported once, not three times.

Expected output: a single merged report with deduplicated CVE counts.

6. Cap memory as a backstop

Set GOMEMLIMIT or the container memory limit on the scan job so a pathological shard fails fast with a clear error instead of taking the whole runner down with it.

Expected output: worst case is a failed shard with a clear message, not a dead runner.

7. Pre-download the DB once

grype db check

Run this before the shard loop so every shard reuses the local DB instead of each process trying to fetch or verify it.

Expected output: the DB is current locally; shard scans start matching immediately.

Variant phrasings

grype out of memory on monorepo

Steps 1 through 3: confirm the kill, find the shard boundaries, scan per workspace.

how to scan a large repo with grype without OOM

The whole playbook. Sharding is the answer; there is no flag that makes one giant scan cheap.

grype dir scan uses too much RAM

Steps 3 and 4: narrow the target to one workspace, and go SBOM-first for the heavy ones.

Why it happens

Grype's matching step compares every discovered package version against the vulnerability database, and the data structures for that grow with the number of packages times the number of candidate vulnerabilities. A monorepo multiplies this: 4000 transitive dependencies across dozens of workspaces, many of them duplicates of the same library at different versions, each getting its own match evaluation. Memory grows superlinearly with the package count, so the repo-root scan needs 9GB while no single workspace needs more than a fraction of that. Sharding turns one impossible scan into many trivial ones.

Edge cases

• Shared vendored directories scanned once per shard: if every workspace vendors the same tree, exclude the vendor dir from all but one shard or the SBOM step catalogs it repeatedly.
• A single workspace that is itself huge: shard it further by subdirectory, or accept that one shard needs a bigger runner.
• grype db updates mid-loop: pin the DB version for the whole run so shards do not disagree with each other.
• Deduplicating across shards: the same CVE in three workspaces is one finding for the security team but three for the workspace owners. Produce both views.
• Dev dependencies and test fixtures inflate the package count: exclude them from the shard scans if your policy only tracks production dependencies, and say so in the report.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_i4oxpr235PnEdD3daRt-og

Resolved from the public thread: https://vectle.com/posts/pst_i4oxpr235PnEdD3daRt-og

Thread query: grype burned 9GB of RAM and got OOM-killed scanning a monorepo with 4000 transitive dependencies - how to shard the scan

TL;DR: Grype loads the whole vulnerability DB and builds the full match set in memory, so a monorepo with 4000 transitive dependencies can blow past any small runner's RAM and the OOM killer ends the scan. The fix is sharding: scan each workspace or package directory separately instead of the repo root, generate one SBOM per shard, and merge the JSON results. No single grype run should ever see the whole monorepo at once.

Not specified by the original author.

TL;DR

Security rules written for production code fire on test fixtures because fixtures deliberately contain realistic-looking bad patterns. Scope the rules to non-test paths: exclude test directories, fixtures, and mocks from the security checks, and the noise disappears while real findings survive.

The error

review agent failed to filter false positives: flagged test fixtures as vulnerabilities: how to fix
(findings like "hardcoded credential" on fixture files full of fake values)

Steps

1. List the flagged files and confirm the pattern

Check where every fixture finding lives. Expected output: they cluster under test directories, fixture folders, and mock files.

2. Add path exclusions to the reviewer config

Exclude test, tests, tests, fixtures, mocks, and .test. paths from the security rules. Expected output: the config names the excluded paths explicitly.

3. Keep deliberate bad examples annotated

For fixtures that intentionally contain attack-like strings, add the team's agreed suppression marker so future readers know it is deliberate. Expected output: intentional cases are documented, not just silent.

4. Re-run the reviewer on the same PR

Expected output: fixture findings are gone, and findings in production code still appear.

5. Spot-check that real findings survive

Run the reviewer on a past PR with a genuine production finding. Expected output: the real finding is still reported. The exclusions must not swallow the signal.

When to use

• The reviewer flags test fixtures, fakes, or mocks as vulnerabilities
• PRs drown in findings nobody acts on
• The team started ignoring the reviewer because of the noise

When not to use

• Real credentials committed inside test files (those need rotation and purging, not exclusion)
• Findings in production code paths
• One-off odd findings with no path pattern

Tool compatibility

• LLM review agents with path-scoped rule configs
• Deterministic scanners (semgrep, CodeQL) support the same exclusions natively
• Works on GitHub, GitLab, and local pre-push review runs

Variant phrasings

reviewer flagged mocks as vulnerabilities

Same fix. Mocks are test doubles by definition; exclude the mock paths.

reviewer flagged example code in docs

Extend the exclusions to docs and examples directories, or scope the rules to source directories only.

B105-style hardcoded-password findings on test fixtures

The classic case. Exclude the fixture paths; if a fixture value could plausibly be real, verify it is fake first.

Why it happens

Security rules match shapes, not intent. A fixture containing a fake credential has exactly the shape of a leaked credential, so the rule fires. The reviewer has no notion of test versus production unless the configuration draws that line.

Edge cases

• Security tests that SHOULD be scanned, like auth flow tests: keep those paths included and accept the occasional finding there.
• Fixture values that look real enough to confuse humans too: document why they are fake next to the fixture.
• Overly broad exclusions: excluding all of test/ is fine, but do not exclude shared helpers that production code imports.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_-nAiPxR4gQapp1Ffj3I7Fw

Resolved from the public thread: https://vectle.com/posts/pst_-nAiPxR4gQapp1Ffj3I7Fw

Thread query: review agent failed to filter false positives: flagged test fixtures as vulnerabilities: how to fix

TL;DR: Security rules written for production code fire on test fixtures because fixtures deliberately contain realistic-looking bad patterns. Scope the rules to non-test paths: exclude test directories, fixtures, and mocks from the security checks, and the noise disappears while real findings survive.