the agent closed 200 CVEs as "fixed" because the new base image changed the package manager - the scanner couldn't...

TL;DR

"Not found by the scanner" is not "fixed". The new base image switched package managers, the scanner lost visibility into the packages, and the agent celebrated. Reopen the tickets, require positive evidence of the fixed version in a rescan before any closure, and alert whenever a finding count drops sharply between scans.

the agent closed 200 CVEs as "fixed" because the new base image changed the package manager - the scanner couldn't see the packages anymore

Use this when

• Hundreds of findings closed in one run with no corresponding package bumps.
• A base-image or distro change coincided with a finding-count cliff.
• The scanner's package inventory shrank between two scans of "the same" image.

Not for this skill when

• Packages were actually upgraded to fixed versions and the rescan confirms it. That is a real fix; close away.
• The image was decommissioned. Then the right action is closing as "asset retired", not "fixed".

Steps

1. Reopen the 200 tickets with a note: "closed on scanner-visibility loss, not on fix evidence". Verify: the backlog reflects the true open count again.
2. Compare the scanner's package inventory before and after the base-image change. Verify: you can point to the package manager switch that blinded the scanner.
3. Fix scanner coverage for the new package manager (update the scanner, add the new lockfile paths, or scan the image instead of the repo). Verify: the package inventory is back to the expected size.
4. Set the closure rule: a CVE closes only when a rescan shows the package present at a fixed version. "Absent from results" keeps it open. Verify: the rule is encoded in the agent's close-finding tool, not just in a doc.
5. Add a guardrail: if closed-findings spike or total findings drop more than 30 percent between consecutive scans, block auto-close and alert a human. Verify: replay the bad run against the guardrail and confirm it would have fired.

Variant phrasings

CVEs closed after base image change

The short phrasing.

scanner missed packages new base image

The visibility phrasing.

findings vanished after distro switch

The symptom search.

Compatibility: any scanner with package-manager-specific cataloging (Trivy, Grype, Snyk); the positive-evidence rule is scanner-independent.

Why it happens

The agent's close logic was "finding absent from latest scan implies fixed". That logic is only valid if scanner visibility is constant. A base-image change silently violated the precondition, and the agent had no check for it, so absence of evidence became evidence of absence at scale.

Edge cases / pitfalls

• Partial visibility loss is sneakier than total: spot-check package counts per image on every scanner or base-image upgrade.
• If the new base image genuinely removed the vulnerable packages, that IS a fix, but it still needs the positive evidence: show the package gone from the inventory diff, not just the CVE gone from results.
• Do not let the agent "fix" coverage by excluding the new package manager from scans; that trades one blindness for another.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_VE4OQEC2GbN6XsWgJx5vhw