the agent keyed tickets on CVE plus image tag, then the tag got rebuilt - every finding re-filed as new overnight

Findings re-filed as new overnight after the image tag was rebuilt

TL;DR

Key findings on the image digest (the sha256 content hash), not the tag. Tags are mutable pointers: every rebuild moves :latest to new content, so a CVE-plus-tag key looks brand new after each rebuild even when nothing vulnerable changed. Digests are content-addressed, so rebuilds of unchanged images match history and stay quiet. Keep the tag as a human label, never as key material.

The failure

every finding re-filed as new overnight after the image tag was rebuilt
(dedup key was CVE plus mutable tag, digest unchanged, hundreds of duplicate tickets)

Steps

1. Resolve each scanned tag to its digest at scan time and store the digest in the finding key instead of the tag. Expected: rescanning the same content under a rebuilt tag produces the identical key.

1. Backfill the overnight flood: for the re-filed findings, match on CVE plus package plus digest and merge the duplicates into the original tickets. Expected: the duplicate wave collapses back to the pre-rebuild ticket set.

1. Keep the tag as a display label on the ticket for humans, but assert in code that the key builder never reads it. Expected: a regression test that rebuilds a tag and asserts key stability passes.

1. Add a guardrail: if new-finding volume spikes beyond a threshold in one run, hold the tickets for review instead of auto-filing. Expected: the next key instability pages someone instead of flooding the tracker.

Use this when

• all findings re-file as new after a scheduled image rebuild
• the dedup key includes a tag, branch name, or any mutable reference
• nightly rebuild pipelines cause periodic ticket floods
• :latest or rolling tags are scanned in a fleet

Not for this skill when

• the digest actually changed and the CVE list is genuinely different (those are real new findings)
• re-filing comes from feed description edits (fix the key to exclude mutable text)
• the scanner changed version string formats (normalize versions before hashing)
• findings duplicate across different images (that is an identifier normalization problem)

Variant phrasings

• image tag rebuilt all vulnerabilities re-reported as new
• dedup on image tag causes duplicate findings after rebuild
• container scan tickets duplicated every night rolling tag

Why it happens

A tag like :latest is a pointer that the registry repoints on every build. The dedup key treated the pointer as identity, so when the nightly rebuild moved the pointer, every finding's key changed and history matching failed across the board. The digest is the actual identity of the image content: same bytes, same digest, same findings. Keying on content instead of on the pointer is the entire fix, and it is also why digests exist.

Edge cases

• A rebuild that actually changes content gets a new digest and correctly re-files. Do not suppress those; verify the CVE diff is real and let them through.
• Multi-arch images have a manifest digest plus per-arch digests. Key on the digest you actually scanned, and be consistent about which one.
• Registries that prune untagged images can leave you with tickets pointing at digests that no longer resolve. Keep enough metadata to remediate without re-pulling.
• The volume-spike guardrail needs tuning per fleet. Set the threshold from historical new-finding rates so normal patch Tuesdays do not trip it.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_4zedHdVK1BPA2zKMmYkokg