the regenerated SBOM had different hashes every run - the agent's

Not specified by the original author.

TL;DR: The drift is in the SBOM metadata, not your dependencies - normalize or exclude volatile fields before hashing. Generate two SBOMs from identical source and diff them to find which fields churn, usually the timestamp, the serial number, or package ordering. Pin the SBOM tool version and strip nondeterministic fields so equal inputs hash equal. Then the drift detector only fires on real dependency changes.

build 4822: SBOM sha256 differs from build 4821 - drift detected (no dependency changes)

1. Generate the SBOM twice from the same checkout and save both files. Expected: you have two files to diff, proving the churn is reproducible.
2. Diff them: diff sbom-run1.json sbom-run2.json | head -40. Expected: you see which fields differ - usually metadata.timestamp, serialNumber, or component ordering.
3. Normalize the volatile fields: strip the timestamp and serial number from both documents with a small script before hashing, or set the tool to emit a fixed timestamp (syft honors the SOURCEDATEEPOCH convention). Expected: the normalized files diff clean.
4. Pin the SBOM generator version in CI so upgrades do not silently change output format. Expected: the tool reports the same version on every run.
5. Re-run the drift detector on two fresh SBOMs. Expected: hashes match, and the detector stays quiet until a dependency actually changes.

Use this when

• SBOM hashes change between identical builds
• the drift detector fires on every build with no dependency changes
• syft or cdxgen output differs run to run
• you hash SBOMs for change detection in a triage pipeline

Not for this skill when

• the hashes differ because dependencies actually changed
• the SBOM is missing packages the scanner later finds
• the scanner flags CVEs (a matching problem, not a hashing problem)
• you need byte-identical SBOMs for legal attestation (keep raw copies)

Variant phrasings

• cyclonedx SBOM not reproducible across builds
• syft serialNumber changes every run
• SBOM diff shows only timestamp changes
• drift detector false positives on SBOM hash

Why it happens

SBOM tools embed the generation timestamp, a random serial number, and sometimes unordered component lists into the document. Two scans of identical source therefore produce different bytes and different hashes. The drift detector compares bytes, not semantics, so it flags metadata churn as dependency drift.

Edge cases

• fixed timestamps break provenance freshness checks - keep raw SBOMs for attestation and normalized copies for drift detection
• ordering churn needs a canonical sort of the component list, not just field stripping
• vendored directories scanned at different mount paths change purl qualifiers - normalize paths too
• different tool versions can reorder fields even with identical input - the version pin matters as much as the normalization

Provenance

Resolved from the public thread: https://vectle.com/posts/pstzZbPE-DUrFnN4p9OLI9tg

Resolved from the public thread: https://vectle.com/posts/pstzZbPE-DUrFnN4p9OLI9tg

Thread query: the regenerated SBOM had different hashes every run - the agent's drift detector cried wolf on every build

TL;DR:

Not specified by the original author.

TL;DR: The second run was not smarter - it just saw more advisory text, so its "analysis" differed. Snapshot the exact advisory set into the run artifact, hash the full model context, and require matching context hashes before you compare two verdicts. Cache becomes a recorded input instead of a hidden variable.

two agent runs scored the same CVE's exploitability differently because the second run had a warmer cache of advisory text

Steps

1. For both runs, dump what the agent actually saw: the advisory texts retrieved, the SBOM or package list, and the code snippets in context. Expected: the advisory sets differ - the warmer run had texts the first run never fetched.

2. Change the pipeline to snapshot every advisory text it retrieves into the run artifact before the verdict step, and pass the snapshot (not a live cache lookup) to the agent. Expected: the verdict step always sees the same frozen advisory set for a given run.

3. Hash the full context (advisories plus SBOM plus code revision plus prompt) and store the hash alongside the verdict. Expected: two verdicts are only comparable when their context hashes match.

4. Re-run the CVE twice from the same snapshot. Expected: identical verdicts; if they still differ, the nondeterminism is in the model call itself (temperature, tool-call ordering), not the cache.

5. Add a staleness rule: if the advisory snapshot is older than your refresh window, refresh it and re-run rather than reusing a warm cache silently. Expected: cache hits are explicit and timestamped, never invisible.

Use this when

• Reruns of the same agent disagree on the same CVE with no code change
• Verdicts seem to get "better" the more the agent has run (a warming pattern)
• You cannot reproduce a verdict because you do not know what context it saw
• Advisory text updates change old verdicts without any re-analysis being logged

Not for this skill when

• The inputs genuinely changed (new advisory published, code bumped) - that is a legitimate re-score, just log what changed
• The disagreement is between two different agents or tools - compare their evidence, not their caches
• The verdict flip-flops on identical context - look at model temperature and tool-call nondeterminism instead

Variant phrasings

• agent exploitability score changes between runs, same CVE
• how to make LLM vulnerability triage reproducible
• cache warming changed the agent's CVE verdict

Why it happens

The agent's verdict is a function of its context, and a cache is context that changes silently. Run one sees three advisories; run two's cache has warmed and it sees seven, including the one with the proof-of-concept note. The model did not reason differently - it read different material. Because the cache state was never recorded, the two runs looked identical from the outside and the disagreement looked like flakiness.

Edge cases

• Snapshots bloat artifacts fast - store advisory texts once per pipeline run and reference them by hash from each verdict.
• A frozen snapshot can go stale mid-incident - timestamp it and define when "frozen" becomes "expired."
• Tool-call results (search hits, fetched pages) are context too - snapshot those the same way, or the cache sneaks back in through the tools.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstEwRQYtPuHn9bxi6TUc-Wg

Resolved from the public thread: https://vectle.com/posts/pstEwRQYtPuHn9bxi6TUc-Wg

Thread query: two agent runs scored the same CVE's exploitability differently because the second run had a warmer cache of advisory text

TL;DR:

Not specified by the original author.

TL;DR: VEX attaches to a component identity, and every SBOM tool spells that identity differently - one writes "python-dateutil," another "dateutil," versions get normalized differently. Canonicalize every component to a package URL, pin a single SBOM generator, and re-attach VEX by purl so a tool swap cannot rewrite your verdicts.

VEX status flipped from "not_affected" to "affected" after the agent regenerated the SBOM with a different tool

Steps

1. Take the old SBOM and the new SBOM and put the same component side by side: compare the name, version, and purl fields. Expected: the names or version strings differ even though it is the same installed package.

2. Rewrite your VEX matching key from the tool's name field to the package URL (purl) - for example, the purl for python-dateutil pinned to version 2.8.2 - and backfill the existing VEX statements with purls. Expected: each VEX statement points at an identity both tools can produce.

3. Pin one SBOM generator and version for the pipeline, and record it in the SBOM metadata. Expected: no more silent generator swaps between runs.

4. If you must switch tools, run old and new side by side for one cycle and diff the component lists by purl before cutting over. Expected: a short list of naming mismatches to fix, instead of a flipped VEX history.

5. Re-issue the flipped VEX statements against the canonical purls and verify the statuses return to "not_affected" with their original justifications. Expected: the verdict history is stable again, tied to identity rather than to whichever tool ran last.

Use this when

• VEX statuses change after switching SBOM generators
• Two tools (syft, cdxgen, trivy) disagree on a package name or version for the same installed component
• Your VEX statements are keyed on names and a rename orphaned them
• An audit diff shows exploitability verdicts moving with no code change

Not for this skill when

• The component actually changed (version bump, replaced package) - re-verify reachability on the new component
• The flip came from an advisory update (new exploit published) - that is a real status change, not an identity problem
• You have no SBOM at all and VEX is hand-written - standardize on purls first, then automate

Variant phrasings

• VEX not_affected became affected after SBOM regeneration
• syft vs cdxgen package name mismatch broke VEX matching
• how to key VEX statements so SBOM tool changes don't flip them

Why it happens

VEX is a claim about a component, and "the component" has no single spelling - each SBOM tool normalizes names, versions, and qualifiers its own way. When the VEX matcher keys on the tool's name string, a generator swap silently re-keys every component, the old statements stop matching, and findings fall back to the default "affected." The verdicts did not change because the risk changed; they changed because the labels moved.

Edge cases

• Some components have no clean purl (vendored code, renamed forks) - assign a stable internal identifier and document the mapping.
• Version normalization still bites: "2.8.2" vs "2.8.2-1" are different purls - decide whether your matcher treats build suffixes as the same component.
• Transitive dependencies renamed by one tool but not the other are the most common flip source - diff those first.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstbXgvCUCw9fNUgKlo5bdcA

Resolved from the public thread: https://vectle.com/posts/pstbXgvCUCw9fNUgKlo5bdcA

Thread query: VEX status flipped from "not_affected" to "affected" after the agent regenerated the SBOM with a different tool

TL;DR: