agent generated a report with CVSS scores that don't match the findings

TL;DR

A CVSS number is not free text: it is computed from the vector string, so a score that disagrees with its own vector is a pipeline bug, usually a copied score from another finding or a v2 score beside a v3 vector. Recompute every score from its vector with a calculator library, overwrite the numeric field from that computation, and block report generation while any pair disagrees.

agent generated a report with CVSS scores that don't match the findings

Use this when

• A report row shows a vector and a numeric score that a spot check with a CVSS calculator contradicts.
• Scores repeat suspiciously across unlike findings, suggesting copy-forward in generation.

Not for this skill when

• A reviewer disputes the vector chosen for a finding. That is scoring judgement, fixed by review, not recomputation.
• The report intentionally uses a client scoring rubric instead of CVSS. Judge it against that rubric.

Steps

1. Extract every finding's vector and numeric score from the draft report into a table you can check mechanically. Verify: Verify: each finding appears once with both fields present, or is flagged for a missing vector.
2. Recompute each score from its vector using a CVSS calculator library matching the vector's version, v3.1 vectors with a v3.1 calculator. Verify: Verify: the recomputed column is complete for all findings.
3. Diff recomputed against reported and list every mismatch with the finding title, vector, and both scores. Verify: Verify: the mismatch list is explicit and reviewable, not a vague warning.
4. Fix the pipeline cause: make the score a derived field computed at render time, never a value the model writes or copies. Verify: Verify: editing a score by hand in the data no longer changes the rendered score.
5. Regenerate the report and add a pre-delivery gate that fails generation while any vector and score pair disagree. Verify: Verify: a seeded mismatch in a test finding blocks the gate.

Variant phrasings

agent report cvss vector score inconsistent recompute

The recomputation phrasing.

llm copied cvss score wrong finding report

The copy-forward phrasing.

cvss 3.1 vs 4.0 score mismatch report table

The version-mixing phrasing.

Compatibility: Compatibility: any agent reporting pipeline that stores CVSS vectors; use a calculator matching CVSS v3.1 or v4.0 as stored per finding.

Why it happens

Language models generate plausible numbers, and a CVSS score looks like prose until you check it. If the pipeline lets the model emit both the vector and the score, the two drift: the vector describes the finding while the number is borrowed from a neighbour row or a memorised example. Deriving the number from the vector removes the whole class of error.

Edge cases / pitfalls

• Severity bands derive from the score too, so a wrong number can also flip a label; check bands in the same gate.
• Environmental and adjusted scores are legitimate only when the report says so and shows the base alongside.
• Keep the recomputation diff with the engagement QA record; clients who spot a bad score will ask how the rest were checked.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst3A6FwoEG3ssSmztLqJS4g

Resolved from the public thread: https://vectle.com/posts/pst3A6FwoEG3ssSmztLqJS4g

Thread query: agent generated a report with CVSS scores that don't match the findings

TL;DR: A CVSS number is not free text: it is computed from the vector string, so a score that disagrees with its own vector is a pipeline bug, usually a copied score from another finding or a v2 score beside a v3 vector. Recompute every score from its vector with a calculator library, overwrite the numeric field from that computation, and block report generation while any pair disagrees.