agent classified a scanner false positive as critical with no evidence

TL;DR

Scanners flag possibilities; testers confirm realities. An agent that promotes a scanner's tentative match straight to critical, without the verification request, an exploit check, or even the response evidence, will eventually cry wolf on the report's most important line. Insert a verification state: no finding renders as critical until a human or a scripted check confirms the condition, with the scanner-only original kept as unconfirmed until then.

agent classified a scanner false positive as critical with no evidence

Use this when

• A critical finding's entire evidence is one scanner line, and re-requesting the alleged issue on the authorized target does not reproduce it.
• The scanner itself labelled the result as a potential or pattern match rather than a confirmed issue.

Not for this skill when

• The finding was manually verified and the verification evidence is filed with the engagement, even if the report paragraph is brief. Attach the evidence to the finding.
• The issue is a known exploited vulnerability on an exposed, version-confirmed service. That can stand as critical with the version evidence cited.

Steps

1. Isolate the finding's evidence: list exactly what supports the critical rating, separating scanner assertion from any independent confirmation. Verify: Verify: the gap, scanner assertion only, is stated plainly.
2. Attempt verification on the authorized target within the rules of engagement: re-request the condition, check the version or configuration the scanner inferred, and record the outcome either way. Verify: Verify: a verification record with a timestamp and result exists for this finding.
3. Reclassify based on verification: confirmed issues keep their scored severity, unreproducible ones are downgraded to a false-positive note or an unconfirmed observation, with the reasoning shown. Verify: Verify: the report's critical count now contains only verified findings.
4. Add the pipeline rule: critical and high findings require a verification field that references a stored check; the renderer blocks unverified top severities. Verify: Verify: a test scanner-only critical fails to render as critical.
5. Feed the false positive back: note the scanner, template, and target traits that produced it in the engagement tuning notes so the next run flags it for review automatically. Verify: Verify: the tuning note exists and the next import of the same pattern is marked unconfirmed from the start.

Variant phrasings

scanner false positive reported critical no verification

The symptom phrasing.

no critical without verification report pipeline rule

The process phrasing.

agent trusted nuclei template match as confirmed finding

The template-specific phrasing.

Compatibility: Compatibility: any agent reporting pipeline; verification records can be manual tester notes or scripted rechecks stored with the finding.

Why it happens

Scanner severities describe the template's potential impact if true, not the probability it is true on this target. Pattern matches on error pages, WAF responses, and shared hosting fingerprints routinely look like famous critical issues. A model summarising scanner output inherits the impact number and drops the uncertainty, because scanner output rarely carries its doubt in the same field.

Edge cases / pitfalls

• Downgrading must stay honest in both directions: a verified critical must not be softened just to keep the report calm.
• Some false positives reveal a real lesser issue, such as a verbose error page; record that as its own correctly rated finding.
• Time pressure is the usual reason verification is skipped; the renderer block exists precisely for deadline day.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Z4yOO74n2Ky5U1NHLItgjg