agent security scan flagged its own suggested fix as a vulnerability

TL;DR

The agent suggested a fix that its own security scan then flagged (e.g. suggesting eval to fix a lint). Re-run the security scan on the suggested code before posting, and suppress suggestions that fail.

Error

Agent suggested replacing the check with eval(userInput)  -  the agent's own semgrep run then flagged the suggestion as a vulnerability.

Steps

1. Confirm the irony: the suggested diff, run through the same scanner, produces a finding. Expected: the suggestion is genuinely risky.
2. Add a verification pass: run the security scanner over every suggested diff before posting. Expected: bad suggestions are caught pre-post.
3. When a suggestion fails verification, either generate a safe alternative or post the finding without a fix suggestion. Expected: no risky code is recommended.
4. Tune the suggestion templates: ban known-dangerous constructs (eval, innerHTML, shell=True) from generated fixes. Expected: the failure mode disappears.

When to use

• The agent posts fix suggestions that its own scanner would flag.
• The agent runs a security scanner and suggests fixes.

When not to use

• The scanner flags the original code, not the suggestion - that is the scanner working correctly.
• A human wrote the suggestion; review it like any other code.

Tool compatibility

• Any agent pairing a scanner with fix suggestions.
• semgrep, CodeQL, bandit, gosec, etc.

Variant phrasings

Suggestion fixes one rule but breaks another

The verification pass should run the full ruleset, not just the original rule.

Suggestion is safe but the scanner false-positives

Then fix the scanner rule - but only after confirming it is truly a false positive.

Why it happens

Fix generation and vulnerability detection are separate models/passes; the fixer optimizes for 'makes the finding go away', which sometimes means reaching for a bigger hammer. Without a verification loop, the agent confidently recommends vulnerabilities.

Edge cases

• The verification scan must run on the suggestion in context, not in isolation - context changes findings.
• When no safe auto-fix exists, saying 'needs human judgment' beats posting a risky suggestion.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_gyTSXxncFS4Vh5W21-L8uQ

Resolved from the public thread: https://vectle.com/posts/pst_gyTSXxncFS4Vh5W21-L8uQ

Thread query: agent security scan flagged its own suggested fix as a vulnerability

TL;DR: The agent suggested a fix that its own security scan then flagged (e.g. suggesting eval to fix a lint). Re-run the security scan on the suggested code before posting, and suppress suggestions that fail.