TL;DR: Require every vulnerability finding to quote an exact changed line, then programmatically verify each quote against the actual diff and drop anything that does not match. Models pattern-match on scary-looking code shapes and describe vulnerabilities they expect to see rather than ones present in the diff. Grounding each claim in a verified quote kills the phantoms.

```text
code review agent hallucinating vulnerabilities that are not in the diff
```

1. Collect every vulnerability finding and its cited location.
   Expected: a list of claims, each pointing at a file and line.
2. Verify each cited line exists in the actual diff: extract the diff's changed lines and check that the quote matches verbatim.
   Expected: each claim either matches a real changed line or is flagged as unverified.
3. Drop every finding whose cited code is not in the diff. No exceptions, no "it might be nearby".
   Expected: the finding list shrinks to only grounded claims.
4. For the survivors, check the claim against the real code semantics (is that input actually unsanitized?).
   Expected: each remaining finding is both grounded and correct.
5. Tighten the review instructions: require exact code quotes for every finding and forbid claims about code outside the diff.
   Expected: the next run produces fewer, better findings.
6. Track the phantom rate over runs: dropped findings divided by total findings.
   Expected: the rate trends down; if it does not, the instructions need another pass.

## Use this when
- the agent reports vulnerabilities that are not in the changed code
- security findings cite functions or patterns absent from the PR
- developers have started ignoring the agent's security comments

## Not for this skill when
- the findings are real but low-severity (that is a tuning problem, not hallucination)
- the agent misses real vulnerabilities (that is recall, the opposite problem)
- a human wrote the findings (check the human, not the agent)

## Variant phrasings
### review bot reported SQL injection that isnt in the code
### ai reviewer inventing security issues on the PR
### phantom vulnerability findings in automated code review

## Why it happens
Models pattern-match on scary-looking code shapes, and review instructions that emphasize security amplify it. Without a grounding requirement, the model completes the pattern: it describes the vulnerability it expects to see rather than the code in front of it. The stronger the instructions push on security, the more phantoms appear.

## Edge cases
- A finding can cite a real line but misread it (a parameterized query called "unsanitized"); grounding is necessary but not sufficient, so keep the semantics check.
- Findings about unchanged context lines are legitimate only if the diff actually changed the risk; default to changed-lines-only.
- Adversarial PRs can plant misleading comments; the agent should review code, not comments.
- Do not silently drop findings without logging them; the phantom log is what lets you measure improvement.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_yJED-GkzwJ-nBfXrKN9wTw