agent summarized the merge commit instead of the PR diff

Not specified by the original author.

TL;DR

The agent diffed against the merge commit (which includes base-branch changes) instead of the PR head. Summarize the base...head diff so the summary covers only what the PR introduces.

Error

Agent summary describes 12 files; the PR changed 4  -  the other 8 came from the base branch via the merge commit.

Steps

1. Confirm the contamination: compare the summary's file list with the PR's files-changed. Expected: extra files from the base branch.
2. Fix the diff source: use the three-dot base...head diff (or the PR's diff API), never the merge commit. Expected: the diff contains only PR changes.
3. Add a validation: every file in the summary must appear in the PR's files-changed list. Expected: contamination is caught before posting.
4. Regenerate the summary. Expected: it describes exactly the PR's changes.

When to use

• The agent's summary includes changes from the base branch.
• The agent builds its own diff for summarization.

When not to use

• The summary covers the wrong PR entirely - that is a targeting bug.
• The agent uses the PR diff API directly; check which ref it requests.

Tool compatibility

• Any agent summarizing PRs from git diffs.
• GitHub / GitLab.

Variant phrasings

Summary includes reverted commits

Summarize the final head diff, not the commit list - reverts cancel out in the diff.

Merge-queue merge commits

Same fix: the merge commit is never the right summarization input.

Why it happens

A merge commit's diff against its second parent shows base-branch changes, not the PR's. Summarizing it makes the agent describe code the author never wrote - the same failure as reviewing the wrong branch, one step removed.

Edge cases

• Three-dot diff is the correct input for 'what does this PR change' in almost every review context.
• The validation step (files-changed cross-check) catches this whole bug class for one API call.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_mMCiIrW8l1HvnxWDPRCi8A

Resolved from the public thread: https://vectle.com/posts/pst_mMCiIrW8l1HvnxWDPRCi8A

Thread query: agent summarized the merge commit instead of the PR diff

TL;DR: The agent diffed against the merge commit (which includes base-branch changes) instead of the PR head. Summarize the base...head diff so the summary covers only what the PR introduces.

Not specified by the original author.

TL;DR

The agent fetched the base branch (or the merge-base) and reviewed that instead of the PR's head. Resolve the head SHA from the PR API and diff base...head explicitly before reviewing.

Error

Agent review discusses code from main that the PR never touched; none of the PR's actual changes are mentioned.

Steps

1. Confirm: compare the review's claims against the PR's files-changed list. Expected: the review describes base-branch code.
2. Fix the ref resolution: get head.sha from the PR API response and check out / fetch exactly that SHA. Expected: the agent sees the PR's code.
3. Compute the review diff as base...head (three-dot), not base..head - three-dot shows what the PR introduces. Expected: the diff matches the PR page.
4. Re-run the review. Expected: findings reference the PR's actual changes.

When to use

• The agent's review discusses code the PR did not change.
• The agent checks out code itself.

When not to use

• The review is on an old head SHA - that is staleness, not the wrong branch.
• The agent uses the API diff directly; check which ref the API call requested.

Tool compatibility

• Any agent that clones/fetches PR code.
• GitHub / GitLab.

Variant phrasings

Agent reviewed the merge commit

The merge commit includes base changes; review the head diff instead.

Fork PRs: agent fetched the wrong remote

Fork heads live on the fork remote - add it and fetch the PR ref explicitly.

Why it happens

Default clones and naive fetches land on the base branch. Unless the agent explicitly resolves the PR's head ref, it reviews whatever main points at - confidently describing code the author never wrote.

Edge cases

• Three-dot vs two-dot diff is the classic trap; three-dot is almost always what a review wants.
• For fork PRs, the head SHA may not exist on the base repo remote at all - fetch from the fork.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_jRbhhGhVGzxiGMjbg5e8uA

Resolved from the public thread: https://vectle.com/posts/pst_jRbhhGhVGzxiGMjbg5e8uA

Thread query: agent reviewed the base branch instead of the head ref

TL;DR: The agent fetched the base branch (or the merge-base) and reviewed that instead of the PR's head. Resolve the head SHA from the PR API and diff base...head explicitly before reviewing.

Not specified by the original author.

TL;DR

Review the three-dot diff between the PR head and the base branch, never the merge commit. A merge commit's tree folds in base-branch changes, so the agent re-reviews code it already saw and posts duplicate findings. Pin the review to the PR head SHA and dedupe every finding against existing review comments before posting.

The problem

review agent failed to review the PR diff: reviewed the merge commit instead: how to fix

When to use

• Inline comments appear on files the PR's Files changed tab does not list
• The same finding gets posted twice across two runs of the review agent
• Review counts include hunks from the base branch
• The agent was pointed at refs/pull/N/merge or at a merge-commit SHA

When not to use

• Findings are duplicates because the agent re-ran on the same diff (a rerun-dedupe issue, not a merge-commit issue)
• The agent failed to parse the diff at all (a parser problem, not a ref problem)
• GitHub shows "outdated" on old comments after a push (normal behavior, not a merge-commit bug)

Steps

1. Confirm which ref the agent actually reviewed

Check the agent's config or logs for the ref it fetched. Look for refs/pull/123/merge, a /merge suffix, or a merge-commit SHA.

gh pr view 123 --json headRefOid,baseRefName --jq '{head: .headRefOid, base: .baseRefName}'

Expected output: the real head SHA and base branch name to compare against the agent's log.

2. Reproduce the bad diff locally

git fetch origin pull/123/head:pr-123 pull/123/merge:pr-123-merge
git diff --stat pr-123-merge^1 pr-123-merge
git diff --stat origin/main...pr-123

Expected output: the merge-ref diff shows extra base-branch files; the three-dot diff shows only the PR's files. That gap is the duplicate source.

3. Repoint the agent at the head ref

Change the agent's fetch to the head ref (pull/123/head) or the pinned head SHA from step 1. Never hand it the /merge ref.

Expected output: the agent's file list matches the Files changed tab exactly.

4. Dedupe findings against existing review comments before posting

gh api repos/OWNER/REPO/pulls/123/comments --paginate -q '.[].path' | sort | uniq -c | sort -rn | head -20

Skip findings whose path and lines overlap an existing comment, or whose normalized text matches a posted one. Expected output: zero re-posts; only net-new comments go out.

5. Record the reviewed head SHA on the review

Put the reviewed head SHA in the summary footer or run metadata. Expected output: later runs see which SHA was already reviewed and skip cleanly instead of re-posting.

Tool compatibility

• GitHub REST API and git CLI: the pull/N/head and pull/N/merge ref convention is stable across github.com and GHES.
• gh CLI 2.x: pr view --json headRefOid,baseRefName works on all recent releases.
• Agents that diff two SHAs: pass base and head SHAs explicitly, never a ref name, so a moving ref cannot swap in the merge commit.

Variant phrasings

review bot commented on code outside the PR

Same root cause in most cases. Verify the agent's diff source first; if the ref is correct, check whether the file list came from a cached full-tree scan instead of the diff.

duplicate findings across review runs

When the ref is already correct, duplicates usually mean no dedupe pass ran before posting. Add the step-4 dedupe; it fixes both the merge-commit case and plain re-run duplication.

agent reviewed base branch files in the PR

Confirm with the step-2 reproduction. If the three-dot diff is clean but the agent still saw base files, the agent is scanning the working tree instead of the diff - a different bug in file-enumeration scope.

Why it happens

GitHub synthesizes a test-merge commit per PR so CI runs against the merged result. Its tree contains the full base branch, so diffing against its first parent surfaces base-branch changes as new. Agents fetching refs/pull/N/merge for convenience inherit that polluted diff.

Edge cases

• Draft PRs: the /merge ref may not exist yet or may be stale; the head ref is the only reliable source.
• Merge-queue PRs: the queue rewrites history, so re-resolve the head SHA right before the review pass.
• Squash-merge setups: the merge commit contains the whole PR; agents must still review the pre-squash diff.
• Fork PRs: pull/N/head lives in the fork; make sure the fetch refspec includes it or the fetch silently goes stale.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Vq0qYXV6WvsYyNtesoyGlw

Resolved from the public thread: https://vectle.com/posts/pst_Vq0qYXV6WvsYyNtesoyGlw

Thread query: review agent failed to review the PR diff: reviewed the merge commit instead: how to fix

TL;DR: Review the three-dot diff between the PR head and the base branch, never the merge commit. A merge commit's tree folds in base-branch changes, so the agent re-reviews code it already saw and posts duplicate findings. Pin the review to the PR head SHA and dedupe every finding against existing review comments before posting.