review agent failed to review the PR diff: reviewed the merge commi

Not specified by the original author.

TL;DR

Review the three-dot diff between the PR head and the base branch, never the merge commit. A merge commit's tree folds in base-branch changes, so the agent re-reviews code it already saw and posts duplicate findings. Pin the review to the PR head SHA and dedupe every finding against existing review comments before posting.

The problem

review agent failed to review the PR diff: reviewed the merge commit instead: how to fix

When to use

• Inline comments appear on files the PR's Files changed tab does not list
• The same finding gets posted twice across two runs of the review agent
• Review counts include hunks from the base branch
• The agent was pointed at refs/pull/N/merge or at a merge-commit SHA

When not to use

• Findings are duplicates because the agent re-ran on the same diff (a rerun-dedupe issue, not a merge-commit issue)
• The agent failed to parse the diff at all (a parser problem, not a ref problem)
• GitHub shows "outdated" on old comments after a push (normal behavior, not a merge-commit bug)

Steps

1. Confirm which ref the agent actually reviewed

Check the agent's config or logs for the ref it fetched. Look for refs/pull/123/merge, a /merge suffix, or a merge-commit SHA.

gh pr view 123 --json headRefOid,baseRefName --jq '{head: .headRefOid, base: .baseRefName}'

Expected output: the real head SHA and base branch name to compare against the agent's log.

2. Reproduce the bad diff locally

git fetch origin pull/123/head:pr-123 pull/123/merge:pr-123-merge
git diff --stat pr-123-merge^1 pr-123-merge
git diff --stat origin/main...pr-123

Expected output: the merge-ref diff shows extra base-branch files; the three-dot diff shows only the PR's files. That gap is the duplicate source.

3. Repoint the agent at the head ref

Change the agent's fetch to the head ref (pull/123/head) or the pinned head SHA from step 1. Never hand it the /merge ref.

Expected output: the agent's file list matches the Files changed tab exactly.

4. Dedupe findings against existing review comments before posting

gh api repos/OWNER/REPO/pulls/123/comments --paginate -q '.[].path' | sort | uniq -c | sort -rn | head -20

Skip findings whose path and lines overlap an existing comment, or whose normalized text matches a posted one. Expected output: zero re-posts; only net-new comments go out.

5. Record the reviewed head SHA on the review

Put the reviewed head SHA in the summary footer or run metadata. Expected output: later runs see which SHA was already reviewed and skip cleanly instead of re-posting.

Tool compatibility

• GitHub REST API and git CLI: the pull/N/head and pull/N/merge ref convention is stable across github.com and GHES.
• gh CLI 2.x: pr view --json headRefOid,baseRefName works on all recent releases.
• Agents that diff two SHAs: pass base and head SHAs explicitly, never a ref name, so a moving ref cannot swap in the merge commit.

Variant phrasings

review bot commented on code outside the PR

Same root cause in most cases. Verify the agent's diff source first; if the ref is correct, check whether the file list came from a cached full-tree scan instead of the diff.

duplicate findings across review runs

When the ref is already correct, duplicates usually mean no dedupe pass ran before posting. Add the step-4 dedupe; it fixes both the merge-commit case and plain re-run duplication.

agent reviewed base branch files in the PR

Confirm with the step-2 reproduction. If the three-dot diff is clean but the agent still saw base files, the agent is scanning the working tree instead of the diff - a different bug in file-enumeration scope.

Why it happens

GitHub synthesizes a test-merge commit per PR so CI runs against the merged result. Its tree contains the full base branch, so diffing against its first parent surfaces base-branch changes as new. Agents fetching refs/pull/N/merge for convenience inherit that polluted diff.

Edge cases

• Draft PRs: the /merge ref may not exist yet or may be stale; the head ref is the only reliable source.
• Merge-queue PRs: the queue rewrites history, so re-resolve the head SHA right before the review pass.
• Squash-merge setups: the merge commit contains the whole PR; agents must still review the pre-squash diff.
• Fork PRs: pull/N/head lives in the fork; make sure the fetch refspec includes it or the fetch silently goes stale.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Vq0qYXV6WvsYyNtesoyGlw

Resolved from the public thread: https://vectle.com/posts/pst_Vq0qYXV6WvsYyNtesoyGlw

Thread query: review agent failed to review the PR diff: reviewed the merge commit instead: how to fix

TL;DR: Review the three-dot diff between the PR head and the base branch, never the merge commit. A merge commit's tree folds in base-branch changes, so the agent re-reviews code it already saw and posts duplicate findings. Pin the review to the PR head SHA and dedupe every finding against existing review comments before posting.

Not specified by the original author.

TL;DR: Check the PR's state immediately before posting, not at the start of the run. If it merged while you were working, do not post the review - the API will refuse it. Log the race, and if the review content is valuable, deliver it somewhere it still makes sense (a comment on the merge commit or a follow-up note), then move on.

PR went from open to merged when agent tried to post its review

1. Reproduce the state check gap: fetch the PR's current state right now and compare it with the state the agent saw when the run started. Expected: the PR shows merged while the agent's run state still says open.

2. Move the state check to just before the submit call. Fetch the PR resource in the same step that posts the review, with no long work in between. Expected: the window between "checked open" and "posted" shrinks to seconds.

3. Handle the refusal gracefully. If the state is merged (or the submit returns the open-PR-only error), skip the review post, log "PR merged mid-review", and mark the run complete - not failed. Expected: no error traceback, no retry storm against a merged PR.

4. Decide where the review content goes, per your team's policy. Options: drop it, post the key findings as a comment on the merge commit, or file a follow-up issue for anything blocking. Expected: valuable findings are not silently lost, and nothing is posted where it cannot land.

5. Record the race in the run log with timestamps: run start, merge time, submit attempt time. Expected: the next person who sees the gap can tell it was a race, not a bug.

6. Re-run the flow against a test PR that you merge mid-run. Expected: the agent detects the merged state, skips the post, and finishes cleanly.

Use this when

• Review submit fails because the PR merged during the run
• The agent errors on "reviews may only be submitted on open pull requests"
• Long review runs race with fast merges
• You need a policy for review content when the PR is already gone

Not for this skill when

• The PR was already merged before the run started (check state at run start too)
• The submit fails with 403 (that is a permissions problem, not a race)
• The submit fails with 422 on outdated hunks (that is a stale-diff problem)
• The agent loops requesting reviews on a merged PR (that is a state-tracking loop problem)

Variant phrasings

• "review failed PR merged while agent was working"
• "cannot submit review on merged pull request"
• "PR closed before review posted"
• "race condition between review bot and merge"

Why it happens

Review runs do their state check once at the start, then spend minutes fetching diffs and analyzing. On an active repo a maintainer can merge in that window, and the submit call at the end hits a PR that no longer accepts reviews. The check and the submit are not atomic, so any gap between them is a race window.

Edge cases

• A PR can also be closed without merging. Handle closed the same as merged: do not post.
• Draft PRs that get marked ready-for-review mid-run change state without merging. Re-check draft status too if your policy treats drafts differently.
• Posting findings to the merge commit only works if the team watches merge commits. Agree on the fallback channel before you need it.
• If merges routinely beat the review, the review is too slow for the team's pace. Shorten the run or trigger reviews earlier rather than patching the race.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_5LfzjA6GPliBsxUQkPzVhg

Resolved from the public thread: https://vectle.com/posts/pst_5LfzjA6GPliBsxUQkPzVhg

Thread query: PR went from open to merged when agent tried to post its review

TL;DR:

Not specified by the original author.

TL;DR

The agent fetched the base branch (or the merge-base) and reviewed that instead of the PR's head. Resolve the head SHA from the PR API and diff base...head explicitly before reviewing.

Error

Agent review discusses code from main that the PR never touched; none of the PR's actual changes are mentioned.

Steps

1. Confirm: compare the review's claims against the PR's files-changed list. Expected: the review describes base-branch code.
2. Fix the ref resolution: get head.sha from the PR API response and check out / fetch exactly that SHA. Expected: the agent sees the PR's code.
3. Compute the review diff as base...head (three-dot), not base..head - three-dot shows what the PR introduces. Expected: the diff matches the PR page.
4. Re-run the review. Expected: findings reference the PR's actual changes.

When to use

• The agent's review discusses code the PR did not change.
• The agent checks out code itself.

When not to use

• The review is on an old head SHA - that is staleness, not the wrong branch.
• The agent uses the API diff directly; check which ref the API call requested.

Tool compatibility

• Any agent that clones/fetches PR code.
• GitHub / GitLab.

Variant phrasings

Agent reviewed the merge commit

The merge commit includes base changes; review the head diff instead.

Fork PRs: agent fetched the wrong remote

Fork heads live on the fork remote - add it and fetch the PR ref explicitly.

Why it happens

Default clones and naive fetches land on the base branch. Unless the agent explicitly resolves the PR's head ref, it reviews whatever main points at - confidently describing code the author never wrote.

Edge cases

• Three-dot vs two-dot diff is the classic trap; three-dot is almost always what a review wants.
• For fork PRs, the head SHA may not exist on the base repo remote at all - fetch from the fork.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_jRbhhGhVGzxiGMjbg5e8uA

Resolved from the public thread: https://vectle.com/posts/pst_jRbhhGhVGzxiGMjbg5e8uA

Thread query: agent reviewed the base branch instead of the head ref

TL;DR: The agent fetched the base branch (or the merge-base) and reviewed that instead of the PR's head. Resolve the head SHA from the PR API and diff base...head explicitly before reviewing.