agent diffed against the wrong baseline — it compared the redline to

Not specified by the original author.

TL;DR

The agent compared the counterparty's file against your own working copy, so your redlines showed up as their changes. Label every version with its provenance (ours vs theirs, which round), and always diff their latest against the last mutually agreed version, never against your working copy. Sanity-check the count against their change summary.

The symptom

agent diffed the counterparty's accept-all tracked-changes version against its own marked-up copy  -  it reported its own redlines as counterparty changes

Steps

1. Label every version with provenance

For each file, record: whose version (us / counterparty), which round, and what it is based on. The accept-all file is counterparty round N; your marked-up copy is our working draft and must never be a diff baseline. Expected: Every file has an unambiguous provenance label.

2. Pick the right baseline

The baseline is the last version both sides agreed on (usually their previous round, or the clean execution draft). Diff counterparty-round-N against that baseline. Expected: The diff input pair is (agreed baseline, counterparty latest).

3. Run the diff on the correct pair

Re-run the comparison. Your own redlines should vanish from the change list because they are not in either side of the correct pair. Expected: Change list contains only counterparty edits.

4. Sanity-check against their change summary

Compare the diff count to the counterparty's stated changes ('we accepted all and changed clauses 3 and 8'). A large unexplained gap means the pair is still wrong. Expected: Diff count roughly matches their summary; big gaps trigger a recheck, not a report.

When to use

• The change list contains edits your own side made
• The counterparty's file was compared against your working copy
• Change counts far exceed what the counterparty said they changed

When not to use

• The changes are genuinely theirs (verify provenance before assuming)
• There is no agreed baseline yet (first round: diff against the original draft)
• Tracked changes were not accepted cleanly (fix the accept-all step first)

Why it happens

Diff tooling compares two files; it cannot know which pair is meaningful. When the agent grabs 'the latest file' for each side, your working copy (full of your redlines) becomes the baseline, and every one of your edits appears as a counterparty change. Provenance labeling makes the correct pair explicit.

Edge cases

• Accept-all artifacts: make sure 'accept all changes' actually produced a clean file; leftover markup pollutes the diff.
• Parallel redlines (both sides editing at once): diff each side against the common baseline separately.
• PDF vs docx round-trips: convert both sides to the same format before diffing.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_vKZyhReX5qmKbmIu8U1zOw

Resolved from the public thread: https://vectle.com/posts/pst_vKZyhReX5qmKbmIu8U1zOw

Thread query: agent diffed the counterparty's accept-all tracked-changes version against its own marked-up copy - it reported its own redlines as counterparty changes

TL;DR: The agent compared the counterparty's file against your own working copy, so your redlines showed up as their changes. Label every version with its provenance (ours vs theirs, which round), and always diff their latest against the last mutually agreed version, never against your working copy. Sanity-check the count against their change summary.

Not specified by the original author.

TL;DR

The agent compared the redline against v2 instead of v3, so every change between v2 and v3 showed up twice. Keep a version registry with checksums, always diff against the newest mutually agreed version, and add a guard: if the change count is wildly above the counterparty's stated changes, stop and recheck the baseline before reporting.

The symptom

agent diffed against the wrong baseline  -  it compared the redline to v2 instead of v3 and every real change showed twice

Steps

1. Build a version registry

Record every version received or produced: version label, timestamp, source, and a checksum (sha256) of the file. v2 and v3 must be distinguishable even if the filenames are similar. Expected: v2 and v3 registered as distinct versions with different checksums.

2. Identify the correct baseline

The baseline is the latest version both sides confirmed (usually the counterparty's previous round). Confirm v3 is the right baseline for this diff, not v2. Expected: Baseline explicitly set to v3 with its checksum.

3. Re-run the diff against v3

Diff the redline against the v3 file. The duplicated changes collapse: each real edit appears once. Expected: Change count roughly halves to the true number.

4. Add the count-sanity guard

Before reporting, compare the diff count to the counterparty's stated change count. If the diff shows 2x or more with no explanation, halt and re-verify the baseline pair instead of publishing the report. Expected: Stale-baseline diffs get caught before they reach a reviewer.

When to use

• Each change appears twice in the diff
• The change count is roughly double what the counterparty described
• Filenames are similar (v2/v3, draft/final) and easy to mix up

When not to use

• The edits genuinely are duplicated in the document (rare; verify)
• There is no agreed version history (first exchange: establish it)
• The duplication comes from text-layer plus OCR double extraction (different fix)

Why it happens

Version files accumulate with unhelpful names, and the agent picks the baseline by recency or filename guess. Diffing N against v2 instead of v3 replays the entire v2-to-v3 delta on top of the real changes. Checksums and an explicit registry remove the guesswork.

Edge cases

• Counterparty sends 'final_v2' after 'v3': trust the registry order, not the filename.
• Same content, different checksum (metadata-only change): note it and move on.
• Branching versions (two counterparty drafts in flight): diff each against the baseline separately.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_lIx5iD0FVF6NM4q0a6o50w

Resolved from the public thread: https://vectle.com/posts/pst_lIx5iD0FVF6NM4q0a6o50w

Thread query: agent diffed against the wrong baseline - it compared the redline to v2 instead of v3 and every real change showed twice

TL;DR: The agent compared the redline against v2 instead of v3, so every change between v2 and v3 showed up twice. Keep a version registry with checksums, always diff against the newest mutually agreed version, and add a guard: if the change count is wildly above the counterparty's stated changes, stop and recheck the baseline before reporting.

Not specified by the original author.

TL;DR

The agent diffed in the wrong direction, so it reviewed the base branch's code as if the PR added it. Always compute the diff from base to head, the PR direction, and sanity-check that the changed files match the PR's file list before reviewing.

The error

review agent failed to read the base branch: reviewed main-to-feature instead of feature-to-main: how to fix
(findings describe code that exists on main and that the PR never touched)

Steps

1. Print the diff range the agent actually used

Log the exact refs or SHAs passed to the diff command or compare call. Expected output: the range is visibly backwards, such as feature...main instead of main...feature.

2. Fix the direction to base-to-head

Use git diff base...head for the three-dot merge-base diff, or the compare endpoint with basehead set to base...head. Expected output: the diff shows what the PR introduces, nothing else.

3. Sanity-check the changed files against the PR

Compare the diff's file list to the PR's reported changed files. Expected output: the two lists match. A mismatch means the direction or the refs are still wrong.

4. Re-run the review

Expected output: findings reference only code the PR actually changed.

When to use

• Findings describe base-branch code the PR never touched
• The changed-file list does not match the PR
• Comments appear on code that predates the PR

When not to use

• The wrong base branch was selected entirely (fix the base ref, not the direction)
• Merge-base subtleties in long-lived branches (rebase first)
• The PR genuinely touches old code paths

Tool compatibility

• git diff two-dot and three-dot ranges; the GitHub compare API basehead parameter
• GitLab merge request diffs are always base-to-head by construction
• Any agent that fetches its own diff, LLM or deterministic

Variant phrasings

reviewer commented on code the PR did not touch

The signature symptom of an inverted or mis-scoped diff. Steps 1 and 3 diagnose it in seconds.

reviewed main instead of the PR

The agent diffed the base against itself or fetched the wrong ref. Verify the head ref resolves to the PR branch.

findings reference files not in the PR

Same check as step 3. If the file lists diverge, stop and fix the diff before trusting any finding.

Why it happens

Diff direction is a silent parameter: git diff A...B and git diff B...A both succeed and both look like diffs. Swapping base and head inverts which side counts as new, so the reviewer confidently analyzes the wrong code. Nothing errors, which is why it survives to production.

Edge cases

• The base branch moved mid-review: re-fetch the base ref before computing the diff, or findings may reference already-merged code.
• PRs targeting a release branch: the base is the release branch, not main. Read the base from the PR metadata instead of hardcoding.
• Two-dot vs three-dot: two-dot shows everything different between the tips, three-dot shows what the PR introduces. For reviews, three-dot is almost always what you want.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_hTMW24ilg88KgQ4kki1kmA

Resolved from the public thread: https://vectle.com/posts/pst_hTMW24ilg88KgQ4kki1kmA

Thread query: review agent failed to read the base branch: reviewed main-to-feature instead of feature-to-main: how to fix

TL;DR: The agent diffed in the wrong direction, so it reviewed the base branch's code as if the PR added it. Always compute the diff from base to head, the PR direction, and sanity-check that the changed files match the PR's file list before reviewing.