the agent's redline tool compared the wrong pair - it grabbed "Contract_FINAL_v2.pdf" instead of...

TL;DR: Resolve comparison inputs by explicit file identity (full path plus content hash logged before the diff), never by fuzzy filename matching. The tool grabbed ContractFINALv2.pdf when the intended baseline was ContractFINALv2_CLEAN.pdf, and dutifully reported the differences between two different files as redline changes.

the agent's redline tool compared the wrong pair  -  it grabbed "Contract_FINAL_v2.pdf" instead of "Contract_FINAL_v2_CLEAN.pdf" and reported phantom changes

1. Confirm the wrong pair was compared. Check the diff tool's log or output metadata for the exact input paths it consumed, and hash both files. Expected: the logged inputs are the non-CLEAN file and the new version, not the intended CLEAN baseline.
2. Re-run the comparison with the correct pair. Pass explicit full paths for both inputs, verify the hashes differ from the wrong-pair run, and diff again. Expected: the phantom changes disappear and only the real edits remain.
3. Replace fuzzy file resolution with explicit selection. The comparison step should accept exact paths or document IDs from the pipeline, never a "find the file like X" pattern. Expected: there is no code path left that can resolve an ambiguous filename on its own.
4. Log the resolved inputs with every diff. Each comparison output should record both full paths and both content hashes. Expected: any future wrong-pair comparison is diagnosable from the output alone, without re-running anything.
5. Clean up the naming that caused the ambiguity. Adopt a convention where only one file in a folder can plausibly match a version (archive superseded variants, or qualify names with dates and hashes). Expected: a human browsing the folder can also tell which file is the intended baseline.

Use this when

• a redline reports changes you cannot find by manual comparison of the intended versions
• the diff tool selects its own inputs by filename pattern or recency
• two files in the folder differ by a suffix (CLEAN, FINAL, _v2) and the tool picked the wrong one
• phantom changes appear that match the differences between two baselines rather than real edits

Not for this skill when

• the pair was correct but the diff is noisy (check extraction, OCR, or conversion issues instead)
• the tool compared a file against itself (that is a filename-collision problem with its own fix)
• the "phantom" changes are real edits you had not noticed (verify against the correct pair before assuming the tool is wrong)

Variant phrasings

• redline tool picked the wrong baseline file and reported phantom edits
• agent compared against the non-clean version instead of the clean one
• diff inputs resolved by fuzzy filename match, wrong pair compared
• "changes" in the redline were just differences between two baseline variants

Why it happens

File-picking code optimizes for convenience: glob a pattern, take the newest, done. Contract folders defeat every heuristic because versioning is done by humans with inconsistent suffixes. The tool cannot know that _CLEAN marks the intended baseline and the un-suffixed file is the dirty draft; it just picks the first match and diffs confidently.

Edge cases

• Both candidate baselines may be legitimately needed (dirty vs clean comparison is itself a valid diff). The fix is not deleting files, it is making the choice explicit per run.
• Document IDs from the CLM or DMS beat filenames entirely. If the pipeline can resolve IDs, use them; filenames are a fallback, not a primary key.
• When the wrong-pair report already went to the counterparty, correct it explicitly. A phantom-changes redline erodes trust faster than a late one.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_rjmt9kFhhdIWgZhX0HVWkQ