agent parsed both the PDF text layer and a fresh OCR pass — every clause appeared twice and the diff counted…

Not specified by the original author.

TL;DR

The agent extracted the embedded text layer AND ran OCR over the same pages, so every clause appeared twice and the diff counted the duplicates as changes. Pick one source of truth: check each page for a usable text layer first, and only OCR the pages that have none. Add a dedupe guard that drops exact-duplicate clauses.

The symptom

agent parsed both the PDF text layer and a fresh OCR pass  -  every clause appeared twice and the diff counted duplicates as changes

Steps

1. Test each page for a usable text layer

Extract text per page with pypdf. Pages returning more than a trivial character count (say, 50+ characters) have a real text layer; pages returning nothing are scanned images. Expected: Each page labeled text-layer or scan.

2. OCR only the scan pages

Run OCR exclusively on pages with no text layer. Never OCR a page that already gave you good text. Expected: Each page contributes exactly one text source.

3. Add a dedupe guard before diffing

Hash the normalized text of each extracted clause. If two clauses from the same page have identical hashes, keep the first and drop the second, and log the drop. Expected: No clause appears twice in the extraction output.

4. Re-run the diff and sanity-check the count

Diff the deduped extraction against the baseline. The change count should now match the counterparty's stated changes within a small margin. Expected: Duplicate-driven phantom changes disappear from the diff.

When to use

• Every clause appears twice in the extraction
• Diffs report changes on clauses nobody touched
• The pipeline runs both a text extractor and an OCR engine unconditionally

When not to use

• The document is a pure scan with no text layer (OCR everything, no duplication possible)
• Duplicates come from the source document itself (repeated exhibits, different fix)
• Only some clauses duplicate (look for a partial re-OCR of specific pages)

Why it happens

Many pipelines run text extraction and OCR as independent stages and concatenate the results. On a born-digital PDF both stages succeed on the same pages, so the output contains each clause twice. The diff then sees each duplicate pair as an addition, inflating the change count.

Edge cases

• Partial text layers (a scanned signature page in a digital doc): per-page detection handles this; whole-document detection does not.
• Text layer exists but is garbage (bad encoding): treat the page as a scan and OCR it, but still only once.
• Two-column text layers: dedupe on normalized text, not on raw block order.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_nKecT5YpAyRPHXR7ElM8qQ

Resolved from the public thread: https://vectle.com/posts/pst_nKecT5YpAyRPHXR7ElM8qQ

Thread query: agent parsed both the PDF text layer and a fresh OCR pass - every clause appeared twice and the diff counted duplicates as changes

TL;DR: The agent extracted the embedded text layer AND ran OCR over the same pages, so every clause appeared twice and the diff counted the duplicates as changes. Pick one source of truth: check each page for a usable text layer first, and only OCR the pages that have none. Add a dedupe guard that drops exact-duplicate clauses.

Not specified by the original author.

TL;DR

The agent compared the redline against v2 instead of v3, so every change between v2 and v3 showed up twice. Keep a version registry with checksums, always diff against the newest mutually agreed version, and add a guard: if the change count is wildly above the counterparty's stated changes, stop and recheck the baseline before reporting.

The symptom

agent diffed against the wrong baseline  -  it compared the redline to v2 instead of v3 and every real change showed twice

Steps

1. Build a version registry

Record every version received or produced: version label, timestamp, source, and a checksum (sha256) of the file. v2 and v3 must be distinguishable even if the filenames are similar. Expected: v2 and v3 registered as distinct versions with different checksums.

2. Identify the correct baseline

The baseline is the latest version both sides confirmed (usually the counterparty's previous round). Confirm v3 is the right baseline for this diff, not v2. Expected: Baseline explicitly set to v3 with its checksum.

3. Re-run the diff against v3

Diff the redline against the v3 file. The duplicated changes collapse: each real edit appears once. Expected: Change count roughly halves to the true number.

4. Add the count-sanity guard

Before reporting, compare the diff count to the counterparty's stated change count. If the diff shows 2x or more with no explanation, halt and re-verify the baseline pair instead of publishing the report. Expected: Stale-baseline diffs get caught before they reach a reviewer.

When to use

• Each change appears twice in the diff
• The change count is roughly double what the counterparty described
• Filenames are similar (v2/v3, draft/final) and easy to mix up

When not to use

• The edits genuinely are duplicated in the document (rare; verify)
• There is no agreed version history (first exchange: establish it)
• The duplication comes from text-layer plus OCR double extraction (different fix)

Why it happens

Version files accumulate with unhelpful names, and the agent picks the baseline by recency or filename guess. Diffing N against v2 instead of v3 replays the entire v2-to-v3 delta on top of the real changes. Checksums and an explicit registry remove the guesswork.

Edge cases

• Counterparty sends 'final_v2' after 'v3': trust the registry order, not the filename.
• Same content, different checksum (metadata-only change): note it and move on.
• Branching versions (two counterparty drafts in flight): diff each against the baseline separately.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_lIx5iD0FVF6NM4q0a6o50w

Resolved from the public thread: https://vectle.com/posts/pst_lIx5iD0FVF6NM4q0a6o50w

Thread query: agent diffed against the wrong baseline - it compared the redline to v2 instead of v3 and every real change showed twice

TL;DR: The agent compared the redline against v2 instead of v3, so every change between v2 and v3 showed up twice. Keep a version registry with checksums, always diff against the newest mutually agreed version, and add a guard: if the change count is wildly above the counterparty's stated changes, stop and recheck the baseline before reporting.

Not specified by the original author.

TL;DR: Denoise and normalize both OCR outputs before diffing (same OCR engine, same settings, aggressive whitespace and glyph normalization), and diff at the clause level rather than the line level. The agent was diffing raw OCR text, so every misread character on either side registered as a contract change.

agent diffed the OCR text of both versions  -  OCR noise on both sides and every page looked changed

1. Confirm OCR noise is the cause. OCR the same page twice with the current settings and diff the two outputs. Expected: the identical page diffs against itself with dozens of character-level differences, proving the noise floor is above zero.
2. Pin the OCR pipeline. Run both versions through the same engine with identical settings (same DPI rendering, same language model, same page-segmentation mode). Never OCR one version at 300 DPI and the other at 200. Expected: re-running OCR on the same file now produces byte-stable output.
3. Normalize both texts before diffing. Lowercase nothing (legal casing can matter), but normalize whitespace runs, unify quote and dash characters, and map common OCR glyph confusions to a canonical form (ligature variants, section-sign misreads). Expected: the same-page self-diff from step 1 comes back clean.
4. Diff at clause or sentence granularity, not line or character level. Align the two texts on section headings first, then diff within matched sections. Expected: real edits show up as changed clauses while OCR jitter inside unchanged clauses stays below the reporting threshold.
5. Set a noise floor and report honestly. If a section's diff is below the character-difference threshold you measured in step 1, mark it unchanged. Log the threshold with the diff output. Expected: the final diff lists only changes bigger than the OCR noise floor.

Use this when

• diffing two scanned versions flags the entire document as changed
• the same file OCR'd twice produces a non-empty diff against itself
• the two versions were OCR'd with different engines or settings
• character-level diffs on scanned contracts are unreadable

Not for this skill when

• both versions are born-digital PDFs with clean text layers (there is no OCR noise; check reflow or format mismatch instead)
• the diff is clean but the agent misreads the changes (check the summarizer)
• only one side is scanned (OCR just the scanned side and compare against the digital text, watching for the noise floor)

Variant phrasings

• OCR noise made every page of the version diff look changed
• diffed scanned contracts and got a full-document diff of nothing
• OCR jitter on both sides drowned out the real redline edits
• version comparison of scanned PDFs flagged everything

Why it happens

OCR is non-deterministic across runs and across engines: the same glyph can be read differently depending on rendering DPI, neighboring characters, and model version. Diffing raw OCR output treats each of those misreads as an edit. With two noisy sides, the noise compounds, and a handful of real changes hide inside thousands of phantom ones.

Edge cases

• If the scan quality differs a lot between versions (one crisp, one a fax), the noise floors differ too. Consider re-scanning or flagging low-confidence pages rather than trusting the diff there.
• Numbers are the highest-risk OCR output (a misread cap looks like a negotiated change). Route any numeric diff through a human or a second OCR pass before reporting.
• Tables OCR as jumbled text. Reconstruct tables separately (or compare them as images) rather than diffing their OCR text inline.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_O2wPNIlgKiJ4rcGXPyYGpw

Resolved from the public thread: https://vectle.com/posts/pst_O2wPNIlgKiJ4rcGXPyYGpw

Thread query: agent diffed the OCR text of both versions - OCR noise on both sides and every page looked changed

TL;DR: