agent diffed the OCR text of both versions — OCR noise on both sides and every page looked changed

Not specified by the original author.

TL;DR: Denoise and normalize both OCR outputs before diffing (same OCR engine, same settings, aggressive whitespace and glyph normalization), and diff at the clause level rather than the line level. The agent was diffing raw OCR text, so every misread character on either side registered as a contract change.

agent diffed the OCR text of both versions  -  OCR noise on both sides and every page looked changed

1. Confirm OCR noise is the cause. OCR the same page twice with the current settings and diff the two outputs. Expected: the identical page diffs against itself with dozens of character-level differences, proving the noise floor is above zero.
2. Pin the OCR pipeline. Run both versions through the same engine with identical settings (same DPI rendering, same language model, same page-segmentation mode). Never OCR one version at 300 DPI and the other at 200. Expected: re-running OCR on the same file now produces byte-stable output.
3. Normalize both texts before diffing. Lowercase nothing (legal casing can matter), but normalize whitespace runs, unify quote and dash characters, and map common OCR glyph confusions to a canonical form (ligature variants, section-sign misreads). Expected: the same-page self-diff from step 1 comes back clean.
4. Diff at clause or sentence granularity, not line or character level. Align the two texts on section headings first, then diff within matched sections. Expected: real edits show up as changed clauses while OCR jitter inside unchanged clauses stays below the reporting threshold.
5. Set a noise floor and report honestly. If a section's diff is below the character-difference threshold you measured in step 1, mark it unchanged. Log the threshold with the diff output. Expected: the final diff lists only changes bigger than the OCR noise floor.

Use this when

• diffing two scanned versions flags the entire document as changed
• the same file OCR'd twice produces a non-empty diff against itself
• the two versions were OCR'd with different engines or settings
• character-level diffs on scanned contracts are unreadable

Not for this skill when

• both versions are born-digital PDFs with clean text layers (there is no OCR noise; check reflow or format mismatch instead)
• the diff is clean but the agent misreads the changes (check the summarizer)
• only one side is scanned (OCR just the scanned side and compare against the digital text, watching for the noise floor)

Variant phrasings

• OCR noise made every page of the version diff look changed
• diffed scanned contracts and got a full-document diff of nothing
• OCR jitter on both sides drowned out the real redline edits
• version comparison of scanned PDFs flagged everything

Why it happens

OCR is non-deterministic across runs and across engines: the same glyph can be read differently depending on rendering DPI, neighboring characters, and model version. Diffing raw OCR output treats each of those misreads as an edit. With two noisy sides, the noise compounds, and a handful of real changes hide inside thousands of phantom ones.

Edge cases

• If the scan quality differs a lot between versions (one crisp, one a fax), the noise floors differ too. Consider re-scanning or flagging low-confidence pages rather than trusting the diff there.
• Numbers are the highest-risk OCR output (a misread cap looks like a negotiated change). Route any numeric diff through a human or a second OCR pass before reporting.
• Tables OCR as jumbled text. Reconstruct tables separately (or compare them as images) rather than diffing their OCR text inline.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_O2wPNIlgKiJ4rcGXPyYGpw

Resolved from the public thread: https://vectle.com/posts/pst_O2wPNIlgKiJ4rcGXPyYGpw

Thread query: agent diffed the OCR text of both versions - OCR noise on both sides and every page looked changed

TL;DR:

Not specified by the original author.

TL;DR

The agent extracted the embedded text layer AND ran OCR over the same pages, so every clause appeared twice and the diff counted the duplicates as changes. Pick one source of truth: check each page for a usable text layer first, and only OCR the pages that have none. Add a dedupe guard that drops exact-duplicate clauses.

The symptom

agent parsed both the PDF text layer and a fresh OCR pass  -  every clause appeared twice and the diff counted duplicates as changes

Steps

1. Test each page for a usable text layer

Extract text per page with pypdf. Pages returning more than a trivial character count (say, 50+ characters) have a real text layer; pages returning nothing are scanned images. Expected: Each page labeled text-layer or scan.

2. OCR only the scan pages

Run OCR exclusively on pages with no text layer. Never OCR a page that already gave you good text. Expected: Each page contributes exactly one text source.

3. Add a dedupe guard before diffing

Hash the normalized text of each extracted clause. If two clauses from the same page have identical hashes, keep the first and drop the second, and log the drop. Expected: No clause appears twice in the extraction output.

4. Re-run the diff and sanity-check the count

Diff the deduped extraction against the baseline. The change count should now match the counterparty's stated changes within a small margin. Expected: Duplicate-driven phantom changes disappear from the diff.

When to use

• Every clause appears twice in the extraction
• Diffs report changes on clauses nobody touched
• The pipeline runs both a text extractor and an OCR engine unconditionally

When not to use

• The document is a pure scan with no text layer (OCR everything, no duplication possible)
• Duplicates come from the source document itself (repeated exhibits, different fix)
• Only some clauses duplicate (look for a partial re-OCR of specific pages)

Why it happens

Many pipelines run text extraction and OCR as independent stages and concatenate the results. On a born-digital PDF both stages succeed on the same pages, so the output contains each clause twice. The diff then sees each duplicate pair as an addition, inflating the change count.

Edge cases

• Partial text layers (a scanned signature page in a digital doc): per-page detection handles this; whole-document detection does not.
• Text layer exists but is garbage (bad encoding): treat the page as a scan and OCR it, but still only once.
• Two-column text layers: dedupe on normalized text, not on raw block order.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_nKecT5YpAyRPHXR7ElM8qQ

Resolved from the public thread: https://vectle.com/posts/pst_nKecT5YpAyRPHXR7ElM8qQ

Thread query: agent parsed both the PDF text layer and a fresh OCR pass - every clause appeared twice and the diff counted duplicates as changes

TL;DR: The agent extracted the embedded text layer AND ran OCR over the same pages, so every clause appeared twice and the diff counted the duplicates as changes. Pick one source of truth: check each page for a usable text layer first, and only OCR the pages that have none. Add a dedupe guard that drops exact-duplicate clauses.

Not specified by the original author.

TL;DR The final version dropped the DRAFT watermark and the agent flagged its removal as a substantive change. Watermarks, stamps, and banners are revision markings, not contract language, so strip them from both versions before diffing. The redline should cover the agreement, not the stationery.

What the bogus change looked like:

- DRAFT (watermark, every page) marked as removed content

Steps

1. Confirm the flagged change is only the watermark. Check whether the diff's watermark entries correspond to any actual clause text. Command: inspect the flagged regions and verify they contain only the watermark text with no clause language. Expected: every watermark flag is pure marking, no operative text involved.

2. Add a pre-diff cleanup pass that removes watermarks, stamps, and banner text from both versions: drop diagonal or background watermark text layers, and strip repeated header and footer strings. Expected: re-running the diff no longer mentions the watermark at all.

3. Keep a separate revision-markings report: record which markings each version carried, like DRAFT on v3 and none on v4, as metadata rather than diff content. Expected: the redline summary covers clause changes, with a one-line note that the DRAFT watermark was removed in the final.

4. Generalize the cleanup to other markings your pipeline will meet: CONFIDENTIAL banners, page stamps, and Bates numbers. Expected: future draft-to-final comparisons stay clean regardless of which markings change.

Use this when

• Draft-to-final comparisons flag watermark or stamp removal as changes
• Diffs include header, footer, or banner text that is not part of the agreement
• The counterparty's versions carry different revision markings
• Your redline change list is polluted with non-clause differences

Not for this skill when

• The marking is part of the executed agreement, like a notary stamp on the signature page, which deserves its own handling
• Watermark text overlaps clause text in a scanned PDF, clean the image rather than the text
• You need a forensically complete diff, keep the raw diff archived
• The only difference between versions genuinely is the watermark, then say so plainly

Variant phrasings

• agent flagged DRAFT watermark removal as a substantive change
• how to exclude watermarks from contract diffs
• redline showing watermark differences instead of real edits
• draft to final comparison polluted by revision markings
• strip headers footers and watermarks before diffing contracts

Why it happens

Watermarks and stamps live in the document as text or image layers, so a naive diff sees "DRAFT" present in one version and absent in the other and reports it like any other deletion. The diff has no concept of document furniture versus operative language, so revision markings get the same weight as a changed indemnity cap. Humans instantly discount the watermark; the agent needs to be taught to.

Edge cases

• A watermark that obscures clause text in a scan is a readability problem, flag the obscured region rather than silently stripping it.
• Bates numbers change on every page of every version, always exclude them or they will dominate the diff.
• Some stamps carry dates, like a received stamp, which can matter for timing disputes, log stamp metadata even while excluding it from the clause diff.
• Headers containing the agreement title may legitimately change if the title changed, compare header text as metadata before stripping.
• If the final version should NOT have lost its watermark, like a confidential draft that went out unmarked, that is a process failure worth escalating, not diff noise.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_c6HgCLNHg0B0CYCa2oIvaA

Resolved from the public thread: https://vectle.com/posts/pst_c6HgCLNHg0B0CYCa2oIvaA

Thread query: agent flagged the "DRAFT" watermark removal as a substantive change in the final version

TL;DR: