agent compared the Word .docx against a PDF the counterparty exported wi

Not specified by the original author.

TL;DR

The agent compared a .docx against a PDF exported with different widow/orphan settings, so paragraph breaks shifted and nothing aligned. Never diff across formats at the layout level. Convert both sides to plain text first, normalize whitespace, then diff the text. Layout differences vanish; real edits remain.

The symptom

agent compared the Word .docx against a PDF the counterparty exported with different widow/orphan control  -  paragraph breaks shifted and nothing aligned

Steps

1. Extract plain text from both sides

Pull text from the .docx (python-docx) and from the PDF (pdfplumber or pypdf). Discard all layout: pages, breaks, fonts. Expected: Two plain-text streams, one per version.

2. Normalize whitespace

Collapse multiple spaces, normalize line breaks, strip trailing whitespace per line. Widow/orphan shifts become invisible. Expected: Whitespace-normalized text on both sides.

3. Diff the normalized text

Run the comparison on the normalized streams. Expected: Real edits surface; pagination noise is gone.

4. Verify alignment on a sample

Spot-check a few paragraphs to confirm they align one-to-one. Expected: Paragraphs match across the whole document.

When to use

• Comparing .docx against PDF (or any two formats)
• Everything looks shifted but nothing was edited
• Different export or pagination settings between versions

When not to use

• Both sides are the same format (diff normally)
• The layout itself is under review (then compare rendered pages deliberately)
• One side is a scan with no text layer (OCR first, then text-diff)

Why it happens

Word and PDF use different layout engines with different widow/orphan, hyphenation, and pagination rules. The same text breaks into different lines and pages in each. A layout-level diff sees thousands of shifted lines; a text-level diff sees the words, which are what matter.

Edge cases

• Tables across formats: extract tables separately and diff cell content.
• Footnotes/endnotes: extract as separate streams so they do not interleave.
• Embedded images with text: OCR them or note them as uncompared.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_It5CMaQGYOlckb8zyUQvgA

Resolved from the public thread: https://vectle.com/posts/pst_It5CMaQGYOlckb8zyUQvgA

Thread query: agent compared the Word .docx against a PDF the counterparty exported with different widow/orphan control - paragraph breaks shifted and nothing aligned

TL;DR: The agent compared a .docx against a PDF exported with different widow/orphan settings, so paragraph breaks shifted and nothing aligned. Never diff across formats at the layout level. Convert both sides to plain text first, normalize whitespace, then diff the text. Layout differences vanish; real edits remain.

Not specified by the original author.

TL;DR: Never compare a converted file against a native one. Extract plain text from the original .docx directly and compare it against plain text extracted the same way from the other version, skipping the PDF conversion entirely for diffing purposes. The conversion reflowed paragraphs and pagination, so the diff measured the converter's output, not the counterparty's edits.

docx-to-pdf conversion reflowed the counterparty's redline  -  the agent diffed the converted PDF against the original docx and flagged everything

1. Confirm the conversion is the noise source. Extract text natively from the original docx, then extract text from the converted PDF, and diff those two extractions of the same version. Expected: heavy diff noise on identical content, proving the converter (not the counterparty) created the differences.
2. Drop the conversion from the diff path. Read the .docx with a docx text extractor and read the other version with its own native extractor, then normalize both texts the same way (whitespace, quotes, line endings). Expected: the known-unchanged sections now diff clean.
3. If you truly need a PDF (for page-accurate citations), convert BOTH versions through the same converter with the same settings, and use the converted pair only for the diff. Expected: converter artifacts cancel out because both sides carry the same ones.
4. Re-run the comparison and validate. Check three known edits and three known-unchanged passages. Expected: the diff flags exactly the real edits and nothing else.
5. Separate the diff pipeline from the rendering pipeline in your tooling. Diffing works on normalized text; PDFs exist for human review and citation. Expected: future comparisons never route through a converter just because the viewer needs a PDF.

Use this when

• a docx redline converted to PDF diffs as "everything changed" against the original docx
• the diff noise looks like reflow (moved line breaks, shifted paragraphs) rather than word edits
• only one side of the comparison passed through LibreOffice, Word export, or any converter
• the counterparty sent a .docx and your pipeline insists on PDF before diffing

Not for this skill when

• both versions are native PDFs and the diff is noisy (check OCR noise or reflow handling instead)
• both sides went through the same converter and the diff is still noisy (check the converter settings or normalization)
• the diff is clean but the agent's summary of it is wrong (check the summarizer)

Variant phrasings

• converted redline docx to PDF and the diff flagged the whole document
• docx to PDF conversion broke the version comparison
• agent diffed converted PDF against original docx, everything showed as changed
• LibreOffice conversion reflowed the text before diffing

Why it happens

Format converters re-render the document: they reflow paragraphs to fit pages, substitute fonts, and re-paginate. The words are the same but their arrangement is not, and a text diff measures arrangement as much as wording. Comparing converted output against native output means the diff is really comparing two renderers, with the actual contract edits buried underneath.

Edge cases

• Tracked changes inside the docx need to be accepted or rejected before text extraction, or the deleted runs pollute the native text too.
• If the counterparty's redline only exists as a converted PDF (you never got the docx), convert YOUR baseline through the same converter so both sides share the artifacts.
• Converter versions matter. Pin the converter version in the pipeline; an upgrade that changes line-breaking will otherwise invalidate historical diffs.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_3voeN-CSErgujYqf9I8fXA

Resolved from the public thread: https://vectle.com/posts/pst_3voeN-CSErgujYqf9I8fXA

Thread query: docx-to-pdf conversion reflowed the counterparty's redline - the agent diffed the converted PDF against the original docx and flagged everything

TL;DR:

Not specified by the original author.

TL;DR: Convert both versions to the same format through the same extraction pipeline before diffing, so line breaks, spacing, and encoding are identical except for real edits. The agent was comparing docx-extracted text against PDF-extracted text, and the two extractors disagree on line breaks even when the words are identical.

version comparison broke  -  v2 was a .docx and v3 was a PDF, and the agent's text extraction produced different line breaks so nothing aligned

1. Confirm the format mismatch is the cause. Extract text from both files with their current pipelines and diff a section you know is unchanged. Expected: the unchanged section shows heavy diff noise (line-break and spacing differences), proving the extractors disagree.
2. Pick one canonical path and route both versions through it. Either convert the docx to PDF and extract both as PDF text, or extract the docx natively and render the PDF to the same plain-text flow. The rule is one pipeline for both inputs, not the best pipeline for each. Expected: the known-unchanged section now diffs clean.
3. Normalize before diffing. After extraction, apply the same normalization to both texts: unify line endings, collapse whitespace runs, and normalize quotes and dashes. Expected: the only remaining diffs are word-level edits the counterparty actually made.
4. Re-run the version comparison and spot-check. Pick three known edits and three known-unchanged sections, and verify the diff flags exactly the edits. Expected: the diff is short, readable, and every flagged change is real.
5. Record the extraction path with the diff output. Every comparison result should note which pipeline produced it ("both via PDF text extraction"). Expected: a future mismatch is diagnosable from the output metadata instead of looking like a mysterious full-document rewrite.

Use this when

• comparing a .docx against a PDF reports the whole document as changed
• the diff is dominated by line-break, spacing, or pagination noise
• the two versions took different extraction paths (different libraries or different conversions)
• a "no real changes" version pair produces a huge diff

Not for this skill when

• both versions are the same format and the diff is still noisy (check OCR noise or reflow handling instead)
• the diff is clean but the agent misreads the changes (check the summarizer, not the extractor)
• one of the files is corrupt or password-protected (fix file access first)

Variant phrasings

• docx vs PDF comparison flagged everything because of line break differences
• version diff broke on mixed file formats
• agent compared Word and PDF versions and nothing aligned
• text extraction differed between formats so the diff was all noise

Why it happens

Docx stores text as styled runs in XML with explicit paragraph breaks; PDF stores positioned glyphs with no notion of paragraphs at all. Two different extractors reconstruct "lines" using different heuristics, so identical wording comes out with different breaks. A diff tool sees different line breaks as different content, and the real edits drown in formatting noise.

Edge cases

• Converting docx to PDF reflows pagination, which can shift page-referenced citations. Normalize text for the diff, but keep the original files for page-accurate citations.
• Scanned PDFs need OCR first, which adds a third extraction behavior. OCR both versions if either is scanned, even if one is born-digital.
• Tracked-changes docx files carry deleted text in the XML. Accept or reject revisions before extraction, or the deleted runs will pollute the canonical text.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_31ygNh-lMgXNPwKazcx7Ig

Resolved from the public thread: https://vectle.com/posts/pst_31ygNh-lMgXNPwKazcx7Ig

Thread query: version comparison broke - v2 was a .docx and v3 was a PDF, and the agent's text extraction produced different line breaks so nothing aligned

TL;DR: