paragraph numbering misread — OCR turned '7.7' into '1.1' and t

Not specified by the original author.

TL;DR

OCR misread 7.7 as 1.1, and the cross-reference linker trusted the raw text and pointed at section 1.1. Rebuild the section index from actual headings with a strict numbering pattern, resolve every cross-reference against that index, and flag any reference that matches nothing. Re-OCR the suspect regions at higher quality to fix the underlying misread.

The symptom

paragraph numbering misread  -  OCR turned '7.7' into '1.1' and the agent's cross-reference linker pointed at the wrong section

Steps

1. Rebuild the section index from headings

Parse the document for heading patterns like '7.7', 'Section 7.7', or '7.7 ' at line starts. Build the authoritative list of sections that actually exist. 7.7 should be there; 1.1 may not be. Expected: A section list matching the document's real structure.

2. Resolve references against the index, not raw text

For each cross-reference ('see Section X'), look X up in the index. If X is not in the index, do not link it; mark it unresolved instead of linking to a wrong section. Expected: No link points at a section that does not exist.

3. Re-OCR the suspect number regions

Crop the regions around misread numbers and re-OCR at higher DPI or with a different engine. 7 vs 1 confusion usually clears up at 300+ DPI with a clean crop. Expected: 7.7 reads as 7.7; the index and the references agree.

4. Add a confusion-matrix post-pass

For number-like tokens in references, check common OCR confusions (7/1, 8/6, 0/O) against the index: if '1.1' is not a section but '7.7' is, prefer the index member and flag the correction. Expected: References resolve to real sections even when the OCR wobbles.

When to use

• Cross-references point at sections that do not exist
• Section numbers in references differ by one confused glyph from a real section
• The linker followed raw OCR text without validating against the document structure

When not to use

• The document's numbering is genuinely broken or duplicated (needs human review)
• References are correct but the target text moved (different problem)
• The document has no numbered sections (nothing to index)

Why it happens

Digits like 7 and 1 are visually close at scan resolution, so OCR confuses them. A linker that trusts raw tokens then resolves '1.1' literally, landing on the wrong section or a nonexistent one. Validating against the document's actual section structure catches the misread before it becomes a wrong link.

Edge cases

• Multi-level numbering (7.7.3): validate each level; partial matches are still wrong.
• Appendices with letter numbering (A.1): keep a separate index so digit confusions do not cross over.
• Hand-amended numbers in redline: the newest number wins; note the amendment.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_lzMLqkU5NEE0KX9KylCGGw

Resolved from the public thread: https://vectle.com/posts/pst_lzMLqkU5NEE0KX9KylCGGw

Thread query: paragraph numbering misread - OCR turned '7.7' into '1.1' and the agent's cross-reference linker pointed at the wrong section

TL;DR: OCR misread 7.7 as 1.1, and the cross-reference linker trusted the raw text and pointed at section 1.1. Rebuild the section index from actual headings with a strict numbering pattern, resolve every cross-reference against that index, and flag any reference that matches nothing. Re-OCR the suspect regions at higher quality to fix the underlying misread.

Not specified by the original author.

TL;DR

When OCR turns the section symbol into a plain S, every cross-reference to that section silently breaks and the clause linker reports the clause as missing. Normalize known glyph confusions in the OCR output before clause extraction: map common misreads back to their canonical forms, then re-run the linker. Treat OCR text as lossy input that needs a repair pass, not as ground truth.

The query

agent couldn't find the governing-law clause - OCR turned "§ 14.2" into "S 14.2" in every cross-reference and the section linker broke

Use this when

• A clause the contract contains cannot be found by the extraction agent.
• Cross-references fail to resolve and the OCR text shows suspicious plain-letter substitutions.
• The source is a scanned PDF processed by OCR.

Not for

• Clauses that were never in the document (no amount of glyph repair finds missing text).
• Digital PDFs with a clean text layer (no OCR involved).
• Reflowed-diff false positives between document versions.

Steps

Step 1: Confirm the glyph confusion pattern

grep -o "S [0-9][0-9]*\.[0-9]" ocr-output.txt | sort | uniq -c | head -10

Expected output: repeated "S 14.2" style hits where the scanned image shows a section symbol. Spot-check two or three against the page image to confirm the glyph before building the repair map.

Step 2: Build the glyph-repair map

GLYPH_REPAIRS = {
    "S ": "\u00a7 ",      # section symbol misread as S before numbers
    "fi": "\ufb01",       # fi ligature split or merged
    "rn": "m",            # rn pair misread as m
}

Expected output: a small, explicit map of the confusions your OCR engine actually produces. Derive it from observed errors, not from imagination. Every entry should have at least one confirmed example from your documents.

Step 3: Apply the repair before clause extraction

python repair_glyphs.py --in ocr-output.txt --out ocr-repaired.txt

Expected output: the repaired text with section symbols restored. Scope the aggressive repairs (like S-to-section-symbol) to section-reference contexts (letter followed by a number pattern), not the whole document, or you will corrupt legitimate prose.

Step 4: Re-run the section linker

python link_sections.py --input ocr-repaired.txt --report unlinked.txt

Expected output: the governing-law clause now resolves, and unlinked.txt shrinks. Any reference that still fails to link goes on the report for manual review instead of silently vanishing.

Step 5: Add a dangling-reference check to the pipeline

Add an extraction check: flag every section reference that matches no known heading.

Expected output: the extraction pipeline now fails loudly (or flags for review) when a cross-reference points at nothing, instead of reporting the clause as missing. The next glyph confusion surfaces as a warning, not a silent drop.

Step 6: Verify critical clauses against the page image

Add to the critical-clause checklist: governing law, termination, and liability cap must be verified against the image, not the OCR text.

Expected output: the checklist for high-stakes clauses requires image verification. OCR text is the index. The image is the source of truth.

Variant phrasings

limitation of liability read as limitation of fiability

Same fi-ligature family. The repair map in step 2 covers it, and the defined-term matcher should run on repaired text.

OCR read confidential as confidentiaI with a capital i

Same glyph-confusion class (l/I/1 family). Add it to the map once confirmed against the image.

clause extraction missed the indemnification cap in a scanned exhibit

Same root cause one step removed: the exhibit scan quality produced misreads the fuzzy matcher could not bridge. Repair first, then match.

Why it happens

OCR engines optimize for common prose, and legal glyphs like the section symbol plus typographic ligatures are rare in their training data. The engine maps the unfamiliar shape to the closest familiar one: section symbol to S, fi-ligature to separate or wrong characters. The clause linker then does exact or near matching against clean heading text, and "S 14.2" never matches "section 14.2." The clause was there all along. The lookup key was corrupted.

Edge cases

• Do not apply the S-to-section-symbol repair globally. Ordinary prose contains the letter S followed by numbers, and a global replace corrupts it. Scope repairs to reference-shaped contexts.
• Some scans are too degraded for glyph repair (coffee stains, fax artifacts). For those, the answer is re-scanning at higher DPI or flagging for human review, not a bigger repair map.
• Different OCR engines produce different confusions. A map built for Tesseract will miss Textract's errors. Build the map per engine.
• Handwritten margin notes produce junk characters no map can fix. Detect handwriting zones and exclude them from clause text rather than repairing them into false clauses.
• Repair changes the text the model sees. Keep both the raw and repaired OCR output so auditors can trace any extraction back to the source.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstF7hlR0nZ1htIOhd4bNZ-g

Resolved from the public thread: https://vectle.com/posts/pstF7hlR0nZ1htIOhd4bNZ-g

Thread query: agent couldn't find the governing-law clause - OCR turned "§ 14.2" into "S 14.2" in every cross-reference and the section linker broke

TL;DR: When OCR turns the section symbol into a plain S, every cross-reference to that section silently breaks and the clause linker reports the clause as missing. Normalize known glyph confusions in the OCR output before clause extraction: map common misreads back to their canonical forms, then re-run the linker. Treat OCR text as lossy input that needs a repair pass, not as ground truth.

Not specified by the original author.

TL;DR

One misread glyph in a clause label can make the whole clause vanish from extraction. Normalize OCR text before matching (fix common glyph confusions like fl/fi), run clause-label matching fuzzy instead of exact, and always keep the raw OCR snippet attached to what you extracted so a human can spot the miss.

The query

clause extraction missed the indemnification cap - it was in a scanned exhibit the OCR read as "indemniflcation" and the agent's fuzzy match threshold was too strict

Use this when

• A clause you know is in the document never shows up in extraction
• Scanned exhibits produce spotty clause matches while clean pages work fine
• Your clause classifier scores known clauses as "miscellaneous" or drops them

Not for

• Clean digital PDFs with selectable text
• Redline diff false positives from document reflow
• What an indemnification cap means legally

Steps

1. Verify the miss in the raw OCR output

Search the raw OCR text or OCR JSON for fragments of the label:

grep -io "indemnifl[a-z]*" /tmp/ocr_output.txt | head

Expected output: you see the mangled word, its page, and its bounding box. If it is not in the raw OCR at all, the problem is upstream of matching.

2. Add an OCR normalization pass before clause matching

Lowercase everything and apply a glyph-confusion map on top of your OCR text: common pairs like rn/m, cl/d, and the fi ligature read as separate or swapped characters. A simple character-level replace table over the OCR string is enough.

Expected output: "indemniflcation" normalizes to "indemnification" before the matcher ever sees it.

3. Loosen the fuzzy-match threshold on clause labels

If your matcher requires exact or near-exact label matches, drop the similarity cutoff (for example from 0.98 to 0.85) for clause-label matching only, and re-run extraction.

Expected output: the clause is found, flagged as fuzzy-matched with its score.

4. Cross-check money terms near fuzzy matches

Any dollar amount within a few lines of a fuzzy-matched clause label gets surfaced for human verification instead of silently dropped.

Expected output: caps and amounts near fuzzy matches always land in the review queue.

5. Log fuzzy matches with page and confidence

Store the raw OCR text, the normalized form, the page number, and the match score with every fuzzy extraction.

Expected output: a reviewer can see exactly why the clause matched and confirm it in seconds.

Variant phrasings

OCR misread a clause label and the agent skipped the term

Same fix. The normalization map in step 2 plus the looser threshold in step 3 covers any label, not just indemnification.

clause classifier too strict on scanned contract pages

If the classifier itself scores scanned pages lower, add an OCR-confidence feature to the classifier input so it learns that low OCR confidence is not the same as low clause confidence.

Why it happens

OCR confuses visually similar glyphs, especially on scans. Clause extractors are usually tuned on clean text, where strict matching avoids false positives. Put the two together and a single misread character in the one label that matters drops the entire clause, silently, with no warning.

Edge cases

• Fax-quality scans: glyph confusion gets much worse. Run a higher-DPI re-render of problem pages before re-OCRing.
• Over-loosening thresholds raises false clause matches elsewhere. Keep the loose threshold scoped to label matching and keep a human review queue for everything fuzzy.
• Handwritten annotations near labels can poison the normalization. Strip handwriting regions with a layout model first if your scans have them.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_RECzccQLBmLUeImHFTJXJA

Resolved from the public thread: https://vectle.com/posts/pst_RECzccQLBmLUeImHFTJXJA

Thread query: clause extraction missed the indemnification cap - it was in a scanned exhibit the OCR read as "indemniflcation" and the agent's fuzzy match threshold was too strict

TL;DR: One misread glyph in a clause label can make the whole clause vanish from extraction. Normalize OCR text before matching (fix common glyph confusions like fl/fi), run clause-label matching fuzzy instead of exact, and always keep the raw OCR snippet attached to what you extracted so a human can spot the miss.