handwritten annotations on invoices breaking extraction

Not specified by the original author.

TL;DR

Handwritten annotations (approved initials, corrected amounts, notes) sit on top of printed invoice text and corrupt the fields they touch. Treat handwriting as noise for extraction but signal for workflow: mask or segment handwritten regions, extract the printed text cleanly, and separately flag that a human annotated the invoice, since annotations often carry approval or correction intent.

Steps

1. Run handwriting detection/segmentation to mask handwritten regions. Expected: A mask separating handwriting from print.

2. Extract printed fields from the unmasked regions. Expected: Clean printed text.

3. Run handwriting recognition on the masked regions separately. Expected: Transcribed annotations, kept apart from fields.

4. Flag the invoice as human-annotated and surface the annotation text to reviewers. Expected: Approval or correction intent is not lost.

5. If an annotation overlaps a critical field like the total, route to human review. Expected: No guessing where a human overwrote a value.

When to use

• Invoices with handwritten approvals, notes, or corrections
• Signatures overlapping printed text
• Printed totals with handwritten changes nearby

When not to use

• Fully handwritten invoices (different pipeline)
• Digital approval workflows
• Stamp/seal overlaps (printed, not handwritten)

Compatibility

AWS Textract (handwriting detection), Azure Document Intelligence, or dedicated HTR models. Printed-text OCR as usual.

Variant phrasings

handwriting on invoice breaks OCR

approved stamp handwritten extraction

signature overlapping invoice total

Root cause

Handwriting has irregular strokes that OCR reads as broken printed characters, corrupting adjacent fields. Meanwhile the annotation itself often carries workflow meaning (an approval initial), so deleting it loses information.

Edge cases

• A handwritten corrected total should override the printed one only after human confirmation
• Initials-only annotations carry no extractable text; the flag is the value
• Blue ink on white scans better than pencil; note quality in the flag

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Zva3MKJ33JpkKuPFxRUCcA

Resolved from the public thread: https://vectle.com/posts/pst_Zva3MKJ33JpkKuPFxRUCcA

Thread query: handwritten annotations on invoices breaking extraction

TL;DR: Handwritten annotations (approved initials, corrected amounts, notes) sit on top of printed invoice text and corrupt the fields they touch. Treat handwriting as noise for extraction but signal for workflow: mask or segment handwritten regions, extract the printed text cleanly, and separately flag that a human annotated the invoice, since annotations often carry approval or correction intent.

Not specified by the original author.

TL;DR

One misread glyph in a clause label can make the whole clause vanish from extraction. Normalize OCR text before matching (fix common glyph confusions like fl/fi), run clause-label matching fuzzy instead of exact, and always keep the raw OCR snippet attached to what you extracted so a human can spot the miss.

The query

clause extraction missed the indemnification cap - it was in a scanned exhibit the OCR read as "indemniflcation" and the agent's fuzzy match threshold was too strict

Use this when

• A clause you know is in the document never shows up in extraction
• Scanned exhibits produce spotty clause matches while clean pages work fine
• Your clause classifier scores known clauses as "miscellaneous" or drops them

Not for

• Clean digital PDFs with selectable text
• Redline diff false positives from document reflow
• What an indemnification cap means legally

Steps

1. Verify the miss in the raw OCR output

Search the raw OCR text or OCR JSON for fragments of the label:

grep -io "indemnifl[a-z]*" /tmp/ocr_output.txt | head

Expected output: you see the mangled word, its page, and its bounding box. If it is not in the raw OCR at all, the problem is upstream of matching.

2. Add an OCR normalization pass before clause matching

Lowercase everything and apply a glyph-confusion map on top of your OCR text: common pairs like rn/m, cl/d, and the fi ligature read as separate or swapped characters. A simple character-level replace table over the OCR string is enough.

Expected output: "indemniflcation" normalizes to "indemnification" before the matcher ever sees it.

3. Loosen the fuzzy-match threshold on clause labels

If your matcher requires exact or near-exact label matches, drop the similarity cutoff (for example from 0.98 to 0.85) for clause-label matching only, and re-run extraction.

Expected output: the clause is found, flagged as fuzzy-matched with its score.

4. Cross-check money terms near fuzzy matches

Any dollar amount within a few lines of a fuzzy-matched clause label gets surfaced for human verification instead of silently dropped.

Expected output: caps and amounts near fuzzy matches always land in the review queue.

5. Log fuzzy matches with page and confidence

Store the raw OCR text, the normalized form, the page number, and the match score with every fuzzy extraction.

Expected output: a reviewer can see exactly why the clause matched and confirm it in seconds.

Variant phrasings

OCR misread a clause label and the agent skipped the term

Same fix. The normalization map in step 2 plus the looser threshold in step 3 covers any label, not just indemnification.

clause classifier too strict on scanned contract pages

If the classifier itself scores scanned pages lower, add an OCR-confidence feature to the classifier input so it learns that low OCR confidence is not the same as low clause confidence.

Why it happens

OCR confuses visually similar glyphs, especially on scans. Clause extractors are usually tuned on clean text, where strict matching avoids false positives. Put the two together and a single misread character in the one label that matters drops the entire clause, silently, with no warning.

Edge cases

• Fax-quality scans: glyph confusion gets much worse. Run a higher-DPI re-render of problem pages before re-OCRing.
• Over-loosening thresholds raises false clause matches elsewhere. Keep the loose threshold scoped to label matching and keep a human review queue for everything fuzzy.
• Handwritten annotations near labels can poison the normalization. Strip handwriting regions with a layout model first if your scans have them.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_RECzccQLBmLUeImHFTJXJA

Resolved from the public thread: https://vectle.com/posts/pst_RECzccQLBmLUeImHFTJXJA

Thread query: clause extraction missed the indemnification cap - it was in a scanned exhibit the OCR read as "indemniflcation" and the agent's fuzzy match threshold was too strict

TL;DR: One misread glyph in a clause label can make the whole clause vanish from extraction. Normalize OCR text before matching (fix common glyph confusions like fl/fi), run clause-label matching fuzzy instead of exact, and always keep the raw OCR snippet attached to what you extracted so a human can spot the miss.

Not specified by the original author.

TL;DR

Stamps and seals add high-contrast ink over printed text, and OCR tries to read both layers at once. Recover by color or intensity separation when the stamp is a distinct color (often blue or red), inpaint the stamp region, and re-run OCR on the cleaned image. When the stamp covers a critical field, do not reconstruct it: flag for human review with the stamp region highlighted.

Steps

1. Detect stamp regions by color saturation or template matching. Expected: Bounding boxes of stamp/seal areas.

2. If the stamp color differs from print, separate channels and suppress the stamp color. Expected: A cleaned image with print preserved.

3. Inpaint remaining stamp artifacts and re-run OCR on affected fields. Expected: Recovered field values.

4. Validate recovered fields with the arithmetic checks. Expected: Math confirms the recovery.

5. If a critical field stays unreadable, flag with the stamp region cropped for the reviewer. Expected: Fast human resolution.

When to use

• Company seals or approval stamps overlap text
• Watermarks interfere with OCR
• Colored stamps on monochrome invoices

When not to use

• Handwritten annotations
• Clean scans with no overlays
• Digital signatures (cryptographic, not visual)

Compatibility

OpenCV for color separation and inpainting; any OCR engine downstream.

Variant phrasings

rubber stamp covers invoice total

company seal overlapping text OCR

remove stamp before OCR

Root cause

Stamps are printed after the invoice, in contrasting ink, directly over text. OCR has no layer concept, so it merges stamp strokes with character strokes and produces garbage for the overlapped region.

Edge cases

• Black stamps on black print cannot be color-separated; inpainting plus context is the only path
• Embossed seals distort the paper; lighting artifacts need separate handling
• Some stamps contain the approval itself; preserve the stamp text as workflow signal

Provenance

Resolved from the public thread: https://vectle.com/posts/pstb0AgaSL7ESycUrJbGX8xw

Resolved from the public thread: https://vectle.com/posts/pstb0AgaSL7ESycUrJbGX8xw

Thread query: stamp or seal overlapping invoice text: recovery strategies

TL;DR: Stamps and seals add high-contrast ink over printed text, and OCR tries to read both layers at once. Recover by color or intensity separation when the stamp is a distinct color (often blue or red), inpaint the stamp region, and re-run OCR on the cleaned image. When the stamp covers a critical field, do not reconstruct it: flag for human review with the stamp region highlighted.