tabula read_pdf_with_template shifts columns on a scanned fee schedule

TL;DR

tabula's readpdfwith_template shifting columns on a scanned fee schedule happens because tabula reads text positions from the PDF - and a scanned PDF has no text positions, only images. The template's coordinates then map onto nothing, and columns drift. OCR the scan first (ocrmypdf adds a real, positioned text layer), then apply the same template to the OCR'd file. If columns still shift, re-measure the areas on the OCR'd version, because OCR can slightly change page geometry.

The query

tabula read_pdf_with_template shifts columns on a scanned fee schedule

Use this when

• tabula readpdfwith_template shifts or scrambles columns on a scanned schedule
• The template works on digital PDFs but fails on scanned ones
• Extracted columns contain text from neighboring columns

Not for

• Templates failing on digital PDFs - those need area re-measurement, not OCR
• tabula finding no tables at all - check the template areas and the PDF text layer first
• OCR'd PDFs where columns are right but rows are merged - a row-detection tuning problem

Steps

1. Prove the PDF has no text layer

Try selecting text in a PDF reader, or extract with a text tool. If nothing selects, tabula has no positions to work with and the template cannot function - this check takes ten seconds and prevents an hour of template tuning.

pdftotext scanned-fees.pdf - | head -5

Expected output: empty output, confirming the scan has no text layer.

2. OCR the scan to add a positioned text layer

Run ocrmypdf on the scanned file. The output is a PDF with the original images plus an invisible, positioned text layer - exactly what tabula needs.

ocrmypdf scanned-fees.pdf ocr-fees.pdf

Expected output: ocr-fees.pdf, in which text selects and pdftotext returns the fee rows.

3. Apply the template to the OCR'd file

Run readpdfwith_template against ocr-fees.pdf with your existing template. In most cases the columns now land correctly because the coordinates finally have text to attach to.

import tabula
tabula.read_pdf_with_template("ocr-fees.pdf", "template.json", pages="all", output_path="fees.csv")

Expected output: fees.csv with columns aligned to the visible schedule grid.

4. Re-measure areas if columns still drift

OCR can deskew and slightly rescale pages, moving content relative to the template's coordinates. Open the OCR'd PDF in tabula's template designer (or measure with pdfplumber) and adjust the areas, then re-run. Expected output: the adjusted template produces clean columns on the OCR'd file.

5. Bake OCR into the pipeline before tabula

Make the OCR step unconditional for scans: detect text-layer presence at ingest, OCR when absent, then run tabula. Never let a raw scan reach the template step again. Expected output: scanned schedules flow through the same template path as digital ones.

Tool compatibility

tabula-py with a Java runtime; ocrmypdf for the OCR step; pdftotext (poppler) for the text-layer check. The template JSON format is unchanged - only the input PDF changes.

Variant phrasings

• tabula readpdfwith_template shifts columns scanned fee schedule
• tabula template wrong columns scanned PDF
• tabula OCR PDF before template extraction
• tabula columns drift scanned schedule no text layer

Why it happens

tabula is a positional extractor: it places the template's rectangles onto the PDF's text objects and reads what falls inside. A scanned PDF contains images, not text objects, so the rectangles land on empty space and tabula grabs whatever fragments are nearest - producing shifted, interleaved columns. OCR inserts the missing text objects with real coordinates, which is the only input tabula's algorithm can consume.

Edge cases / pitfalls

• OCR quality bounds extraction quality: a bad OCR pass gives well-positioned garbage - check the OCR text first.
• Handwritten annotations on the scan become text objects too and can pollute columns; mask or accept them.
• Multi-page scans need per-page OCR; a single bad page's geometry can throw off a shared template.
• Keep the original scan archived - the OCR'd file is a working copy, and re-OCRing with better settings later is normal.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstt-LkhAgHu049qL7FkIu1w

Resolved from the public thread: https://vectle.com/posts/pstt-LkhAgHu049qL7FkIu1w

Thread query: tabula readpdfwith_template shifts columns on a scanned fee schedule

TL;DR: tabula's readpdfwith_template shifting columns on a scanned fee schedule happens because tabula reads text positions from the PDF - and a scanned PDF has no text positions, only images. The template's coordinates then map onto nothing, and columns drift. OCR the scan first (ocrmypdf adds a real, positioned text layer), then apply the same template to the OCR'd file. If columns still shift, re-measure the areas on the OCR'd version, because OCR can slightly change page geometry.