agent's image preprocessor crashed with memory error on 50mb scans

TL;DR

A 50 MB scan decompresses to far more in memory, crashing the preprocessor. Downscale to a working resolution (around 300 DPI) before processing, stream the image instead of loading it fully, and process pages one at a time.

Error

agent's image preprocessor crashed with memory error on 50mb scans

Steps

1. Check the failing image's dimensions and color depth. Expected: a very large image (high megapixels, 48-bit color) that explodes in memory when decompressed.
2. Downscale to a working size before any processing: resize so the long edge is about 3500 pixels. Expected: the working image is a few megabytes, still plenty for OCR.
3. Convert to grayscale early in the pipeline. Expected: memory use drops by two thirds with no OCR accuracy loss on receipts.
4. Process multi-page scans one page at a time, releasing each page before loading the next. Expected: peak memory stays flat regardless of page count.
5. Add a pre-check: if an image exceeds the working size, downscale automatically and log it. Expected: future large scans are handled without crashing.

When to use

• Image preprocessing crashes with out-of-memory on large scans.
• The crash correlates with file size, not image content.

When not to use

• OCR is slow but does not crash (performance tuning, different fix).
• Small images crash the preprocessor (a code bug, not a size problem).

Tool compatibility

• Pillow, OpenCV-based preprocessing pipelines in custom agents.

Variant phrasings

Preprocessor out of memory on large scan

Same crash, shorter wording.

Image too large to preprocess

The size-check phrasing of the same problem.

Why it happens

Compressed file size lies: a 50 MB JPEG can decompress to hundreds of megabytes of pixels. Preprocessors that load the full bitmap before resizing blow the memory budget on the load step itself.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_aQoZP0ngizDhBByhc50AGA