fraud agent's evidence pack download timed out on 200mb pdf batch

TL;DR

A 200 MB evidence batch will defeat a single in memory download with one global timeout every time, usually near the end. Change the shape of the work: list the documents, download them one at a time with streaming to disk and resumable requests, verify each file size and page count on arrival, and start OCR or review on completed files while the rest download. A failure then costs one document, which retries on its own, instead of the whole pack and the whole review session.

fraud agent's evidence pack download timed out on 200mb pdf batch

Use this when

• Evidence pack downloads die at large sizes or near completion.
• The batch mixes small documents with a few huge scanned bundles.
• The agent holds the whole pack in memory before processing.

Not for this skill when

• Downloads fail in seconds with 403 or 404. That is permissions or links, not size.
• The PDFs download fine and OCR fails on them. That is document quality, a different stage.

Steps

1. Reproduce with size logging: record each document size and where the current download stalls. Verify: you know the size threshold where failures start.
2. Rewrite the fetch to stream each file to disk individually with a per file timeout and progress logging, never buffering a full batch in memory. Verify: a single huge file downloads to disk without memory growth in monitoring.
3. Add integrity checks per file: expected size when provided, and a PDF page count read on arrival, logged to the case. Verify: each completed file shows size and pages on the case record.
4. Process incrementally: send each verified file to the next stage, OCR or review, as soon as it lands, and track batch completion as a per file checklist. Verify: review work starts before the last file finishes downloading in a test batch.
5. Retry failed files individually with backoff, and escalate only the specific document if it fails repeatedly, naming it in the escalation. Verify: a staged corrupt file is the only item escalated while the rest of the pack completes.

Variant phrasings

agent timeout downloading large evidence pdf

Large file phrasing.

chargeback evidence batch download fails size

Batch size phrasing.

streaming download evidence pack agent fix

Solution shape phrasing.

Compatibility: Any agent file retrieval from Stripe files, processor portals, S3, or case management systems. Resumable requests need server range support; where absent, per file downloads still contain the blast radius.

Why it happens

Evidence packs grow by accretion: one bank request bundles months of statements, scans at excessive resolution, and merged PDFs from several systems. Agent code written against tidy test files assumes downloads are fast and atomic, so it couples the whole case to the slowest, largest file and to a single timeout chosen for API calls, not file transfer.

Edge cases / pitfalls

• Verify downloaded PDFs are complete before OCR. A truncated PDF can open and silently drop its last pages.
• Watch disk, not just memory: streaming to disk trades one resource failure for another if nobody rotates temp files.
• Some portals hand out short lived signed URLs. Download promptly after minting, and re-mint per file on retry instead of reusing a dead link.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_sEdJU134ltZy3ZR1GWdlNA