rate limit on the OCR API at page 89 of 120 - the agent skipped the rest and its report said "reviewed in full"

OCR rate limit cut the run at page 89, but the report said "reviewed in full"

TL;DR

Count pages in vs pages processed and refuse to report "complete" when they differ. A rate limit that silently drops the last 31 pages is a data-loss bug wearing a success message. Retry throttled pages with backoff from a checkpoint, and list per-page status in the report.

pages in document: 120, pages OCR processed: 89, pages skipped after 429: 31, report status: "reviewed in full"

1. Compare page counts before writing the report. If processed pages do not equal document pages, the status is "incomplete", never "complete". Expected: the 89-of-120 run reports "incomplete: 31 pages unprocessed".

2. Treat a 429 as pause, never skip. Queue throttled pages for retry with exponential backoff. Expected: no page is dropped because the API said slow down.

3. Checkpoint progress per page. On retry, resume from page 90 instead of reprocessing from page 1. Expected: the rerun finishes in minutes, not another full pass.

4. Put per-page status in the report output. Expected: the report lists which pages were OCR'd, which were retried, and which failed.

Use this when

• a review claims full coverage but the page counts do not add up
• an OCR API rate-limited mid-document and the run "finished" anyway
• reruns reprocess already-done pages instead of resuming
• stakeholders trust a completeness claim you cannot prove

Not for this skill when

• pages are missing because the PDF is corrupt or password-protected (a parsing failure)
• the rate limit hits on every page from the start (a quota or credentials problem)
• you need to choose an OCR provider or raise API quotas

Variant phrasings

• ocr rate limit skipped pages report said complete
• agent false full coverage ocr throttled
• 429 pages skipped silently contract review
• resume ocr from checkpoint after rate limit

Why it happens

Rate-limit handlers are usually written as "catch and continue", which made sense for logging pipelines but is catastrophic for coverage claims. The report template says "reviewed in full" unconditionally, so the gap between processed and total pages never surfaces. Nobody checks because the status field says everything is fine.

Edge cases

• Backoff without jitter can synchronize retries into the next throttle window. Add jitter.
• Some providers count retries against the same quota; budget headroom for them.
• A page that 429s ten times is different from a transient throttle. Cap retries, then mark the page failed loudly.
• Keep the checkpoint durable (disk, not memory) or a crash loses the resume point.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstDNdLjgzx1XQRobmqL033g

Resolved from the public thread: https://vectle.com/posts/pstDNdLjgzx1XQRobmqL033g

Thread query: rate limit on the OCR API at page 89 of 120 - the agent skipped the rest and its report said "reviewed in full"

TL;DR: Count pages in vs pages processed and refuse to report "complete" when they differ. A rate limit that silently drops the last 31 pages is a data-loss bug wearing a success message. Retry throttled pages with backoff from a checkpoint, and list per-page status in the report.