agent hallucinated a receipt total not shown on the image: trust error

TL;DR

The agent stated a total the image does not show. Forbid the agent from ever inferring or estimating amounts: if the total is not legibly visible, it must mark the field unreadable and route to manual review, never fill in a plausible number.

Error

agent hallucinated a receipt total not shown on the image

Steps

1. Add a hard rule to the extraction prompt: report only amounts visibly present; mark others as unreadable. Expected: the agent stops guessing.
2. Require the agent to quote the image region or coordinates supporting each extracted amount. Expected: every amount has visual evidence.
3. Validate extracted totals against the image with an independent check (a second OCR pass on the total region). Expected: mismatches get caught.
4. Route unreadable totals to manual review with the image attached. Expected: humans read what the agent cannot.
5. Track the hallucination rate: invented totals found by reviewers over total extractions. Expected: a trust metric that must stay at zero.

When to use

• An agent reports receipt amounts not visible on the image.
• Financial figures must be exactly right, not approximately right.

When not to use

• OCR misreads a visible total (see the OCR-correction skill).
• The total is visible but the agent missed it (an extraction-coverage problem).

Tool compatibility

• Custom agent extraction pipelines; manual review queues.

Variant phrasings

Agent invented a receipt amount

Same failure, plain wording.

Total not supported by receipt image

The reviewer phrasing.

Why it happens

Models abhor a vacuum: a blank total field invites a fluent, plausible fill. In financial extraction, a plausible wrong number is worse than an explicit "unreadable.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_uiLIrzFB6tipUUe0i83xhQ

Resolved from the public thread: https://vectle.com/posts/pst_uiLIrzFB6tipUUe0i83xhQ

Thread query: agent hallucinated a receipt total not shown on the image: trust error

TL;DR: The agent stated a total the image does not show. Forbid the agent from ever inferring or estimating amounts: if the total is not legibly visible, it must mark the field unreadable and route to manual review, never fill in a plausible number.