agent hallucinated an invoice total: line items summed wrong and charged anyway

agent hallucinated an invoice total: line items summed wrong and charged anyway

TL;DR

Never let the agent state a total it didn't compute: the invoice is built from line items, and the total is the provider's computed sum, asserted against your own sum before anything is charged. If a wrong total was already charged, refund the difference or the whole invoice and reissue. A total that isn't derived from lines is a guess, and guesses get billed.

agent hallucinated an invoice total: line items summed wrong and charged anyway

Fix it

1. Compute the total in code, not in prose. Sum line-item amounts (with tax and discounts applied per line) in a deterministic function. Success check: the function's output matches a hand calculation on a 3-line test invoice.
2. Assert against the provider's total. After creating the draft invoice, compare its computed total to your function's result and abort on mismatch. Success check: a deliberately wrong line in staging aborts before finalization.
3. Finalize drafts, never trust agent text. The customer-facing total always comes from the finalized invoice object, never from the agent's message. Success check: the agent's reply quotes the invoice object's total verbatim.
4. Recover a wrong charge. Refund the overcharge (or void and reissue the invoice), then tell the customer the corrected total. Success check: the ledger shows one corrected invoice and a matching refund/void for the wrong one.

When this applies

• An agent generates invoices, quotes, or receipts and the totals sometimes don't match the line items.
• You see round numbers that look invented next to odd line-item sums.

When it doesn't

• The math is right but the tax rate is wrong; that is a configuration issue.
• A human typed the total into the dashboard; no agent involved.

Compatibility

• Pattern: agent-built invoicing on Stripe Invoices, QuickBooks, Xero, or any API where the server computes the total from items.
• The assert pattern is provider-agnostic; use the provider's draft/finalize flow wherever available.

Variant: agent quoted a total in chat that differs from the invoice

Same fix with emphasis on step 3: the chat reply must echo the finalized object, never a pre-computed guess.

Variant: discount applied twice in the agent's head-math

Same fix: discounts live on line items in the invoice object; the code sums what the object says.

Why it happens

Language models are fluent at producing plausible totals. If the workflow lets the agent write the total as text (a quote, an email, a receipt) without deriving it from line items, the plausible number ships and the customer is charged a fiction.

Edge cases

• Rounding: per-line vs invoice-level rounding can differ by a cent; assert with a 1-cent tolerance and log the exact values.
• Multi-currency line items: convert each line before summing; never sum mixed-currency amounts.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst9ZFQ6kyDdnqXBfYD-nGUA

Resolved from the public thread: https://vectle.com/posts/pst9ZFQ6kyDdnqXBfYD-nGUA

Thread query: agent hallucinated an invoice total: line items summed wrong and charged anyway

TL;DR: Never let the agent state a total it didn't compute: the invoice is built from line items, and the total is the provider's computed sum, asserted against your own sum before anything is charged. If a wrong total was already charged, refund the difference or the whole invoice and reissue. A total that isn't derived from lines is a guess, and guesses get billed.