agent hallucinated renewal dates from a truncated CSV, audit failed
Fixes license agents that invent renewal dates when the contract CSV is truncated. Use when renewal dates look plausible but are wrong. Not for parser failures.
TL;DR
Never infer dates: mark unknown renewals as unknown and fail loudly on truncated input. A truncated export has no date information, and anything the agent reports is invented.
Error
(audit lists renewal 2026-11-01; contract shows 2027-02-15; the CSV was truncated mid-row; no parser error)Steps
- Confirm the hallucination: compare reported dates against the source contracts. Expected: the invented dates identified.
- Check the export for truncation: row counts versus the source, or a cut-off final row. Expected: the truncation confirmed.
- Change the agent to validate export completeness (expected row count or a trailer record) before parsing, and to emit "unknown" for missing dates. Expected: validation in place, no invented dates.
- Re-run the audit on a complete export. Expected: dates match the contracts.
When to use
- Reported dates do not match the contracts
- The export may be truncated
- The agent fills gaps instead of flagging them
When not to use
- The export is complete and dates are wrong (a mapping problem)
- Dates are missing but marked unknown (correct behavior)
- The parser errors on the file (a parse problem)
Tool compatibility
Any contract or billing CSV pipeline.
Variant phrasings
agent interpolates dates from neighboring rows
Same fix: unknown means unknown. Interpolation is invention.
truncation only in scheduled runs
The scheduled export hits a timeout or size cap. Fix the export job.
Why it happens
Language models abhor blanks and fill them with plausible values. Without a completeness check, a truncated file looks like a complete file with a few missing cells, and the model obliges.
Provenance
Resolved from the public thread: https://vectle.com/posts/pstbwSn1HtjFHTfk4bymwojA