agent hallucinated seat counts from a truncated CSV, audit failed
Stops agents from hallucinating seat counts off truncated CSVs by validating export completeness first and marking missing data as unknown. Use when an agent invents numbers from partial data. Not for complete exports with wrong counts.
TL;DR
The agent filled gaps in a truncated file with plausible numbers. Never let it invent data URIs validate the export is complete before counting, re-fetch when it is not, and mark unfetchable values as unknown, not as guesses.
agent hallucinated seat counts from a truncated CSV, audit failedUse this when
- Seat counts look suspiciously round or convenient.
- The source CSV was truncated.
- The agent produced numbers without a complete input.
Not for this skill when
- The export is complete and counts are wrong. That is a logic problem.
- The numbers are estimates by design. Then label them as estimates.
- The agent refused to answer. That is over-caution, a different tuning.
Steps
- Check the export for truncation: abrupt end, row count versus expected. Verify: completeness is assessed.
- If truncated, re-fetch the export; do not proceed on partial data. Verify: the new export is whole.
- Mark values that cannot be fetched as unknown in the output. Verify: unknowns are labeled.
- Add a completeness check before any counting step. Verify: the check gates the count.
- Confirm every reported number traces to a data row. Verify: the audit is grounded.
Variant phrasings
hallucinated counts
The short phrasing.
truncated CSV
The cause-first search.
agent invented data
The behavior-first phrasing.
Compatibility: Any agent producing counts from exported files.
Why it happens
Language models pattern-complete. A cut-off CSV looks like a complete one to a model that is not checking, so it continues the pattern with invented rows.
Edge cases / pitfalls
- Silent truncation producing a valid but short CSV; the row-count check is what catches it.
- The agent helpfully estimating when asked for a number; the instruction must forbid it.
- Partial exports from a previous run being treated as current; timestamp the inputs.
- Unknowns being unacceptable to the requester; then the answer is to fix the export, not to guess.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_Iopitkgrlw7aUuuMQGVDsg