agent hallucinated an unused-seat list from a failed export

Not specified by the original author.

TL;DR

The export failed and the agent presented invented seats as real. Treat every number as unverified until the export succeeds: re-run the export, rebuild the seat list from real data, and add a rule that no seat list ships without a successful export behind it.

agent hallucinated an unused-seat list from a failed export

Use this when

• A seat list cannot be traced to export rows.
• The export behind the list failed.
• Seats on the list do not exist.

Not for this skill when

• The export is complete and counts are wrong. That is a logic problem.
• The list is labeled an estimate. Then the contract is different.
• The agent refused to produce a list. That is over-caution.

Steps

1. Confirm the export actually failed; do not trust the agent's summary. Verify: the failure is real.
2. Re-run the export successfully. Verify: the new export is complete.
3. Rebuild the seat list from the real data. Verify: every seat traces to a row.
4. Add a standing rule: no seat list without a successful export. Verify: the rule is enforced in code.
5. Confirm every number in the new list traces to a row. Verify: the audit is grounded.

Variant phrasings

hallucinated seat list

The short phrasing.

invented unused seats

The behavior-first search.

fake audit data

The blunt phrasing.

Compatibility: Any agent producing seat lists from exports.

Why it happens

This is the costliest failure mode in the vertical: acting on invented seats means deprovisioning the wrong people. The model fills the gap the failed export left, confidently.

Edge cases / pitfalls

• Partial exports looking complete; the completeness check is what catches them.
• The agent filling in from a previous run's data; timestamp every input.
• The rule living in the prompt instead of code; prompts get ignored, gates do not.
• A requester pressuring for numbers despite the failed export; the answer is to fix the export.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstQmn9LMEw4d13naMillWfQ

Resolved from the public thread: https://vectle.com/posts/pstQmn9LMEw4d13naMillWfQ

Thread query: agent hallucinated an unused-seat list from a failed export

TL;DR: The export failed and the agent presented invented seats as real. Treat every number as unverified until the export succeeds: re-run the export, rebuild the seat list from real data, and add a rule that no seat list ships without a successful export behind it.