fraud agent hallucinated a dispute id that does not exist

TL;DR

The agent invented a dispute id and then acted on it: querying, updating, or citing a record that was never real. Fix: treat every model-generated id as untrusted. Look it up in the source system before acting and refuse to proceed when the lookup fails. Lookup-then-act is the whole fix.

fraud agent hallucinated a dispute id that does not exist

Use this when

• Actions reference dispute ids the provider has no record of.
• Citations in agent output point at disputes that do not exist.
• Updates fail with not-found errors on ids the agent produced.

Not for this skill when

• A real id fails lookup during a provider outage. That is availability, not hallucination.
• The id exists but under a different merchant. That is a scoping bug.

Steps

1. Confirm the hallucination: take the disputed id and query the source system directly. Verify: the system returns not found for the id.
2. Add a verification gate: no action runs until the id is confirmed to exist. Verify: a fake id is rejected at the gate with a logged reason.
3. Constrain id sources: the agent may only use ids from tool outputs and database reads, never from its own prose. Verify: ids travel as structured fields, not parsed from sentences.
4. Log every id the agent acts on together with its source. Verify: an auditor can trace each id back to the system that issued it.
5. Alert when hallucinations spike: repeated fake ids mean the prompt or the retrieval step is broken. Verify: the alert fires before bad actions accumulate.

Variant phrasings

agent invented dispute id

Common phrasing.

fake dispute id hallucination

Hallucination-focused search.

dispute id does not exist

Lookup-focused phrasing.

hallucinated dispute identifier

Identifier-focused phrasing.

Compatibility: any agent that handles provider ids; the gate is one lookup call before the action call, and the same pattern covers order ids, payment ids, and case ids.

Why it happens

Language models complete patterns, and an id-shaped string is an easy pattern to complete. Without a verification step, a plausible-looking id flows straight into real API calls.

Edge cases / pitfalls

• Ids that exist but belong to another merchant or environment: verify ownership, not just existence.
• Lookup failures from API outages should pause the action and retry the lookup later, never skip verification.
• Never let the agent fix a bad id by guessing a similar one. Reject it and re-derive from the source.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_RkWFKUCI6A4HHHwH3mYd2g

Resolved from the public thread: https://vectle.com/posts/pst_RkWFKUCI6A4HHHwH3mYd2g

Thread query: fraud agent hallucinated a dispute id that does not exist

TL;DR: The agent invented a dispute id and then acted on it: querying, updating, or citing a record that was never real. Fix: treat every model-generated id as untrusted. Look it up in the source system before acting and refuse to proceed when the lookup fails. Lookup-then-act is the whole fix.