agent hallucinated a ticket id linking related tickets

Not specified by the original author.

TL;DR

An agent that "links related tickets" from memory invents ids that look right and point nowhere. Never let the agent emit a ticket id it did not read from the API in this run: link by searching first (find the real ticket, then link its real id), verify every id against the API before writing the link, and treat any id from model memory as untrusted. Ids come from the system of record, never from the model's imagination.

The query

agent hallucinated a ticket id linking related tickets

Use this when

• Related-ticket links point to wrong tickets
• Agents cite ticket ids that do not exist
• You are auditing agent output accuracy
• Building related-ticket automation

Not for

• Designing ticket link UI
• Deciding which tickets are related
• Tuning search relevance
• Ticket merge strategy

Steps

1. Require every linked id to come from an API read

The rule is absolute: an id the agent did not fetch this run cannot be written. Enforce it in the tool design (link tools accept only ids returned by search tools) rather than in the prompt. Structural enforcement beats instructions.

Expected output: link tools accepting only API-sourced ids.

2. Link by search, not by memory

The workflow is search-then-link: query for the related ticket, read the result's id, link that id. The search grounds the link in reality; skipping the search is where hallucination enters.

Expected output: search-then-link as the only linking path.

3. Verify each id exists before writing the link

Read the ticket by id before linking to it. The read confirms existence and gives the agent the real title to reference. A link written without verification is a guess with formatting.

Expected output: existence verification on every linked id.

4. Surface the verification in the agent's output

When the agent links tickets, show what it verified: "Linked ticket 4821 (verified: 'Refund not received')." Visible verification lets humans spot the rare miss and builds trust in the common hit.

Expected output: verified links with titles in agent output.

5. Track hallucination rate on links

Sample linked ids and check them against the API. The rate should be zero; any non-zero rate means the grounding discipline has a hole. Investigate the hole, do not just fix the instance.

Expected output: a link-accuracy metric at zero.

Template: the checklist

Ticket link grounding:
[ ] Links accept only API-read ids (structural)
[ ] Search-then-link workflow enforced
[ ] Existence verified before writing each link
[ ] Verification visible in agent output
[ ] Link accuracy tracked, target zero misses

Variant phrasings

agent invented ticket id

Steps 1 and 2. API-sourced ids, search first.

related ticket link wrong

Steps 3 and 4. Verify, show the verification.

ai hallucinating ticket references

Steps 1 and 5. Structural grounding plus tracking.

Why it happens

Language models are trained to produce plausible text, and ticket ids are short plausible-looking strings: the model completes the pattern confidently whether or not the ticket exists. The agent has no internal sense of "I do not know this id"; it has only fluency. Grounding discipline supplies externally what the model lacks internally: a check against reality before writing.

Edge cases

• Ids from earlier in a long run: still verify; tickets can be merged or deleted mid-run. Freshness matters.
• Cross-system ids: an id valid in one system (the shop) is not valid in another (the helpdesk). Verify in the target system.
• The search returning the wrong ticket: verification by title catches mis-searched links. Read what you link.
• Agents quoting ids from ticket text: ids mentioned by customers are claims, not facts. Verify them too.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstrhYF9kxSwyfVN4ujwK6Qw

Resolved from the public thread: https://vectle.com/posts/pstrhYF9kxSwyfVN4ujwK6Qw

Thread query: agent hallucinated a ticket id linking related tickets

TL;DR: An agent that "links related tickets" from memory invents ids that look right and point nowhere. Never let the agent emit a ticket id it did not read from the API in this run: link by searching first (find the real ticket, then link its real id), verify every id against the API before writing the link, and treat any id from model memory as untrusted. Ids come from the system of record, never from the model's imagination.