agent hallucinating department names that do not exist: generated wrong values and validation failed with error

TL;DR

Constrain the agent to source values only and validate every invented-prone field against the system of record before it writes. The agent generated department names that do not exist because a required field was missing or ambiguous in its input and it filled the gap plausibly instead of stopping. Ground the step with a direct lookup, reject any value not traceable to the source, and make missing data a halt with a clear request, never a creative writing exercise.

agent hallucinating department names that do not exist: generated wrong values and validation failed

Use this when

• The agent output contains values that appear in no source record, such as invented assignments, people, or stock items.
• Failures happen on hires whose source records have missing or unusual fields the agent must interpret.
• Validation in the target rejects the generated values, or worse, occasionally accepts them.

Not for this skill when

• The source record itself holds the wrong value entered by a person. Correct the source data process instead.
• The agent copies a correct value into the wrong field. That is a mapping bug, not hallucination.

Steps

1. Capture the failing case completely: the agent input record, the exact values it generated, and its reasoning trace if available, and mark each generated value as present in the source, derived by a stated rule, or invented. Verify: every generated value is classified with evidence
2. Find the gap that invited invention: the missing, blank, or ambiguous source field the agent needed for this step, such as an absent assignment list, an unnamed approver, or an unchecked availability. Verify: the triggering gap is identified in the input record
3. Ground the step: give the agent a lookup tool against the authoritative list in ServiceNow, for example the real assignment catalogue, people directory, or inventory, and require it to select only values the lookup returns. Free-text generation for these fields is disabled by the step design. Verify: the step can only emit values returned by the lookup in a test with a missing field
4. Add a validation gate before any write: each critical value is checked against the source record or the lookup result, and any mismatch or untraceable value halts the step with a specific message naming the field and the hire. Verify: the gate blocks a seeded invented value in testing
5. Change the missing-data behaviour in the agent instructions and step logic: when a required value is absent, the agent stops that hire, records the exact missing field, and queues the hire for a human data fix, then continues with other hires. Verify: a test hire with a missing field is queued with the field named, not invented around
6. Re-run the affected hires once their source gaps are fixed, confirm the create the ServiceNow onboarding ticket step uses only traceable values, and audit recent completed runs for silently accepted invented values that need correction in ServiceNow. Verify: the audit finds no remaining untraceable values or lists them as owned cleanups

Why it happens

Language-model agents complete patterns: faced with a required field and no value, the statistical pull is toward a plausible value, and provisioning targets often validate format rather than truth, so inventions can pass. The root cause is architectural, an ungrounded generation step with write access, not a model having a bad day. Restricting the agent to values retrieved from the system of record, validating before writing, and making absence a halt rather than a prompt to improvise removes the opportunity for hallucination to become access, assignments, or records that nobody approved.

Variants and edge cases

Invented values pass target validation

Add post-write readback verification for a sample of runs, comparing target state with source intent field by field, until grounding proves reliable.

Agent invents only under time pressure prompts

Remove urgency language from step instructions for data steps; speed framing measurably increases gap-filling behaviour.

Lookup itself returns stale data

Grounding is only as good as the catalogue. Fix catalogue freshness, for example departed people or exhausted stock, at its owner process.

Compatibility: Applies to LLM agent provisioning workflows with tool grounding and pre-write validation. Requires agent workflow editor access.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_TJ-DBBDKdXzIFH-JlHGsbQ