agent hallucinated a matching account that doesn't exist in the crm: verification error
Catches agents hallucinating matching accounts: verify every claimed match with a live read-back by id before acting. Use when merges or updates target phantom records. Not for fuzzy-match tuning.
TL;DR
The agent claimed a matching account that does not exist. Never act on a claimed match without verifying it: read the record back by id from the CRM, and only proceed if the read succeeds and the fields match the claim. Verification is a separate step, not an optimization.
Error
Verification failed: claimed matching account id 001... returned "record not found" on read-back.Steps
- Take the agent's claimed account id and read it directly from the CRM. Expected: either the record or a not-found.
- If not found, reject the match and log the hallucination with the agent's reasoning. Expected: a recorded miss, not a silent skip.
- If found, compare key fields (name, domain) against what the agent claimed. Expected: confirmation the right record was meant.
- Only then proceed with the merge or update. Expected: actions target real records.
- Add read-back verification as a mandatory gate in the agent loop for every claimed match. Expected: hallucinations caught before damage.
When to use
- An agent acts on accounts that do not exist.
- Merge or update targets fail with not-found.
- After model changes, hallucinations spike.
When not to use
- Fuzzy matching accuracy (different tuning task).
- Records deleted between match and action (race, different fix).
Tool compatibility
- Salesforce and HubSpot record read APIs.
- Agent verification gates.
Variant phrasings
phantom account match
Read back by id before acting.
claimed match not found
Reject and log the hallucination.
Why it happens
Language models complete patterns plausibly, including inventing ids that look right. Without a grounding read, the plausible fiction flows straight into a destructive call.
Edge cases
- Read-back must use the id, not a re-search by name; re-searching can hallucinate twice.
- Cache the verification briefly; do not re-verify the same id dozens of times.
- Track hallucination rates per model version; regressions show up here first.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_TuS8CaqRLR9hwduq-YBYaw