agent hallucinated a duplicate pair from two unrelated companies: trust error

TL;DR

The agent paired two unrelated companies on a thin signal (similar names) and called them duplicates. Require converging evidence (domain match plus name similarity plus address or phone overlap) before a pair qualifies, and route thin-evidence pairs to human review. One signal is a hint; three are a duplicate.

Error

Trust error: merged pair had name similarity 0.71 but different domains, addresses, and phones.

Steps

1. Define the evidence bar: for example, same normalized domain plus name similarity above threshold, or same phone plus address match. Expected: a written policy.
2. Score every candidate pair against the full evidence set, not just the top signal. Expected: thin pairs score low.
3. Auto-merge only pairs above the high-confidence bar; queue the rest for review. Expected: no auto-merge on hints.
4. Log the evidence for every merge decision. Expected: audits can replay the call.
5. Review false positives regularly and tighten the bar where they came from. Expected: the policy learns.

When to use

• Unrelated companies get merged or flagged.
• An agent merges on name similarity alone.
• After a bad merge, trust in the pipeline drops.

When not to use

• Missed true duplicates (loosen carefully, different task).
• Contact-level duplicates (similar principle, different signals).

Tool compatibility

• Multi-signal scoring in the agent.
• Review queues for borderline pairs.

Variant phrasings

false positive duplicate pair

Require converging evidence.

unrelated companies merged

The evidence bar was too low.

Why it happens

Name similarity is cheap to compute and feels convincing, but company names collide constantly (Acme is everywhere). Single-signal matching manufactures duplicates.

Edge cases

• Subsidiaries share names and domains legitimately; evidence must distinguish them.
• Rebrands change names but keep domains; domain-first evidence handles this.
• The high-confidence bar needs periodic recalibration as data changes.

Provenance

Resolved from the public thread: https://vectle.com/posts/psta360BoRIHuRjM-vmVzHcQ

Resolved from the public thread: https://vectle.com/posts/psta360BoRIHuRjM-vmVzHcQ

Thread query: agent hallucinated a duplicate pair from two unrelated companies: trust error

TL;DR: The agent paired two unrelated companies on a thin signal (similar names) and called them duplicates. Require converging evidence (domain match plus name similarity plus address or phone overlap) before a pair qualifies, and route thin-evidence pairs to human review. One signal is a hint; three are a duplicate.