intel agent hallucinating competitor pricing, validation failed

Intel agent hallucinating competitor pricing, validation failed

TL;DR

Invented competitor prices are worse than missing prices because they look authoritative in a briefing. The fix is the same grounding rule as financial figures: every price must trace to a fetched page, feed, or vendor record, and untraced prices get dropped. Validate prices against a second source when the briefing depends on them.

The error

(validation failed)
intel agent hallucinating competitor pricing; claimed $99/seat, source page shows $149/seat

When this helps

• an agent invents competitor prices
• pricing claims fail source checks
• building pricing validation
• auditing competitive intel quality

When it doesn't

• the source page changed; re-fetch before calling it hallucination
• prices are estimates by design; label them as such
• you need real-time prices; validation adds latency

Works with

python 3.8+ with re and json. Any pricing source.

Steps

1. Extract every claimed price with its source

import re
brief = open("brief_draft.md").read()
claims = re.findall(r"\$([0-9,.]+)\s*(?:/|per\s+)?(seat|mo|month)?\s*\[([^\]]+)\]", brief)
print("price claims:", len(claims))

Expected: Claimed prices with citations. Uncited prices are dropped immediately.

2. Verify each price against its cited source

import json
checks = [{"price": "$99/seat", "source_price": "$149/seat", "match": False}]
open("price_checks.json", "w").write(json.dumps(checks, indent=2))
print("mismatches flagged for removal")

Expected: A verification record. The source of truth is the fetched page, not the agent's memory.

3. Drop unverified prices and note the gap

import json
checks = json.load(open("price_checks.json"))
kept = [c for c in checks if c["match"]]
print("kept", len(kept), "prices; gaps noted explicitly in the briefing")

Expected: A brief with only verified prices. A noted gap beats a fabricated number.

4. Cross-check critical prices against a second source

import json
print("for prices the briefing depends on: verify against a vendor feed or second fetch")
print("single-source prices carry a confidence note")

Expected: A cross-check policy. High-stakes numbers earn two sources.

Other ways people phrase this

agent hallucinating competitor pricing

Every price traces to a source. No trace, no publish.

competitor price validation failed

The check working as designed. Fix generation, keep the check.

invented prices briefing agent

Verification is a pipeline stage. Cross-check high-stakes numbers twice.

Why it happens

Models invent prices when the briefing template demands a number the sources did not provide. The invented figure looks as confident as a real one. Source-tracing every price turns fabrication into a caught error, and explicit gaps keep the briefing honest.

Edge cases

• Prices change; a verified price needs an as-of date.
• Promotional prices differ from list; record which one the source showed.
• Currency and billing period must match between claim and source.
• A price the source removed is a gap, not a license to keep the old number.
• Vendor list prices and checkout prices differ; record which one the source showed.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_QyrSeQ51dWpv2TFR8iZoMQ