agent hallucinated compliance requirement not in rfp

agent hallucinated compliance requirement not in rfp: how to fix it

Agents hallucinate requirements when they summarize from memory instead of the document. Force every matrix row to carry a verbatim quote and page reference, and reject rows that cannot cite one.

agent hallucinated compliance requirement not in rfp

When to use this skill

• Compliance matrix lists requirements nobody can find
• Agent cites requirement numbers that do not exist

When not to use this skill

• Agent misreads a real requirement (a comprehension issue)
• Matrix misses real requirements (a recall issue)

Tool compatibility

Agent compliance grounding and citations. Related area: agent evaluations.

Related area: proposal automation and agent-assisted RFP workflows.

Cloud UIs change labels over time; if a button moved, search settings for the closest match.

Before you start

• Sign in to the tool and keep one tab open so sessions stay clean
• Copy the exact error text and note the time it happened
• Set aside 15 to 20 minutes; most of these resolve in a single pass

Quick diagnosis

Make sure this is the right skill before you invest the 15 minutes.

• If you are dealing with compliance matrix lists requirements nobody can find, start at step 1 below.
• If instead agent misreads a real requirement (a comprehension issue), stop here; you need a different skill.
• If neither quite matches, compare your screen against the verbatim error block at the top; that block is exactly what this skill covers.

Fix it in steps

1. Require a quote per matrix row

Add a source-quote column; every row must quote the RFP verbatim.

Expected: Expected: rows without quotes are visibly incomplete.

2. Add page references

Record the page number beside each quote.

Expected: Expected: every requirement traces to a page.

3. Reject uncited rows

Delete or flag any row that cannot produce a quote.

Expected: Expected: phantom requirements disappear from the matrix.

4. Spot check five rows

Open the RFP to the cited pages for five random rows.

Expected: Expected: all five quotes exist on those pages.

Confirm each step worked

Run through this checklist before moving on. If a check fails, redo that step instead of pushing ahead.

• Step 1 (require a quote per matrix row): Expected: expected: rows without quotes are visibly incomplete.
• Step 2 (add page references): Expected: expected: every requirement traces to a page.
• Step 3 (reject uncited rows): Expected: expected: phantom requirements disappear from the matrix.
• Step 4 (spot check five rows): Expected: expected: all five quotes exist on those pages.

Quick recap

1. Require a quote per matrix row
2. Add page references
3. Reject uncited rows
4. Spot check five rows

Still stuck?

Gather three things: the exact error text, the time it happened, and what changed just before it broke. Check the vendor status page for an active incident. Then open a support ticket with those three things; it cuts the back-and-forth in half. For agent runs, attach the last 50 lines of the agent log too, since the loop or timeout signature in the log usually names the cause.

Variant phrasings

agent hallucinated compliance requirement

Quote-per-row grounding fixes it. If this matches what you saw, the steps above apply as written.

rfp agent invented requirements

Reject uncited rows. If this matches what you saw, the steps above apply as written.

Why it happens

Summarization without citation lets the model fill gaps with plausible inventions, and compliance language is formulaic enough that inventions look real. A mandatory quote column makes invention structurally impossible.

Edge cases

• Review the rejected rows; some are real requirements the agent phrased badly
• Page numbers shift between RFP versions; cite the version too
• A second agent pass that only verifies citations catches what the first pass invents

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_qzmHtbFqfDndTWMgSGtYQw