agent cited section '14.3' for the indemnity clause - the contract only goes to 14.2 and the model hallucinated the...

TL;DR

The agent cited section 14.3 for the indemnity clause, but the contract only goes to 14.2: the model hallucinated the number. Build the section index from the document's actual headings, validate every citation against it, and reject any citation that does not resolve. Never let a model-generated section number through unchecked.

The symptom

agent cited section '14.3' for the indemnity clause  -  the contract only goes to 14.2 and the model hallucinated the number

Steps

1. Build the real section index

Parse the document headings for section numbers. The index ends at 14.2; 14.3 is not in it. Expected: Authoritative list of sections that exist.

2. Validate every citation

Check each cited section against the index. 14.3 fails validation immediately. Expected: Hallucinated citations flagged, not published.

3. Find the real indemnity clause

Search the clause text (indemnify, hold harmless) and read its actual heading number. Expected: The correct section number for the indemnity clause.

4. Add the citation gate

Require all citations to resolve against the section index before any output ships. Unresolvable citations block the report. Expected: No hallucinated section number can reach a reviewer.

When to use

• A cited section does not exist in the document
• Citations came from model generation rather than extraction
• Clause-type lookups (indemnity, liability) by section number

When not to use

• The numbering is genuinely unusual (verify against the index, then trust it)
• Citations were extracted from cross-reference text (validate those too)
• The document has no numbered sections (cite by heading text instead)

Why it happens

Language models complete patterns: asked for 'the indemnity clause section', the model produces a plausible-looking number like 14.3 whether or not it exists. Without grounding citations in the document's actual structure, plausible fiction ships as fact.

Edge cases

• Appendices with separate numbering: keep a separate index so lookups do not cross over.
• Renumbered sections across versions: validate against the current version's index.
• Defined terms pointing at sections: validate those references the same way.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_tOsH6qXhCfGeY8CAXva8WQ

Resolved from the public thread: https://vectle.com/posts/pst_tOsH6qXhCfGeY8CAXva8WQ

Thread query: agent cited section '14.3' for the indemnity clause - the contract only goes to 14.2 and the model hallucinated the number

TL;DR: The agent cited section 14.3 for the indemnity clause, but the contract only goes to 14.2: the model hallucinated the number. Build the section index from the document's actual headings, validate every citation against it, and reject any citation that does not resolve. Never let a model-generated section number through unchecked.