agent hallucinating answers not in source data for summarize deviations in a redlined NDA error hallucination failure

Not specified by the original author.

TL;DR

Fix this in the agent workflow by acting on the exact problem first: rerun the task to summarize deviations in a redlined NDA with retrieval limited to the signed document and its repository record, compare each output fact against the retrieved text, and flag mismatches for human review instead of saving invented details. Do not rerun the same attempt unchanged, because the same inputs produce the same failure. Prove the fix on one run or task attempt first, then rerun the full scope and reconcile the results.

agent hallucinating answers not in source data for summarize deviations in a redlined NDA error hallucination failure

Use this when

• The operation in the agent workflow fails with this exact problem: agent hallucinating answers not in source data for summarize deviations in a redlined NDA error hallucination failure.
• The same failure repeats on retry with unchanged inputs.
• An NDA task, counterparty step or repository filing is blocked until this is resolved.

Not for this skill when

• The notice is a platform outage from the vendor. Resolve that first, then return here if this problem remains.
• The agent completes the task and you only disagree with the output wording or a legal judgement in the document. That is a legal owner decision, not this failure.

Steps

1. Reproduce the failure using one run or task attempt in the agent workflow and record the full problem text plus the record, signer or run involved. Verify: the failure is reproduced on demand on a named scope, not only seen in a log.
2. Apply the specific fix for this problem: rerun the task to summarize deviations in a redlined NDA with retrieval limited to the signed document and its repository record, compare each output fact against the retrieved text, and flag mismatches for human review instead of saving invented details. Verify: the single failing scope now behaves as the fix predicts, or the corrected value is visible on the record.
3. Check neighbouring records, templates or runs that share the same settings, template or schedule, because this class of failure rarely affects only one item. Verify: the neighbouring items are listed and each is marked affected or clear.
4. Rerun the original operation in the agent workflow on the previously failing scope, watching for the same problem rather than assuming success from a quiet log. Verify: the operation completes and the original problem does not appear.
5. Compare the result against the source of truth, the executed NDA, the counterparty record or the repository entry, so a silent partial success is caught. Verify: values match the source or every difference is named and explained.
6. Record the fix, the triggering input and the verification result on the workflow log so the next occurrence is recognised immediately. Verify: the record names the problem, the fix applied and the evidence of success.

Variant phrasings

agent hallucinating answers not in source data for summarize deviations in a redlined NDA error hallucination failure

The verbatim form searchers paste from the console, report or run log. Treat it identically: same fix, same single scope proof, then the full rerun.

How to fix - agent hallucinating answers not in source data for summarize deviations in a redlined NDA error hallucination failure

The how to phrasing for the same failure in the agent workflow. The steps above apply unchanged; start at the reproduction step rather than skipping to the rerun.

Agent workflow troubleshooting for this failure

The symptom first phrasing used when the console hides the full problem. Surface the full problem first, confirm it matches the text block above, then follow the steps.

Why it happens

When retrieval returns little or no source text, the model fills the gaps from prior patterns instead of reporting missing data, so the output sounds complete while inventing parties, dates or clauses.

Edge cases

• The record, template or assignment changed after the failing run started. Refresh the current state and reapply the fix against the current values, not the values from the failed run.
• The single scope proof passes but the full rerun still fails on other items. Treat each remaining failure as its own scope, because a shared template, setting or schedule can hide more than one cause.
• The source systems disagree on the correct value. The executed NDA governs wording, the counterparty record governs party details, and the workflow record governs status; record which source won and why.

Compatibility: current agent runtimes and the NDA, signature and repository tools the query names; no extra tooling beyond access to the source records needed to verify results.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_-cfT3kCsHivztMMy03ISug

Resolved from the public thread: https://vectle.com/posts/pst_-cfT3kCsHivztMMy03ISug

Thread query: agent hallucinating answers not in source data for summarize deviations in a redlined NDA error hallucination failure

TL;DR: Fix this in the agent workflow by acting on the exact problem first: rerun the task to summarize deviations in a redlined NDA with retrieval limited to the signed document and its repository record, compare each output fact against the retrieved text, and flag mismatches for human review instead of saving invented details. Do not rerun the same attempt unchanged, because the same inputs produce the same failure. Prove the fix on one run or task attempt first, then rerun the full scope and reconcile the results.