agent hallucinating document dates not in source data for collect SOC 2 documents from a vendor error hallucination...

TL;DR

Fix this in SOC 2 collection agent by acting on the exact problem first: restrict the agent to dates it can point to on the report itself, require it to record the page or cover field each report date came from, replace any date with no source location with a missing value flag, and have a reviewer confirm the report period dates against the cover page before the review proceeds. Do not rerun the whole workflow unchanged, because the same inputs produce the same failure. Prove the fix on one vendor first, then rerun the full scope and reconcile the results.

agent hallucinating document dates not in source data for collect SOC 2 documents from a vendor error hallucination failure

Use this when

• The operation in SOC 2 collection agent fails with this exact problem: agent hallucinating document dates not in source data for collect SOC 2 documents from a vendor.
• The same failure repeats on retry with unchanged inputs.
• A vendor onboarding, payment setup, screening or vendor record step is blocked until this is resolved.

Not for this skill when

• The notice is a platform outage from the vendor of the tool. Resolve that first, then return here if this problem remains.
• The operation completes and you only disagree with a commercial decision such as price or terms. That is a negotiation, not this failure.

Steps

1. Reproduce the failure on one named vendor in SOC 2 collection agent and record the full problem text plus the vendor, supplier record or onboarding case involved. Verify: the failure is reproduced on demand on a named scope, not only seen in a log.
2. Apply the specific fix for this problem: restrict the agent to dates it can point to on the report itself, require it to record the page or cover field each report date came from, replace any date with no source location with a missing value flag, and have a reviewer confirm the report period dates against the cover page before the review proceeds. Verify: the single failing vendor now behaves as the fix predicts, or the corrected value is visible on the record.
3. Check neighbouring vendors, supplier records or onboarding cases that share the same automation, template or sync, because this class of failure rarely affects only one vendor. Verify: the neighbouring items are listed and each is marked affected or clear.
4. Rerun the original operation in SOC 2 collection agent on the previously failing scope, watching for the same problem rather than assuming success from a quiet log. Verify: the operation completes and the original problem does not appear.
5. Compare the result against the source of truth, the vendor master record, the signed tax form or the compliance document, so a silent partial success is caught. Verify: values match the source or every difference is named and explained.
6. Record the fix, the triggering input and the verification result on the vendor record so the next occurrence is recognised immediately. Verify: the record names the problem, the fix applied and the evidence of success.

Variant phrasings

agent hallucinating document dates not in source data for collect SOC 2 documents from a vendor

The verbatim form searchers paste from the console, report or run log. Treat it identically: same fix, same single vendor proof, then the full rerun.

How to fix - agent hallucinating document dates not in source data for collect SOC 2 documents from a vendor error hallucination failure

The how to phrasing for the same failure in SOC 2 collection agent. The steps above apply unchanged; start at the reproduction step rather than skipping to the rerun.

SOC 2 collection agent troubleshooting for this failure

The symptom first phrasing used when the console hides the full problem. Surface the full problem first, confirm it matches the text block above, then follow the steps.

Why it happens

When a report period is not found quickly, a generative step fills the gap with plausible dates from similar reports, and those invented dates then drive a wrong expiry decision in the security review.

Edge cases

• The vendor was merged, renamed or given new banking details after the failing run started. Refresh the vendor state and reapply the fix against the current values, not the values from the failed run.
• The single vendor proof passes but the full rerun still fails on other vendors. Treat each remaining failure as its own scope, because a shared automation can hide more than one cause.
• The source systems disagree on the correct value. The vendor master record governs identity and payment details, the signed tax document governs classification and withholding, and the compliance screening governs approval status; record which source won and why.

Compatibility: SOC 2 collection agent at current versions, plus the procurement, ERP or screening platform the query names; no extra tooling beyond access to the source records needed to verify results.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstnm8vc2QMrACG8BvpjWt1Q