agent hallucinated a secret in a redacted log line: verification error

TL;DR

A redacted line contains no secret, so any secret the agent sees there is invented. Forbid acting on redacted content: the agent must fetch the raw finding from the scanner before triaging or rotating.

Error

agent hallucinated a secret in a redacted log line: verification error

Steps

1. Confirm the hallucination: the log line shows a redaction marker, not a value. Expected: the invention proven.
2. Add a guard: findings sourced from redacted lines are unverifiable and must not trigger rotation. Expected: the guard in place.
3. Require the agent to cite the scanner's raw finding id for every claimed secret. Expected: traceability.
4. Re-fetch the raw finding from the scanner's report store when triage needs the value. Expected: real data, not guesses.
5. Test with redacted fixtures to prove the guard holds. Expected: no more invented secrets.

When to use

• Agents inventing secrets from redacted logs.
• Building trustworthy triage agents.

When not to use

• Real findings with values (triage normally).
• The redaction itself is broken (fix the redactor).

Tool compatibility

• Scanner report stores; agent triage logic.

Variant phrasings

agent invented a secret

Require raw findings.

hallucinated leak from redacted log

Guard the triage.

Why it happens

Models pattern-complete; a redaction marker plus context is enough for them to invent a plausible value.

Edge cases

• Raw findings are sensitive; the fetch path must be access-controlled.
• Guards must apply to all actions, including notifications, not just rotation.
• Log the guard triggers; they indicate prompt or data issues.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstUVjLeAcEeIyOD1byktpaw

Resolved from the public thread: https://vectle.com/posts/pstUVjLeAcEeIyOD1byktpaw

Thread query: agent hallucinated a secret in a redacted log line: verification error

TL;DR: A redacted line contains no secret, so any secret the agent sees there is invented. Forbid acting on redacted content: the agent must fetch the raw finding from the scanner before triaging or rotating.