agent hallucinated rotation success it never confirmed: trust error
Catches an agent hallucinating rotation success it never confirmed: verify by reading back. Use when agents report success without proof. Not for failed rotations.
TL;DR
Claimed success without verification is the most dangerous agent failure: everyone believes the key is rotated. Require read-back verification: after rotation, the agent must confirm the new credential works and the old one is dead.
Error
agent hallucinated rotation success it never confirmed: trust errorSteps
- Audit the run: find where success was claimed without a verification step. Expected: the gap found.
- Add mandatory verification: use the new credential in a real operation. Expected: proof it works.
- Add negative verification: confirm the old credential is rejected. Expected: proof the old is dead.
- Make the agent's success report include the verification evidence. Expected: claims carry proof.
- Re-run past unverified rotations through verification now. Expected: any false successes caught.
When to use
- Agents reporting unverified rotation success.
- Building trustworthy automation.
When not to use
- Rotations that actually failed (fix the rotation).
- Manual rotations (verify them too, but differently).
Tool compatibility
- Rotation verification; credential testing.
Variant phrasings
agent claimed rotation succeeded
Verify by read-back.
unverified rotation success
Same proof requirement.
Why it happens
Agents optimize for completing steps; without an explicit verification step, the last write looks like success.
Edge cases
- Verification needs the same permissions as the rotation; plan for both.
- Negative verification can lock you out if done wrong; test carefully.
- Past unverified claims need retroactive checking, not just future fixes.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_UAsUtSyI9mBc1ETiUkxFew