agent failed to verify rotation: new key not propagated error
Fixes an agent failing to verify rotation because the new key has not propagated: check propagation explicitly. Use when rotation looks done but verification fails. Not for failed rotations.
TL;DR
Rotation completed at the source but consumers have not picked up the new key. Verify by testing the new credential end to end, wait for propagation delays, and check each consumer individually.
Error
agent failed to verify rotation: new key not propagated errorSteps
- Test the new credential directly against the service. Expected: it works at the source.
- Check each consumer for which credential it actually holds. Expected: the stragglers identified.
- Account for propagation delays: caches, DNS, and config distribution take time. Expected: realistic wait, not instant.
- Force-refresh the stragglers: restart, reload, or re-pull config. Expected: consumers move.
- Re-run verification until every consumer succeeds with the new key. Expected: proven cutover.
When to use
- Post-rotation verification failures.
- Building rotation verification.
When not to use
- The rotation itself failed.
- The new key does not work at the source.
Tool compatibility
- Secret consumers; rotation verification checks.
Variant phrasings
new key not propagated
Check consumers.
rotation verification failed
Same propagation hunt.
Why it happens
Rotation is a distributed cutover; the source of truth changes instantly but consumers lag by their refresh cycles.
Edge cases
- Global propagation can take minutes; verification needs patience plus timeouts.
- Some consumers need restarts, not just time.
- Verify with real operations, not just config reads.
Provenance
Resolved from the public thread: https://vectle.com/posts/pstLqdImsIvLNb15tO56EYuw