agent hardcoded the webhook secret - the provider rotated it and every delivery failed verification for a day

TL;DR

The agent baked the webhook secret into the code, the provider rotated it, and every delivery failed verification for a day. Load the secret from config or a secret store at startup so rotation is a config change, not a code change.

Error

webhook signature verification failed on every delivery after provider rotated the signing secret

Steps

1. Move the webhook secret out of code into env config or a secret store Expected: the handler reads the secret at startup.
2. Update the stored secret to the new value from the provider dashboard Expected: verification passes again.
3. Re-request or replay the failed deliveries if the provider keeps them Expected: missed events are recovered.
4. Subscribe to the provider's rotation notices or check the secret age in monitoring Expected: the next rotation is not a surprise.

When to use

• Webhook verification fails right after a provider rotation
• The secret is hardcoded

When not to use

• Verification never worked (then the secret or algorithm is wrong)
• Signatures fail intermittently (then check clock skew or payload canonicalization)

Tool compatibility

• Webhook HMAC signature verification
• Secret management for deployables

Variant phrasings

webhook secret rotated

Never hardcode it. Load from config so rotation is a value change.

signature verification failed after rotation

Update the stored secret and replay missed events.

Why it happens

Hardcoding is the fastest way to get a demo working, so agents do it. Providers rotate secrets on a schedule, and the demo becomes an outage.

Edge cases

• Some providers publish two secrets during rotation grace periods; support a primary plus fallback
• Store secrets per environment; staging and prod rotate independently

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_vukAgsixSrCBDD0XyMeYmQ