Ashby webhook "signature mismatch" - the agent's HMAC check fails on replayed events
Fixes Ashby webhook HMAC failures on replayed events by verifying against the raw body. Use when replays fail signature checks. Key trigger: signature mismatches on replayed webhooks.
Ashby webhook "signature mismatch" - the agent's HMAC check fails on replayed events
TL;DR
The HMAC check ran against the parsed-and-reserialized body, which differs byte-for-byte from what Ashby signed. Verify against the raw request body bytes, exactly as received. Replays fail when anything re-serializes the payload.
Ashby webhook "signature mismatch" (HMAC check failing on replayed events)Steps
Capture the raw body bytes before any parsing in the webhook handler. Expected: You have the exact signed bytes.
Verify the HMAC against the raw bytes, not the parsed object. Expected: Replays verify correctly.
Check for middleware that parses or transforms the body before your handler sees it. Expected: The transformer is identified and bypassed for webhooks.
Use a constant-time comparison for the HMAC check. Expected: The check is timing-safe.
Test with Ashby's webhook replay or a captured event to prove the fix. Expected: Replays pass.
When to use
- webhook signatures fail on replays
- the handler parses before verifying
- middleware touches the body
When not to use
- live events also fail (then the secret is wrong, not the body handling)
- the signature header is missing (check the provider config)
- the events are genuinely tampered (then the failure is correct)
Tool compatibility
- Ashby webhooks
- HMAC verification
Variant phrasings
- "ashby webhook signature mismatch"
- "webhook HMAC replay fail"
- "signature mismatch replayed event"
Why it happens
HMAC signs bytes, not objects. JSON parse-then-reserialize changes whitespace, key order, or number formatting, producing different bytes with a different HMAC. Live events sometimes survive because the framework passes them through untouched; replays through test tooling get normalized and fail. The raw body is the only correct input.
Edge cases
- body-parser middleware is the usual culprit; exclude webhook routes from it
- constant-time comparison matters once the check works; do both
- log the raw body on failure (carefully; it may contain PII) for diagnosis
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_KqEWBfj0zSnVcYsbKcT9ww