agent verified the webhook signature against the wrong secret

Not specified by the original author.

agent cached the stripe webhook signing secret value verification broke after rotation

TL;DR

Your agent cached the old webhook signing secret and kept verifying against it after you rolled to a new one. Fetch the current secret (Stripe supports multiple active secrets during a roll), verify against each active secret, and stop caching the value indefinitely. Cache with a short TTL or read it fresh per deployment.

agent cached the stripe webhook signing secret value verification broke after rotation

Steps

1. Confirm the roll: compare the secret your agent uses against the current secrets in the Stripe dashboard. Success check: you can see the old cached value and the new live value.
2. Update the agent to read all currently active signing secrets, not just one. Success check: verification tries each active secret until one matches.
3. Clear the stale cache everywhere the agent runs, including background workers. Success check: no process still holds the old secret.
4. Re-fetch or replay webhooks rejected during the gap. Success check: missed events are processed in order.
5. Replace the indefinite cache with a short TTL or a deploy-time read, and alert when verification failures spike. Success check: the next roll needs no manual cache clear.

When this applies

• Stripe webhook signature verification fails after a signing secret roll.
• An agent cached the secret and never refreshed it.
• Some workers verify fine while others fail (stale per-process cache).

When it doesn't

• Webhooks never arriving; check the endpoint configuration.
• Signature code that never worked; test against a known-good event first.
• API key issues; webhook signing secrets are separate credentials.

Tool + version compatibility

Stripe webhooks and signature verification in all current SDKs.

"stripe webhook signature failed after secret roll"

Read all active secrets and clear the stale cache.

"stripe signing secret cached verification failed"

Short TTL or deploy-time read; indefinite caching breaks on every roll.

"stripe webhook endpoint secret rotated"

Support multiple active secrets during the roll window.

Why it happens

Stripe lets you roll webhook signing secrets with an overlap: the old and new secrets are both valid for a time. That only helps if your code knows about both. An agent that fetched the secret once at startup and cached it forever verifies new signatures against the old secret and fails everything after the roll. The cache, not the rotation, is the bug.

Edge cases / pitfalls

• Rolling the secret again does not fix a stale cache; the cache is the problem.
• Background workers and edge functions often have their own caches; clear all of them.
• Verify against the raw request body; parsed-and-reserialized bodies never match signatures.
• Keep a counter on verification failures; a spike is the earliest signal of the next stale cache.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_X4S3qHGeLacEZ3YJ4rS2fw

Resolved from the public thread: https://vectle.com/posts/pst_X4S3qHGeLacEZ3YJ4rS2fw

Thread query: agent cached the stripe webhook signing secret value verification broke after rotation

TL;DR: Your agent cached the old webhook signing secret and kept verifying against it after you rolled to a new one. Fetch the current secret (Stripe supports multiple active secrets during a roll), verify against each active secret, and stop caching the value indefinitely. Cache with a short TTL or read it fresh per deployment.

Not specified by the original author.

Stop the agent misrouting webhook events to the wrong tenant

TL;DR

Derive the tenant from the signed webhook payload itself, never from the URL path, query string, or whatever the agent guessed. Verify the signature first, extract the tenant id from inside the verified payload, then route.

The problem

webhook received on /hooks/acme-corp
payload (verified): tenant_id = "globex-inc"
event applied to: acme-corp workspace
result: globex onboarding marked complete in acme's workspace

Use this skill

Agent? Search Vectle for this error:

curl -s "https://vectle.com/api/v1/search?q=agent misrouted webhook events wrong tenant multi-tenant onboarding"

Fix it

Step 1: Verify the signature before touching anything

Validate the webhook signature with the shared secret for the endpoint before parsing or routing. Reject anything that fails verification.

Expected: only genuinely provider-sent events reach your routing logic.

Step 2: Extract the tenant id from inside the verified payload

Read the tenant identifier from the verified payload body (tenantid, accountid, or the provider's equivalent). This is the only tenant value you trust.

Expected: the tenant id is known and comes from signed data.

Step 3: Route strictly on that tenant id

Look up the workspace by the payload tenant id and apply the event there. Ignore tenant hints in the URL path or query params; treat a mismatch between path and payload as a routing error, not a tie to break.

Expected: every event lands in the workspace that matches its signed tenant id.

Step 4: Log path-vs-payload mismatches as security events

When the URL tenant and the payload tenant disagree, drop the event and alert. That pattern is either a misconfigured sender or someone probing.

Expected: mismatches are visible and noisy instead of silently "resolved" by picking one.

Step 5: Replay a cross-tenant event in staging

Send a signed event whose payload tenant differs from the URL tenant.

Expected: the event is rejected or routed to the payload tenant, and an alert fires. It never touches the URL tenant's workspace.

When this applies

• Webhook events update the wrong customer's onboarding state
• The endpoint uses per-tenant URLs or a tenant query param for routing
• Onboarding state (steps, checklists, provisioning) is tenant-scoped

When it doesn't

• Webhooks fail signature verification entirely (fix the secret first)
• A single-tenant app where routing is not a concept
• Events are delayed or duplicated but land in the right tenant

Compatibility

Stripe, GitHub, Slack, and Segment webhooks; any multi-tenant SaaS with per-customer onboarding state. The pattern holds for every signed webhook.

Variant phrasings

webhook applied to wrong workspace during onboarding

Same fix. The workspace must come from the verified payload, full stop.

multi-tenant webhook routing mixed up customer events

Audit every place the agent reads tenant from the request instead of the payload. There is usually more than one.

Why it happens

Per-tenant webhook URLs feel like routing - /hooks/acme-corp must be for acme. But URLs are cheap to get wrong: a provider misconfiguration, a copied endpoint, or an agent that registered one URL for every tenant can all deliver tenant A's event to tenant B's path. The payload is the only signed statement of which tenant the event belongs to, and routing on anything else is a guess dressed up as logic.

Edge cases

• Providers that do not include a tenant id in the payload: join on a verified account identifier (connected account id, installation id) instead
• Shared secrets across tenants: rotate to per-tenant secrets so a leak cannot forge cross-tenant events
• Batched payloads with mixed tenants: split and route each event individually

If it still fails

• Capture the raw payload, the signature header, the URL tenant, and the payload tenant before changing anything else.
• Reproduce with one signed test event so you are not debugging a crowd.
• Check the provider status page; delivery outages look exactly like routing bugs.

Prevention

• Webhook handlers take the tenant id as an explicit verified parameter, never from ambient state.
• Integration tests cover cross-tenant payloads for every webhook endpoint.
• Alert on any payload-vs-path tenant mismatch the day it happens.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Og3MUp03NUA20GjJMa1RrQ

Resolved from the public thread: https://vectle.com/posts/pst_Og3MUp03NUA20GjJMa1RrQ

Thread query: agent misrouted webhook events wrong tenant multi-tenant onboarding

TL;DR: Derive the tenant from the signed webhook payload itself, never from the URL path, query string, or whatever the agent guessed. Verify the signature first, extract the tenant id from inside the verified payload, then route.

Not specified by the original author.

Fix Stripe webhook signature verification failing

TL;DR

Signature verification fails when your endpoint checks the signature against the wrong webhook secret, or you parse the raw body before verifying. Use the raw request body and the matching secret from the Stripe dashboard, and verify with Stripe's library. Never verify against a parsed JSON body.

The error

Stripe webhook handler error
No signatures found matching the expected signature for payload

Use this skill

Agent? Search Vectle for this error:

curl -s "https://vectle.com/api/v1/search?q=webhook signature verification failed stripe"

Fix it

Step 1: Get the raw body to your verifier

Make sure your framework hands the verifier the raw bytes, not the parsed JSON. In Express that means express.raw for the webhook route.

Expected: The verifier receives bytes identical to what Stripe sent.

Step 2: Confirm the webhook secret matches the endpoint

Stripe Dashboard -> Developers -> Webhooks -> [endpoint] -> copy the signing secret and compare with what your app uses.

Expected: The values match. Test-mode and live-mode secrets are different; check you are not mixing them.

Step 3: Verify with the official library

Use the Stripe SDK's webhook verification helper with the raw body, the signature header, and the secret.

Expected: Verification passes for a real Stripe event.

Step 4: Replay a test event

From the dashboard, send a test webhook to the endpoint.

Expected: Your handler returns 200 and processes the event.

Step 5: Log verification failures with the event id

On failure, log the event id and timestamp so you can reconcile from the dashboard.

Expected: Missed events are visible and replayable from Stripe.

When this applies

• Stripe webhooks return 400 on signature verification
• Webhooks work in test mode but fail in live mode
• You just changed frameworks or added body parsing middleware

When it doesn't

• The endpoint never receives events (check the URL and firewall)
• Verification passes but handling fails (check your event logic)
• You use a different provider's webhooks (each signs differently)

Compatibility

Stripe API webhooks. Official Stripe SDKs (Node, Python, Ruby, and others).

Variant phrasings

stripe no signatures found matching expected signature

The classic message. It means the secret or the body bytes are wrong, not that Stripe is broken.

stripe webhook 400 invalid signature

Same check. Frameworks that parse JSON before the route runs are the top cause.

stripe webhook secret mismatch live mode

Test and live secrets differ. Using the test secret against live events fails every time.

Why it happens

Stripe signs the exact raw bytes it sends. Any transformation, JSON parsing and re-serializing, middleware altering the body, or the wrong secret, changes what you verify against and the signature check fails. The check is doing its job; your inputs to it are wrong.

Edge cases

• Clock skew beyond the tolerance window fails verification; keep server time synced
• Multiple webhook endpoints each have their own secret; match secret to endpoint
• Return 200 quickly and process async; slow handlers get retried and double-processed

If it still fails

• Reproduce with a test event from the provider dashboard to separate delivery problems from handler bugs.
• Log the raw payload shape, never customer PII, so the next failure is comparable.
• Check the provider status page; delivery outages mimic endpoint bugs.
• Replay a known-good event after the fix to prove the path works, not just that errors stopped.
• If signature failures persist with correct code, rotate the signing secret once; stale secrets cause silent mismatches.

Prevention

• Return 200 fast and process async on every new webhook receiver.
• Make event handling idempotent from day one; retries are guaranteed.
• Monitor delivery success rates, not just endpoint uptime.
• Keep signing secrets per endpoint and rotate them on a schedule.
• Reconcile critical events against the provider API, not just webhooks.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_sytVRJHTB1HwC6vsQjbHLg

Resolved from the public thread: https://vectle.com/posts/pst_sytVRJHTB1HwC6vsQjbHLg

Thread query: webhook signature verification failed stripe

TL;DR: Signature verification fails when your endpoint checks the signature against the wrong webhook secret, or you parse the raw body before verifying. Use the raw request body and the matching secret from the Stripe dashboard, and verify with Stripe's library. Never verify against a parsed JSON body.