Webhook listeners at scale: HMAC validation, token refresh, and queueing the flood

Webhook listeners at scale: HMAC validation, token refresh, and queueing the flood

TL;DR

When building webhook support for many customers: keep user access tokens fresh because subscription create/delete calls need them. Always validate incoming payloads with HMAC using your webhook secret, an open listener will get probed. Tear down subscriptions programmatically when a user uninstalls, or you leak listeners.

Use this when you run into the situation in the title.

When not to use this skill: unrelated tasks. It covers only the procedure above.

Compatibility

The steps above apply to the commands named in them. This skill does not pin a version, so if a flag looks different on your machine, check your installed version's docs first.

Details

And put a queue in front of your listener, a single bulk upload upstream can fire a million events at once and take down a synchronous handler.

Context: Web article (Paragon blog, introducing Custom Webhooks): describes the real challenges of managing webhooks across many integrations. To create or delete webhook subscriptions via a third-party API you need the user's access tokens refreshed. Maintaining listeners across dozens of integrations per customer is computationally heavy, and tearing them down on uninstall has to be programmatic. Security: exposed listeners get hit with unauthorized requests, so you need HMAC validation with a webhook secret and hashing algorithm. Scale: one user uploading 1M records can trigger 1M payloads at your listener.

Source: Web article (Paragon blog, introducing Custom Webhooks): describes the real challenges of managing webhooks across many integrations. To create or delete webhook subscriptions via a third-party API you need the user's access tokens refreshed. Maintaining listeners across dozens of integrations per customer is computationally heavy, and tearing them down on uninstall has to be programmatic. Security: exposed listeners get hit with unauthorized requests, so you need HMAC validation with a webhook secret and hashing algorithm. Scale: one user uploading 1M records can trigger 1M payloads at your listener.

Verified resolution: When building webhook support for many customers: keep user access tokens fresh because subscription create/delete calls need them. Always validate incoming payloads with HMAC using your webhook secret, an open listener will get probed. Tear down subscriptions programmatically when a user uninstalls, or you leak listeners. And put a queue in front of your listener, a single bulk upload upstream can fire a million events at once and take down a synchronous handler.

Source URL: https://www.useparagon.com/blog/introducing-custom-webhooks