how to avoid SSRF in webhook features

how to avoid SSRF in webhook features

TL;DR

Any feature that fetches a user-supplied URL can be steered at your internal network unless you constrain it. Allowlist schemes, resolve the hostname yourself and reject non-public addresses, re-validate on every redirect hop, and block the cloud instance metadata endpoint by name. The check has to happen at request time, not just at input time.

how to avoid SSRF in webhook features

Use this when

• You are building webhooks, URL previews, or link unfurling
• A PR adds "fetch this URL and show the result" functionality
• An importer or integration takes a callback URL from the user
• An agent is reviewing outbound request code for internal-network reach
• A scan flagged a possible SSRF and you need the fix checklist

Not for this skill when

• The redirect happens in the user's browser (open redirect, different skill)
• The question is about XSS through fetched content (sanitize separately)
• You are writing firewall rules rather than application checks
• The fetcher only ever contacts one hardcoded internal service

Steps

1. Inventory every outbound fetch of user-supplied URLs

Find all the places the app makes HTTP requests to addresses influenced by user input: webhooks, previews, avatar URLs, import-from-URL, PDF generators.

grep -rn "requests\.get\|fetch(\|axios\.\|http\.get" --include="*.py" --include="*.js" --include="*.ts" [HOME]/... | head -40

Expected: a list of fetch sites with the source of each URL marked. Anything user-influenced goes through the validation in the next steps.

2. Allowlist schemes and parse strictly

Accept only http and https, reject everything else (file, gopher, dict, and other exotic schemes your HTTP client might honor). Parse with a real URL parser and reject URLs with credentials embedded in them.

Expected: a shared validation helper that every fetch site calls, with tests for scheme smuggling like http:example.com variants and credentials-in-URL inputs.

3. Resolve DNS yourself and reject non-public targets

Resolve the hostname at validation time and refuse loopback, private ranges, link-local, and the cloud provider's instance metadata endpoint. DNS rebinding means the address can change between check and fetch, so resolve once and connect to the validated address.

Expected: the helper blocks the loopback address, private ranges, and the metadata endpoint by name and by resolved address. A test suite feeds it hostile hostnames and asserts rejection.

4. Re-validate on every redirect

A URL that starts public can redirect to an internal address. Either disable redirects and handle them manually, or run the full validation again on each redirect target with a small hop limit.

Expected: redirect following is capped (3 to 5 hops) and each hop passes the same checks as the original URL. A test with a redirect chain ending internal is rejected.

5. Constrain the fetcher itself

Set short timeouts, cap response sizes, and restrict which ports are reachable. Run outbound fetches through an egress proxy with its own allowlist when the feature is high-risk.

Expected: a hung or gigantic response cannot tie up workers, and the proxy logs show only expected destinations.

6. Probe it like an attacker

Point the feature at internal-sounding hostnames, redirect chains, DNS rebinding test domains, and the metadata endpoint name, and confirm each is blocked or neutralized.

Expected: all probes fail safely with clean errors, and the results become regression tests so later refactors cannot reopen the hole.

Variant: URL previews and unfurling

Previews fetch arbitrary user links by design, so they need the strictest version of this checklist plus content-type checks on the response. Consider fetching through a sandboxed renderer rather than the app server.

Variant: webhooks with custom headers

Letting users set arbitrary headers on webhook delivery can smuggle auth into internal requests. Allowlist the headers users may set and never forward internal credentials.

Variant: PDF and screenshot generators

Headless browsers fetching user URLs are SSRF with extra steps and can also read local files. Run them in an isolated network segment with no route to internal services.

Why this happens

Developers picture the feature fetching public web pages, but the fetcher runs inside the private network with credentials and access the outside world does not have. Every SSRF is that mismatch: the URL is attacker-controlled, the network position is privileged, and nothing in between questions the combination. Validation at input time feels sufficient until redirects and DNS rebinding move the target after the check.

Edge cases and pitfalls

• IPv6 forms of private addresses and decimal or octal IP encodings bypass naive string checks; validate the parsed address, not the string.
• Shortened URLs hide the final destination; expand and validate the full chain.
• Webhook retry storms against a blocked target can look like an attack in your logs; distinguish blocked-by-policy from failing deliveries.
• Allowing user-supplied ports widens the target set to every internal service; default to 80 and 443 unless the feature needs more.
• The metadata endpoint has both a name and an address; block both, and recheck after any cloud provider migration.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst2yaW1uWLMFw_-zHGWG2Hw