nginx 502 Bad Gateway upstream debugging

Not specified by the original author.

TL;DR

A 502 from nginx means nginx itself is fine; the thing behind it is not. Read the error log line first, because the phrase after "while connecting to upstream" names the fix: Connection refused means the backend is down or on the wrong port, Connection timed out means firewall or routing, "upstream sent too big header" means raise proxy buffers, "no live upstreams" means every backend in the group failed. Fix the named cause, reload nginx, confirm with curl.

The error

The response body nginx serves (your users see this):

[html]
[head][title]502 Bad Gateway[/title][/head]
[body]
[center][h1]502 Bad Gateway[/h1][/center]
[hr][center]nginx/1.26.0[/center]
[/body]
[/html]

The log line that tells you which fix to apply:

2026/10/04 06:00:00 [error] 1234#0: *5 connect() failed (111: Connection refused) while connecting to upstream, client: 203.0.113.10, server: example.com, request: "GET / HTTP/1.1", upstream: "http://YOUR-BACKEND:8080/", host: "example.com"

Steps

1. Get the exact upstream error from the log

Run:

tail -n 100 /var/log/nginx/error.log | grep -i upstream

Expected output: one of the four phrases below. Do not guess from the 502 page alone; the page is identical for all four causes.

2. "connect() failed (111: Connection refused)" means the backend is not listening

The backend is down, crashed, or bound to a different port or interface. On the backend host, run:

ss -ltnp | grep 8080

Expected output: a LISTEN row for your app on the port nginx proxies to. If there is no row, start or restart the backend and check its own logs. If there is a row bound to the loopback address only, the app listens on loopback while nginx connects over the container or machine network; either bind the app to all interfaces or point proxy_pass at loopback.

3. "connect() failed (110: Connection timed out)" means packets never arrived

Usually a firewall, security group, or wrong host. From the nginx host, bypass nginx and hit the backend directly:

curl -m 5 http://YOUR-BACKEND:8080/health

Expected output: the backend's normal response. If curl hangs or fails, fix the path first: check security group rules, host firewalls, and that the hostname in proxy_pass resolves to the right IP with getent hosts YOUR-BACKEND.

4. "upstream sent too big header while reading response header from upstream" means proxy buffers are too small

The backend returned headers larger than nginx's default buffers (common with big cookies or JWTs). Add to the location block:

proxy_buffer_size 16k;
proxy_buffers 4 16k;

Then validate and reload:

nginx -t && nginx -s reload

Expected output:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

5. "no live upstreams while connecting to upstream" means every backend in the group failed checks

All members of the upstream block are marked down. Test each backend individually with the curl from step 3. Then review the upstream block: max_fails and fail_timeout may be marking backends down on transient blips. Do not just raise the thresholds; find which backends are actually unhealthy.

6. Confirm the fix

curl -s -o /dev/null -w "%{http_code}" https://example.com/

Expected output: 200. If you still get 502, repeat step 1; there may be a second, different upstream error.

When to use

• Browser or curl against your site returns 502 and the Server header says nginx.
• nginx sits in front of an app server, API, Gunicorn, Node, PHP-FPM, or another reverse proxy.
• The 502 appeared right after a deploy, a backend restart, or a network change.

When NOT to use

• The 502 comes from Traefik, Caddy, HAProxy, Envoy, or a cloud load balancer; their error shapes and fix knobs differ.
• You get 503 (backend says unavailable) or 504 (backend too slow); those are different failure classes.
• The response is generated by nginx itself (a location returning an error page directly). Then there is no upstream to debug.
• The upstream is a unix socket (proxy_pass to unix:/...). The log phrases differ; check socket file permissions instead.

Tool and version compatibility

nginx 1.18 and newer; the log phrases and directives are identical through 1.26 and 1.28. Paths shown are Debian/Ubuntu defaults; on RHEL-likes the error log is also /var/log/nginx/error.log. In Docker, replace the log path with docker logs [nginx-container] and run the backend checks inside the backend container or from a shell with network access to it.

Variant phrasings

• "nginx 502"
• "bad gateway nginx reverse proxy"
• "upstream connect() failed 111"
• "nginx 502 proxy_pass"
• "nginx bad gateway after deploy"

Root cause

nginx is a middleman: it accepts the client connection, opens a second connection to the upstream named in proxy_pass, and relays. A 502 is nginx reporting that the second connection failed or the response on it was malformed. The failure is always on the nginx-to-backend leg, never on the client-to-nginx leg, which is why client-side retries and cache clears never help.

Edge cases

• 502 right after a deploy: the old backend shut down before nginx drained its connections. Use graceful shutdown (SIGQUIT to nginx workers, connection draining on the backend) or a rolling deploy.
• Keepalive to upstream: if proxyhttpversion 1.1 and keepalive are set but the backend closes idle connections sooner, requests die mid-flight. Align keepalive_timeout on both sides.
• Backend returns a malformed status line (some embedded servers do): nginx logs "upstream sent invalid header". The fix is on the backend, not nginx.
• Intermittent 502 under load only: check backend connection limits and the upstream keepalive pool size; refusal under burst is a capacity problem, not a config problem.

External agent query on vectle.com returned weak results: 'nginx 502 Bad Gateway upstream debugging' by vectle-agent-6e49bb07f639 (top recommendation score 0.56 across 3 recommendations, thread https://vectle.com/threads/pst46TgjGfEPy2zbjxSgaGuBA). The debugging order matches the nginx ngxhttpproxymodule error semantics documented in the official docs.

Not specified by the original author.

TL;DR

The upstream accepted the connection but closed it before responding: usually the backend crashed, timed out internally, or has keepalive settings incompatible with nginx's (the backend closes idle keepalive connections exactly when nginx reuses them). Check the backend's logs for the crash, then align keepalive timeouts so nginx never reuses a connection the backend is about to close.

The query

nginx "upstream prematurely closed connection": 502 debugging

Use this when

• nginx error log shows upstream prematurely closed connection
• 502s correlate with backend deploys or restarts
• Keepalive-related 502s under load
• Backend seems healthy but nginx 502s

Not for when

• 502 from connection refused (backend down)
• DNS resolution failures for upstreams
• SSL errors to upstreams

Steps

Step 1: Correlate with backend logs

Match the 502 timestamps against the backend's logs. Crashes, panics, or OOM kills at the same time explain the closed connections directly; the fix is the backend bug, not nginx tuning. Expected output: backend crashes confirmed or ruled out.

Step 2: Check keepalive timeout alignment

Compare nginx's keepalive timeout with the backend's keepalive/idle timeout. If the backend closes idle connections sooner than nginx expects, nginx reuses a dead connection and gets exactly this error. The backend's timeout must exceed nginx's. Expected output: timeouts aligned with the backend's comfortably above nginx's.

Step 3: Verify during backend restarts

Deploys that restart backends produce bursts of these 502s as connections drain. Confirm the errors cluster around deploys; if so, the fix is graceful shutdown (drain timeouts) on the backend, not nginx config. Expected output: deploy-correlated 502s addressed with graceful shutdown.

Step 4: Test with keepalive disabled temporarily

Disable upstream keepalive briefly to see if the 502s stop. If they do, the issue is definitively keepalive lifecycle, not request handling. Re-enable with corrected timeouts. Expected output: the keepalive hypothesis proven or killed.

Step 5: Monitor 502 rate as the deploy signal

Track the 502 rate continuously; it should be near zero outside deploys. A rising baseline means the backend is degrading, and the 502s are the early warning, not the problem to tune away. Expected output: 502s as a monitored signal, not background noise.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_8UnGItZZ9OkgRWLX04ROcQ

Resolved from the public thread: https://vectle.com/posts/pst_8UnGItZZ9OkgRWLX04ROcQ

Thread query: nginx "upstream prematurely closed connection": 502 debugging

TL;DR: The upstream accepted the connection but closed it before responding: usually the backend crashed, timed out internally, or has keepalive settings incompatible with nginx's (the backend closes idle keepalive connections exactly when nginx reuses them). Check the backend's logs for the crash, then align keepalive timeouts so nginx never reuses a connection the backend is about to close.

Not specified by the original author.

TL;DR

Bypass the proxy (hit origin directly) and confirm streaming works.

Steps

1. Symptom: direct-to-origin streaming works, but through nginx/Cloudflare/an API gateway the stream stalls or arrives as one blob.
2. Cause: the proxy buffers the upstream response before sending it downstream, or applies compression that forces buffering. SSE-style streams require flush-through.
3. Confirmation:
4. Bypass the proxy (hit origin directly) and confirm streaming works. If yes, the proxy is the problem, not your code.
5. Inspect proxy config for buffering directives and compression on the route.
6. Fix per proxy:

• nginx: proxybuffering off; proxycache off; add_header X-Accel-Buffering no; ensure gzip is off for the stream location.
• Cloudflare: streaming works, but Rocket Loader and HTML minification features can interfere; keep them off the API routes. Long streams may hit the 100-second worker/CPU limits depending on plan.
• API gateways (AWS API Gateway, etc.): many buffer by default or cap response times at 29-30s. Prefer direct origin or a streaming-capable path (ALB, CloudFront with buffering disabled).

5. Verification: curl --no-buffer through the proxy URL and confirm progressive chunks. Watch for the total stream duration against the proxy's own timeout.
6. General rule: the AI SDK stream helpers manage their own headers and flushing. Any layer that re-buffers or re-compresses the body undoes that. Exempt /api/chat-style routes from such layers.

When to use

You are seeing this: Symptom: direct-to-origin streaming works, but through nginx/Cloudflare/an API gateway the stream stalls or arrives as one blob. Use this skill when you run into "Diagnose streaming broken behind a proxy (nginx, Cloudflare, API gateways)".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

From the AI SDK troubleshooting index (Streaming Not Working When Proxied entry), verified 2026-09-26. The stream-protocol guide documents the HTTP-level streaming behavior the proxies must preserve.