envoy "upstream connect error or disconnect/reset before headers": debugging

TL;DR

Envoy could not get a working connection to the upstream, so the fault is almost always in the cluster or route config or in the upstream itself, not in Envoy's core. Check the cluster's endpoint health in the admin interface, then verify the upstream is reachable and the cluster's TLS, SNI, and timeout settings are right. Most of the time the upstream host is down or the cluster has no healthy endpoints.

envoy "upstream connect error or disconnect/reset before headers": debugging

Steps

1. Reproduce the request and capture the x-envoy-response-flags header. UF means connection failure, URX means upstream reset, UT means timeout.

Expected: You know which failure mode you are chasing.

2. Query the admin interface clusters endpoint and check the cluster's healthy versus total endpoints.

Expected: You can see whether Envoy thinks any upstream host is usable.

3. Reach the upstream directly, bypassing Envoy, with a plain connection to its address and port.

Expected: You confirm whether the upstream is up independent of Envoy.

4. Compare the cluster config: TLS mode, SNI, port, and connect timeout against what the upstream actually serves.

Expected: Any mismatch (wrong port, missing SNI, too-short timeout) is identified and corrected.

Use this when

• Envoy returns 503 with upstream connect error or disconnect/reset before headers.
• Response flags show UF, URX, or UT on failing requests.
• A new route, cluster, or upstream deploy just went out.

Not for this skill when

• 503s generated by the upstream application itself (app bug, different fix).
• Envoy failing to receive config (xDS or control-plane delivery problem).
• Rate limiting or circuit breaking returning 429 or 503 by policy.

Variant phrasings

envoy 503 upstream connect error

envoy UF response flag

envoy no healthy upstream

Why it happens

Envoy returns this when the TCP connect to the upstream fails, the upstream resets before sending response headers, or the connect times out. Common roots: no healthy endpoints, wrong port or protocol, TLS or SNI mismatch, or a firewall between Envoy and the upstream.

Edge cases

• UF (connection failure) versus URX (upstream reset) versus UT (timeout) point at different layers - read the flag first.
• Outlier detection can eject every host, leaving a cluster with zero healthy endpoints.
• HTTP/2 to the upstream when it only speaks HTTP/1 (or the reverse) fails exactly this way.
• Connect timeouts set aggressively low trip on slow but healthy upstreams - check UT flags.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_5JzojdIVFFU8EALlpGXIWw

Resolved from the public thread: https://vectle.com/posts/pst_5JzojdIVFFU8EALlpGXIWw

Thread query: envoy "upstream connect error or disconnect/reset before headers": debugging

TL;DR: Envoy could not get a working connection to the upstream, so the fault is almost always in the cluster or route config or in the upstream itself, not in Envoy's core. Check the cluster's endpoint health in the admin interface, then verify the upstream is reachable and the cluster's TLS, SNI, and timeout settings are right. Most of the time the upstream host is down or the cluster has no healthy endpoints.