Diagnose Traefik 502/504 Bad Gateway with the Docker provider

[{"constraint": ">=2.0", "technology": "Traefik", "versionscheme": "semver"}, {"constraint": ">=20.10", "technology": "Docker Engine", "versionscheme": "semver"}]

Diagnose Traefik 502/504 Bad Gateway with the Docker provider

Use this when requests reach Traefik but come back 502 Bad Gateway or 504 Gateway Timeout for a service discovered via the Docker provider. The router usually looks healthy in the dashboard — the failure is in the last hop, between Traefik and the container. Classify from the status code and the debug log line before touching labels: 502 and 504 have different root causes.

1. Read the status code and the debug line first

Turn on debug (--log.level=DEBUG or log.level: DEBUG in static config) and reproduce. Traefik logs the exact backend address it tried:

msg="Creating server 0 http://CONTAINER_IP:80" serverName=0 routerName=my-service@docker

and on failure:

msg="'502 Bad Gateway' caused by: dial tcp CONTAINER_IP:1980: connect: connection refused"

Classify:

• 502 + connect: connection refused — Traefik dialed the right host but the wrong port (or nothing listens there). Go to step 2.

• 502 immediately, no dial line, or router missing from dashboard — the router/service was never built from your labels. Go to step 4.

• 504 / dial tcp ... i/o timeout — Traefik and the container are on different Docker networks (or the target IP is unreachable). Go to step 3.

• 502 + connection reset on a TLS backend — the backend speaks HTTPS with a self-signed cert; see the note at the end of step 2.

2. Wrong port: Traefik picks the first exposed port

By default Traefik uses the first exposed port of the container — not the published port, not the port you think the app uses. The classic failure: you publish 1980:80 and set the service port to 1980 (the published port), but inside the container the app listens on 80, so Traefik dials container-ip:1980 and gets connection refused.

Set the container-internal port explicitly:

labels:
  - "traefik.http.services.myapp.loadbalancer.server.port=80"

Rules:

• The value is the port the process listens on inside the container. Published (ports:) mappings are irrelevant — Traefik talks to the container IP directly.

• If the image exposes multiple ports, don't rely on "first exposed" — declare the port.

• A container that exposes no ports at all gets no usable default; declare the port.

• The service name in the label (myapp above) can be anything; the router picks it up automatically when there is exactly one service for the container, or reference it with traefik.http.routers.myapp.service=myapp@docker.

Self-signed TLS backend note: if the backend serves HTTPS with a cert Traefik can't verify, set the static option --serversTransport.insecureSkipVerify=true (static config serversTransport.insecureSkipVerify: true). This is a static, startup-time option — it cannot be set per-router via labels.

3. Wrong network: Traefik guessed which network to use

When a container is attached to more than one network, Traefik picks one — and it may pick the wrong one, or flip between them. Symptom: 504 timeouts, or "Bad Gateway" that comes and goes, with the dashboard showing the router as healthy.

Pin the network with the label:

labels:
  - "traefik.docker.network=frontend"

Gotchas:

• The value must be the real Docker network name, not the compose shorthand. Compose prefixes network names with the project name, so networks: [frontend] in compose usually creates myproject_frontend. Check with docker network ls. To keep the literal name, set the name: property on the network in compose.

• Traefik itself must be attached to that same network (it needs a route to the container IP it dials). The Traefik container's networks list and the app's networks list must share at least the pinned network.

• The static alternative --providers.docker.network=frontend sets a default for all containers; the label overrides it per container.

4. Router never built: the labels were ignored

If the router doesn't appear in the dashboard at all (or appears disabled):

• traefik.enable=true is required on the container when the provider runs with --providers.docker.exposedByDefault=false. Without it, Traefik silently skips the container.

• Labels must be on the container, not the Traefik service, and spelled exactly: traefik.http.routers.[name].rule=Host(app.example.com) uses backticks, not quotes, around the hostname.

• In Docker Swarm, labels go under the service's deploy: block — Swarm reads service labels, not container labels.

• The v1 label traefik.port does not exist in v2/v3; it is silently ignored. Use loadbalancer.server.port (step 2).

• Confirm the provider is actually watching Docker: --providers.docker=true and the socket mounted (/var/run/docker.sock:/var/run/docker.sock:ro).

5. Checklist

1. Reproduce with --log.level=DEBUG; capture the Creating server 0 ... line and any dial tcp error.

2. connect: connection refused → fix loadbalancer.server.port to the container-internal port (step 2).

3. i/o timeout / 504 → pin traefik.docker.network to the real network name shared with the Traefik container (step 3).

4. Router missing/disabled in dashboard → traefik.enable=true, label spelling, exposedByDefault, Swarm deploy: labels (step 4).

5. Self-signed TLS backend → serversTransport.insecureSkipVerify in static config (restart required).

Built from the Traefik Docker provider reference (first-exposed-port default, traefik.docker.network label, exposedByDefault), the provider-namespace docs, and recurring community forum threads on random-network selection, published-vs-internal port confusion, and Swarm deploy-label placement.