kubectl debug ephemeral containers: how to use them on running pods

TL;DR

Ephemeral debug containers attach a temporary troubleshooting container to a running pod, sharing its namespaces: you get a shell with tools next to a distroless app container without rebuilding anything. They are perfect for live debugging (network, filesystem, process inspection) and vanish when the pod restarts. If your cluster version supports them, they replace most uses of exec into app containers.

The query

kubectl debug ephemeral containers: how to use them on running pods

Use this when

• The app container is distroless with no shell
• You need debug tools (curl, strace, dig) next to a running app
• Avoiding image rebuilds for one-off debugging
• Inspecting a pod's network namespace live

Not for when

• Permanent observability (use real sidecars)
• Debugging crashed pods (they are gone; use logs)
• Clusters too old to support ephemeral containers

Steps

Step 1: Confirm cluster support

Check that the cluster version supports ephemeral containers. On older clusters the command fails outright; knowing this before a 3am incident saves frustration. Expected output: support confirmed, or the version gap identified.

Step 2: Attach a debug container with the tools you need

Run kubectl debug with a debug image containing your tools, targeting the pod. The ephemeral container starts alongside the app container, sharing the pod's network and (optionally) process namespace. Expected output: a shell in the debug container, running in the pod's context.

Step 3: Share the process namespace when needed

For process inspection (seeing the app's processes, signals, /proc), enable process namespace sharing. Without it you see only your own container's processes, which limits debugging. Expected output: the app's processes visible from the debug container.

Step 4: Debug without touching the app container

Run your diagnostics from the debug container: network calls to the app's YOUR_HOST, filesystem inspection of shared volumes, DNS checks. The app container is undisturbed, which matters for heisenbugs. Expected output: diagnostic data gathered with zero impact on the running app.

Step 5: Clean up by letting the pod cycle

Ephemeral containers cannot be removed individually; they disappear when the pod restarts. Do not leave debug pods running: restart or redeploy to return to the clean spec. Expected output: the pod back to its declared state, debug container gone.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_X4rvdvgz09FLLLQLtOlYYQ