how to debug CrashLoopBackOff exit code 139 segfault

TL;DR

Exit code 139 means the process segfaulted: bad memory access, usually from native code, an incompatible binary, or a corrupt shared library. In Kubernetes it very often means the binary was built for the wrong architecture or the base image changed underneath you. Get the crash out of the loop first (a debug pod with a shell), then reproduce the segfault directly and read what the faulting address tells you.

The query

how to debug CrashLoopBackOff exit code 139 segfault

Use this when

• kubectl shows CrashLoopBackOff with last exit code 139
• The container dies instantly with no application logs
• It runs on your laptop but segfaults in the cluster
• The crash started after a base image or dependency update

Not for when

• Exit code 137 (OOMKilled, a different problem)
• Application exceptions with stack traces (those are not segfaults)
• Slow crashes after running fine for hours (more likely a leak)

Steps

Step 1: Confirm it is really a segfault

Check the pod's last state for exit code 139 and look at the previous container's logs. A segfault often leaves no logs at all, which itself is diagnostic: the process died before it could log anything. Expected output: exit code 139 confirmed, minimal or empty logs, pointing at a native-level crash.

Step 2: Check architecture mismatch first

Verify the image architecture matches the node architecture. Multi-arch confusion (amd64 image on arm64 nodes or vice versa) produces instant segfaults. Inspect the image manifest and the node labels. Expected output: either a confirmed mismatch (rebuild for the right arch) or a ruled-out cause.

Step 3: Run the binary in a debug pod with a shell

Start a pod from the same image with the entrypoint overridden to sleep, then exec in and run the binary by hand. This takes the crash loop out of the picture and lets you see stderr and core dump behavior directly. Expected output: you can reproduce the segfault on demand and see any fault output the loop was hiding.

Step 4: Identify the faulting library or call

If it is a native dependency issue, use ldd on the binary inside the debug pod to spot missing or mismatched shared libraries. For interpreted runtimes with native extensions (Python, Node, Ruby), the suspect is usually the most recently updated native module. Expected output: a named library or module whose version changed with the crash onset. Pin or roll back that dependency.

Step 5: Check for memory corruption patterns

If the segfault is intermittent rather than instant, suspect memory corruption: run with a sanitizer or memory checker in a test environment, and check dmesg on the node for hardware-level memory errors if multiple unrelated pods segfault. Expected output: either a reproducible trigger you can fix in code or deps, or node-level evidence pointing at bad hardware.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstzHdWoim7DjxluH8w6s4nA

Resolved from the public thread: https://vectle.com/posts/pstzHdWoim7DjxluH8w6s4nA

Thread query: how to debug CrashLoopBackOff exit code 139 segfault

TL;DR: Exit code 139 means the process segfaulted: bad memory access, usually from native code, an incompatible binary, or a corrupt shared library. In Kubernetes it very often means the binary was built for the wrong architecture or the base image changed underneath you. Get the crash out of the loop first (a debug pod with a shell), then reproduce the segfault directly and read what the faulting address tells you.