docker buildx "process '/bin/sh -c ...' did not complete successfully": RUN failure debugging

TL;DR

A failing RUN line means the command exited nonzero inside the build container, and buildx hides the real error a few lines above the failure marker. Rerun the build with plain progress output to see the failing command's own stderr, then reproduce the exact command outside the build.

process '/bin/sh -c ...' did not complete successfully: exit code: [N]

Steps

1. Rerun the build with plain progress so the failing command's output is visible:

docker buildx build --progress=plain --no-cache -t [IMAGE]:[TAG] -f [DOCKERFILE] [CONTEXT] | tee [LOGFILE]

Expected: the log shows the exact RUN line plus the command's own error output just above the failure marker. Read upward, not at the marker itself.

2. Identify the failing stage and line from the log:

grep -n "did not complete successfully" [LOGFILE]

Expected: the step number and the /bin/sh -c invocation that failed.

3. Reproduce the failing command in a matching base image to iterate fast:

docker run --rm -it [BASE_IMAGE]:[TAG] sh

Expected: an interactive shell in the same base image, where you can paste the failing RUN body line by line and see which part exits nonzero.

4. Once the culprit is found, harden the RUN line so the failure is loud and located:

RUN set -eux; \
    [COMMAND_ONE]; \
    [COMMAND_TWO]

Expected: the build log now prints each command before running it (-x) and stops at the first failure (-e), instead of one opaque line.

5. Rebuild and confirm the step passes:

docker buildx build --progress=plain -t [IMAGE]:[TAG] -f [DOCKERFILE] [CONTEXT]

Expected: the previously failing step completes and the build proceeds to the next stage.

When to use

• docker buildx build fails with a /bin/sh -c nonzero exit on a RUN line
• A Dockerfile that builds locally fails in CI on the same step
• Multi-stage builds fail in a later stage with package or compile errors

When not to use

• The failure is COPY failed: no such file or a missing build arg, which is a context problem, not a command failure
• Buildx fails with daemon or builder errors before any step runs
• The failure is a flaky network fetch inside the step; see the step about retrying before rewriting the command

Why it happens

Each RUN line executes as a single /bin/sh -c string, so the first failing subcommand aborts the rest and buildx reports only the wrapper's exit code. In CI the default progress output also collapses step logs, hiding the command's own stderr. Locally cached layers can additionally mask the failure because the bad step never reruns.

Edge cases

• set -e does not catch failures inside pipes unless pipefail is set; add set -o pipefail when the line pipes output
• APT or APK transient network errors fail the step with exit 100; retry the fetch with a retry flag or a small retry loop rather than restructuring the Dockerfile
• Commands that succeed with warnings locally can fail under set -e if they return nonzero on warnings; check the actual exit code before blaming the tool
• BuildKit cache mounts can persist a corrupted state between builds; --no-cache is the honest rerun when debugging

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_j8yhZkX7sAFR2SgJVD45Yw