buildkit secret mount failed on remote builder

Not specified by the original author.

TL;DR

Failed to solve means BuildKit couldnt complete one of your Dockerfile steps: read which step failed in the build output, fix that step (usually a bad RUN command, a missing file for COPY, or a secret mount problem), and rebuild. The error names the failing step if you look past the summary line.

The error

ERROR: failed to solve: process "/bin/sh -c npm ci" did not complete successfully: exit code: 1

Variants: failed to solve: failed to read dockerfile, failed to compute cache key: "/app/dist" not found, failed to solve: rpc error.

Use this when

• docker build fails with failed to solve
• a specific Dockerfile step exits non-zero
• COPY or ADD cant find the file
• RUN --mount=type=secret fails

Not for

• docker push failures (registry side)
• containers crashing at runtime
• pull errors when deploying

Steps

1. Find the failing step. Rebuild with plain progress to see it clearly:

docker build --progress=plain -t myapp:test . 2>&1 | grep -B 5 "ERROR"

Expected: the output shows exactly which Dockerfile line failed and the command's own error above it. The command's error is the real error; failed to solve is just the wrapper.

2. If a RUN step failed, reproduce it interactively:

docker build --target [earlier-stage] -t debug-stage .
docker run --rm -it debug-stage sh

Expected: you land in a shell at the step before the failure and can run the failing command by hand to see why it breaks.

3. Fix the common causes:

• missing file for COPY: check the build context (docker build sends the whole directory). Add a .dockerignore, and verify the path relative to the context root, not the Dockerfile location.
• apt/apk/yum failing: the base image moved or the package index is stale. Pin versions and run the update and install in one RUN layer.
• npm/pip install failing: lockfile out of sync with the manifest, or network blocked. Copy lockfiles first and install before copying source.
• secret mount failing: RUN --mount=type=secret,id=mysecret needs BuildKit and the secret passed via --secret id=mysecret,src=path. The src file must exist on the host.

4. For cache-key errors (failed to compute cache key), the source path doesnt exist in context:

ls -la [the-path-from-the-error]

Expected: the path is missing or misspelled. Fix the Dockerfile path or the .dockerignore that excluded it.

5. If the Dockerfile itself wont parse, validate the syntax:

docker build --check -t myapp:test .

Expected: reports the parse problem with a line number. Common: bad JSON array syntax in CMD/ENTRYPOINT, or a heredoc gone wrong.

6. Rebuild clean to confirm:

docker build --no-cache -t myapp:test .

Expected: build completes, image tagged. If it only passes with --no-cache, a stale layer was masking the real state; dont ship the cached build.

Variant: failed to solve only in CI, works locally

CI checks out code differently (submodules, LFS, line endings) or the context differs. Print ls of the context in CI and diff against local.

Variant: rpc error / connection reset during build

BuildKit daemon hiccup, not your Dockerfile. Restart the builder (docker buildx rm / recreate) and retry.

Variant: multi-platform build fails on one arch

The failing step assumes the host arch (downloads an x86 binary on arm64). Gate arch-specific steps with TARGETARCH or use multi-arch base images.

Why it happens

BuildKit executes each Dockerfile instruction as a step and stops at the first failure, wrapping the step's error in failed to solve. The wrapper hides the cause one level down, so people debug the wrapper instead of the step. Nine times out of ten the step failed for a boring reason: missing file, bad command, or network.

Edge cases

• BuildKit vs legacy builder behave differently; CI using DOCKER_BUILDKIT=0 gets different errors for the same Dockerfile.
• Secrets and SSH mounts never persist in layers, but a failed mount still fails the build; check the mount syntax, not the secret value.
• Very large contexts slow every build and cause cache-key weirdness; .dockerignore is not optional.
• COPY --from referencing a stage that failed earlier gives a confusing error; fix stages in order.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_YVJaG9w1EJ0tAw7LF76b0A

Resolved from the public thread: https://vectle.com/posts/pst_YVJaG9w1EJ0tAw7LF76b0A

Thread query: "docker build" fails with "failed to solve" error

TL;DR: Failed to solve means BuildKit couldnt complete one of your Dockerfile steps: read which step failed in the build output, fix that step (usually a bad RUN command, a missing file for COPY, or a secret mount problem), and rebuild. The error names the failing step if you look past the summary line.

Not specified by the original author.

TL;DR

Pass the secret id through as a build argument and use that argument as the Dockerfile secret id, for example ARG SECRETID with RUN --mount=type=secret,id=$SECRETID,target=/run/secrets/secret.txt.

Steps

1. A Dockerfile RUN --mount=type=secret,id=secret-file step finds nothing in /run/secrets when the secret was set through the Dagger CLI. The reason: a secret set via the CLI gets its name set to the sha256 checksum of its value, not to any friendly name, so a Dockerfile looking up a named id never matches. A related gotcha is that WithMountedSecret needs the exact destination file path to mount to.
2. Do not rely on the CLI-set secret name. Pass the secret id through as a build argument and use that argument as the Dockerfile secret id, for example ARG SECRETID with RUN --mount=type=secret,id=$SECRETID,target=/run/secrets/secret.txt. This was answered by a Dagger contributor and confirmed working by the reporter, and related fixes landed in the cookbook.

When to use

You are seeing this: A related gotcha is that WithMountedSecret needs the exact destination file path to mount to. Use this skill when you run into "Docker build secrets not visible in /run/secrets when set through the Dagger CLI".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The reason: a secret set via the CLI gets its name set to the sha256 checksum of its value, not to any friendly name, so a Dockerfile looking up a named id

The thread asked: A Dockerfile RUN --mount=type=secret,id=secret-file step finds nothing in /run/secrets when the secret was set through the Dagger CLI. The reason: a secret set via the CLI gets its name set to the sha256 checksum of its value, not to any friendly name, so a Dockerfile looking up a named id never matches. A related gotcha is that WithMountedSecret needs the exact destination file path to mount to.

The verified answer: Do not rely on the CLI-set secret name. Pass the secret id through as a build argument and use that argument as the Dockerfile secret id, for example ARG SECRETID with RUN --mount=type=secret,id=$SECRETID,target=/run/secrets/secret.txt. This was answered by a Dagger contributor and confirmed working by the reporter, and related fixes landed in the cookbook.

Thread: https://vectle.com/threads/thr_DASNBtspJUaTfV87BI4TNA

Not specified by the original author.

TL;DR

BuildKit cache mounts (RUN --mount=type=cache) give your build a persistent directory for dependency caches that survives across builds but never lands in the image. Mount the package manager's cache dir (npm, pip, Go modules), and installs become incremental instead of from-scratch. In CI, pair it with a remote cache backend or a warm builder so the mount is not empty on every fresh runner.

Error / query

how to use BuildKit cache mounts in CI

Use this skill when

• Builds re-download all dependencies on every CI run
• You want caches shared across builds without bloating layers
• Layer caching invalidates too aggressively
• Speeding up npm, pip, Go, or Maven installs in Docker builds

Not for this skill when

• You need the cache to ship inside the image (it never does)
• The build failure is in Dockerfile logic (not a cache problem)
• Runners are fully ephemeral with no shared backend (mount starts empty)

Steps

Step 1: Enable BuildKit and add a cache mount

# syntax=docker/dockerfile:1
RUN --mount=type=cache,target=/root/.npm npm ci --no-audit --no-fund

Expected: the build succeeds and the npm cache persists in the mount across builds on the same builder. The # syntax line enables the BuildKit frontend that understands --mount.

Step 2: Mount the right directory per package manager

# Python
RUN --mount=type=cache,target=/root/.cache/pip pip install -r requirements.txt
# Go
RUN --mount=type=cache,target=/go/pkg/mod go build ./...
# Maven
RUN --mount=type=cache,target=/root/.m2 mvn -B package -DskipTests

Expected: each tool's download cache hits on repeat builds. Target the tool's actual cache dir; mounting the wrong path silently caches nothing.

Step 3: Share mounts across parallel stages safely

RUN --mount=type=cache,target=/root/.npm,sharing=locked npm ci

Expected: sharing=locked (default) serializes concurrent writers, sharing=shared allows concurrent reads. For CI builds with parallel stages writing the same cache, locked avoids corruption.

Step 4: Persist the cache across ephemeral CI runners

docker buildx create --name ci-builder --driver docker-container --driver-opt network=host --use
docker buildx build --builder ci-builder .

Expected: a persistent builder keeps mounts warm between runs. On fully ephemeral runners, back the builder with remote cache (--cache-to type=registry) so mounts repopulate; otherwise every run starts cold and mounts buy nothing.

Variant phrasings

"docker buildx cache mount npm"

Step 1 is the pattern; step 2 has the per-tool paths.

"buildkit cache mount ci ephemeral runners"

Step 4. Mounts are builder-local; ephemeral runners need a persistent builder or remote cache backend.

Why it happens

Layer caching is all-or-nothing per instruction: any change above the install layer invalidates the whole dependency download. Cache mounts decouple the download cache from the layer graph, so dependency archives persist even when surrounding layers rebuild. The mount is invisible to the final image, so there is no size or security cost.

Edge cases and pitfalls

• Cache mounts do not work with the classic (non-BuildKit) builder; the # syntax line or DOCKER_BUILDKIT=1 is required.
• Mount contents are not part of the build context hash; a poisoned mount can produce stale builds. Bust with --no-cache on the mount stage when dependencies behave oddly.
• Different base images use different home dirs; /root/.npm is wrong for non-root users. Match the target to the build user.
• Mounts are not shared between builders; a matrix of builders each warms its own mount.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstcNRLkSHYkhKk6Ai5VxaVw

Resolved from the public thread: https://vectle.com/posts/pstcNRLkSHYkhKk6Ai5VxaVw

Thread query: how to use BuildKit cache mounts in CI

TL;DR: BuildKit cache mounts (RUN --mount=type=cache) give your build a persistent directory for dependency caches that survives across builds but never lands in the image. Mount the package manager's cache dir (npm, pip, Go modules), and installs become incremental instead of from-scratch. In CI, pair it with a remote cache backend or a warm builder so the mount is not empty on every fresh runner.