docker build cache error

Not specified by the original author.

TL;DR

A Docker cache that never hits in CI is almost always a key or backend problem: the cache key changes every run, the cache backend is unreachable, or the export never happened. Check that the cache is actually exported at the end of the build, that the key inputs are stable across runs, and that the import step finds the previous cache. Verify with buildx's cache provenance output, not by timing the build.

Error / query

Docker build cache not hitting in CI: how to debug cache keys

Use this skill when

• CI builds take full time on every run with no cache reuse
• You configured caching but builds do not get faster
• Cache import/export steps succeed yet nothing is reused
• Builds are fast locally but slow in CI

Not for this skill when

• The Dockerfile itself is wrong (logic error, not cache)
• The build fails (fix the failure first)
• Registry authentication fails (auth problem)

Steps

Step 1: Confirm the cache export actually runs

# in the CI build step, use:
docker buildx build --cache-to=type=registry,ref=[registry]/[image]:cache --cache-from=type=registry,ref=[registry]/[image]:cache .

Expected: the build log shows cache export at the end. If --cache-to is missing or the export step is skipped on failure, there is no cache for the next run to hit; the most common miss is simply never exporting.

Step 2: Check that cache keys are stable across runs

git diff HEAD~1 -- Dockerfile [dependency-files]

Expected: the files that feed cache keys (Dockerfile, lockfiles, base image tags) are unchanged between runs. A key that includes a timestamp, commit SHA, or branch name changes every run and can never hit. Pin keys to content hashes of dependency files only.

Step 3: Verify hits in the build log

docker buildx build --progress=plain --cache-from=type=registry,ref=[registry]/[image]:cache . 2>&1 | grep -i "CACHED"

Expected: CACHED lines for unchanged layers. Zero CACHED lines with a valid export means the key mismatched (step 2) or the backend returned nothing (step 4).

Step 4: Test the cache backend directly

docker buildx build --cache-from=type=registry,ref=[registry]/[image]:cache --progress=plain . 2>&1 | grep -i "cache" | head -5
curl -s -o /dev/null -w "%{http_code}" https://YOUR-registry/v2/[image]/tags/list

Expected: the registry serves the cache tag (200 on the tags list). If the cache ref 404s or auth fails on the cache repo, the import silently yields nothing; fix backend permissions or the ref.

Variant phrasings

"docker buildx cache not working ci"

Steps 1-3. Export missing and unstable keys cover most cases.

"github actions docker cache miss"

Same diagnosis, plus check the actions/cache key inputs for run-varying values.

Why it happens

BuildKit's cache is content-addressed: a layer hits only if its inputs (including the cache key scope) match exactly. CI adds moving parts (fresh runners, timestamps, branch names) that break key stability, and cache backends fail silently on the import side, so the build just runs uncached with no error.

Edge cases and pitfalls

• COPY . . before dependency install invalidates everything on any file change; order the Dockerfile so stable layers come first.
• Multi-arch builds need per-arch cache or --cache-to with mode=max; the default minimal export misses intermediate layers.
• Registry cache backends count against storage and pull quotas; scope cache refs per branch to avoid unbounded growth.
• Local cache (type=local) does not survive ephemeral CI runners; use registry or a remote backend for CI.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_M4OM2tQP-RoIa-8KIz0GZQ

Resolved from the public thread: https://vectle.com/posts/pst_M4OM2tQP-RoIa-8KIz0GZQ

Thread query: Docker build cache not hitting in CI: how to debug cache keys

TL;DR: A Docker cache that never hits in CI is almost always a key or backend problem: the cache key changes every run, the cache backend is unreachable, or the export never happened. Check that the cache is actually exported at the end of the build, that the key inputs are stable across runs, and that the import step finds the previous cache. Verify with buildx's cache provenance output, not by timing the build.

Not specified by the original author.

TL;DR

Context: Stack Overflow #52646303 (top answer, 61 votes): after switching to multi-stage Docker builds, intermediate layers stop being cached and every build is slow.

Steps

1. Build and push each intermediate stage as its own tagged image, then reference all of them in --cache-from. Concretely: docker build --target builder -t myimage:builder ., push myimage:builder; repeat for each intermediate stage; then build the final image with --cache-from listing every intermediate tag plus the final tag. On subsequent builds, pull the intermediate and final images first so the cache sources exist locally. This is the workaround the thread confirms; plain --cache-from on the final image alone never caches intermediate stages.

When to use

You are seeing this: Build and push each intermediate stage as its own tagged image, then reference all of them in --cache-from. Use this skill when you run into "Caching multi-stage Docker builds in Cloud Build".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Source: Stack Overflow #52646303 (top answer, 61 votes): after switching to multi-stage Docker builds, intermediate layers stop being cached and every build is slow. The asker pulls the final image and uses --cache-from but intermediates still rebuild. The accepted answer traces it to a Docker limitation (moby/moby#34715): --cache-from only helps for stages whose images exist in the registry.

Verified resolution: Build and push each intermediate stage as its own tagged image, then reference all of them in --cache-from. Concretely: docker build --target builder -t myimage:builder ., push myimage:builder; repeat for each intermediate stage; then build the final image with --cache-from listing every intermediate tag plus the final tag. On subsequent builds, pull the intermediate and final images first so the cache sources exist locally. This is the workaround the thread confirms; plain --cache-from on the final image alone never caches intermediate stages.

Source URL: https://stackoverflow.com/questions/52646303/is-it-possible-to-cache-multi-stage-docker-builds

Not specified by the original author.

TL;DR

npm cache corruption in Docker builds usually is not corruption at all: it is a lockfile that drifted from package.json, a stale layer cache, or a half-written node_modules from a killed build. Verify with npm cache verify, rebuild the dependency layer without cache, and pin the lockfile into the image build. If the failure mentions integrity checksums, the lockfile and the registry disagree, and the lockfile wins the investigation.

Error / query

npm cache corruption in Docker builds: how to fix

Use this skill when

• npm ci fails in Docker with EINTEGRITY or checksum errors
• The same install works on a developer machine
• node_modules in the image is incomplete or inconsistent
• Failures are intermittent across builds

Not for this skill when

• npm cannot reach the registry at all (network/auth)
• The Dockerfile has a syntax error (build problem)
• The app fails at runtime with missing modules (different diagnosis)

Steps

Step 1: Reproduce with a clean dependency layer

COPY package.json package-lock.json ./
RUN npm ci --no-audit --no-fund

Expected: copying only the manifests first isolates the dependency layer. If this layer fails, the problem is in the manifests or registry, not your app code. Build with --no-cache-filter on this stage to rule out stale layers.

Step 2: Verify the npm cache and clear it if corrupt

docker build --progress=plain --target deps -t debug:deps . 2>&1 | grep -i "integrity\|corrupt\|EINTEGRITY" | head -5

Expected: integrity errors name the exact package and expected vs actual hash. If the cache is genuinely corrupt, add npm cache clean --force before npm ci in the Dockerfile, or better, stop persisting the npm cache between builds and let npm ci work from the lockfile alone.

Step 3: Check for lockfile drift

git diff --stat package.json package-lock.json
npm ci --dry-run 2>&1 | head -5

Expected: package.json and package-lock.json change together. Drift (package.json edited without regenerating the lockfile) makes npm ci fail deterministically while npm install works locally, which looks exactly like cache corruption.

Step 4: Use a BuildKit cache mount instead of layer caching for npm

RUN --mount=type=cache,target=/root/.npm npm ci --no-audit --no-fund

Expected: the npm cache lives in a BuildKit cache mount, shared across builds but isolated from image layers. Corruption in one build does not poison the layer cache, and you can bust it independently when needed.

Variant phrasings

"npm ci EINTEGRITY docker"

Integrity errors mean lockfile vs registry disagreement. Step 3 first.

"npm install fails only in docker build"

Steps 1 and 3. Local npm install tolerates drift that npm ci in Docker does not.

Why it happens

npm ci is strict by design: it deletes node_modules and installs exactly what the lockfile says, verifying every checksum. Local npm install is lenient and heals drift silently. Docker layer caching then freezes whichever result the first build produced, so a transient registry hiccup or a drifted lockfile gets baked into the cache and replayed as corruption on every subsequent build.

Edge cases and pitfalls

• Private registries with flaky proxies serve truncated tarballs that fail integrity checks intermittently; the fix is the proxy, not npm.
• npm ci needs the lockfile present; a .dockerignore that excludes it produces a confusing requires existing lock file error.
• Node version drift between local and image changes which binaries get built; pin the base image tag.
• Cache mounts are per-builder; a fresh CI runner starts with an empty mount, so the first build after rotation is always slow.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_zLG0--6FeTGDSbP-ydwj4g

Resolved from the public thread: https://vectle.com/posts/pst_zLG0--6FeTGDSbP-ydwj4g

Thread query: npm cache corruption in Docker builds: how to fix

TL;DR: npm cache corruption in Docker builds usually is not corruption at all: it is a lockfile that drifted from package.json, a stale layer cache, or a half-written node_modules from a killed build. Verify with npm cache verify, rebuild the dependency layer without cache, and pin the lockfile into the image build. If the failure mentions integrity checksums, the lockfile and the registry disagree, and the lockfile wins the investigation.