how to debug a failing GitHub Actions matrix job

TL;DR

A failing GitHub Actions matrix job means one combination of your matrix axes is red while others pass. Do not re-run the whole matrix: find the failing cell in the run summary, re-run only failed jobs, and reproduce that exact combination locally. Most matrix failures are axis-specific (one OS, one Node version, one dependency combo), so isolate the cell first.

Error / query

how to debug a failing GitHub Actions matrix job

Use this skill when

• A matrix build has some green and some red cells
• You need to reproduce one failing combination locally
• A new matrix axis or value started failing
• fail-fast: true cancelled the other cells before you could compare them

Not for this skill when

• All matrix cells fail identically (the problem is in the shared steps, not the matrix)
• The workflow fails before the matrix expands (syntax or expression error in the workflow file)
• You are designing the matrix, not debugging it (strategy, not troubleshooting)
• A single non-matrix job fails (standard job debugging applies)

Steps

Step 1: Identify the exact failing cells

echo "Open the workflow run, expand the matrix job, and note which (os, version, ...) combos are red."
echo "Use 'Re-run failed jobs' instead of 'Re-run all jobs' to save time."

Expected: a short list of failing combinations, e.g. windows-latest / node-18 only. The pattern across axes is the diagnosis.

Step 2: Disable fail-fast so you can compare cells

grep -n -B2 -A8 "strategy:" .github/workflows/[workflow].yml

Expected: the strategy block is visible. Setting fail-fast: false lets all cells finish, so you can tell "fails only on Windows" apart from "fails everywhere but Windows finished first".

Step 3: Read the failing cell's logs for the axis-specific cause

echo "In the failing cell's logs, search for the first error, not the last."
echo "Compare the setup lines (installed versions, paths) against a passing cell."

Expected: the divergence point, e.g. a different Python minor version installed, a path separator issue, or a dependency that resolves differently on that OS.

Step 4: Reproduce the failing combination locally

echo "Recreate the axis values: same OS (container image if Linux), same language version, same env vars."
docker run --rm -v "$PWD":/work -w /work [os-image]:[tag] bash -c "[failing-command]"

Expected: the failure reproduces outside CI. If it does not reproduce, the difference is in the runner environment (preinstalled software, default shell, file permissions), so diff those next.

Step 5: Fix at the right scope and verify the cell

echo "If one axis value is the problem, scope the fix with an if: condition or a per-OS step, not a global change."
echo "Re-run failed jobs and confirm the cell goes green without breaking the others."

Expected: the failing cell passes and previously green cells stay green. A global fix for an axis-specific problem usually breaks another axis.

Variant phrasings

"github actions matrix one job fails"

Isolate the cell (step 1), then diff its environment against a passing sibling (step 3).

"matrix build fails on windows only"

Path separators, line endings, shell differences (PowerShell vs bash), and case sensitivity. Reproduce on a Windows runner or VM; do not guess from Linux.

"how to rerun only failed matrix jobs"

The run page has "Re-run failed jobs"; via CLI, gh run rerun [run-id] --failed. Much faster than the full matrix.

Why it happens

A matrix multiplies environments, and each axis value brings its own toolchain quirks: OS-specific paths and shells, version-specific dependency resolution, architecture differences. The shared workflow steps are usually fine; the failure lives in the interaction between the code and one specific environment combination. That is why isolating the cell beats reading the whole log.

Edge cases and pitfalls

• fail-fast: true (the default) hides the full failure pattern; turn it off while debugging.
• Matrix include:/exclude: entries can silently change which cells run; verify the actual cell list in the run, not just the YAML.
• Secrets unavailable on forks make fork-PR matrix cells fail in ways that look environmental; check the event type first.
• Caches keyed per-cell can poison one cell with stale data; include the matrix values in the cache key.
• Do not "fix" a Windows-only failure by skipping the test on Windows without a tracked issue; that is quarantine, and it needs an owner.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_OD9B4hTbsGCEPW14exNf0w