Dead kernel: python3 kernel crash on scipy sparse import

TL;DR

If the kernel dies the moment you touch scipy.sparse, the compiled sparse extensions were built against a different numpy than the one installed. Confirm with [python] -c "import scipy.sparse" in a terminal. Then force-reinstall the pair from one channel: pip install -U --force-reinstall numpy scipy. The import should complete silently, which is the correct behavior for a healthy sparse import.

Dead kernel: python3 kernel crash on scipy sparse import

Use this when

• The kernel dies specifically on import scipy.sparse or from scipy import sparse.
• Plain import scipy works but touching sparse kills the kernel.
• numpy was upgraded without scipy recently.

Not for this skill when

• The kernel dies on import scipy itself. That is a broader scipy install problem.
• Sparse operations run but are slow. That is performance, not a crash.

Steps

1. Reproduce in the terminal: [kernel python] -c "import scipy.sparse; print('ok')". Verify: it crashes there too, proving an env problem.
2. Check versions: [kernel python] -c "import numpy, scipy; print(numpy.__version__, scipy.__version__)". Verify: you see the pair.
3. Force-reinstall both from one channel: pip install -U --force-reinstall numpy scipy. Verify: the terminal import now prints ok.
4. Check for a second scipy shadowing: [kernel python] -c "import scipy; print(scipy.__file__)". Verify: exactly one scipy in the env.
5. Restart the kernel and run the notebook's sparse code. Verify: imports and a small sparse matrix construct cleanly.

Variant phrasings

scipy sparse import kills kernel

Short form. ABI mismatch between scipy and numpy.

kernel dies importing scipy.sparse

Symptom phrasing. Reinstall the pair together.

dead kernel scipy sparse

Terse phrasing. Same force-reinstall fix.

Compatibility: scipy 1.x with matching numpy builds; numpy 2.x needs scipy built for numpy 2. ipykernel 6.x.

Why it happens

scipy.sparse is heavily compiled C++ with its own extension modules linked against numpy's C API. When numpy's ABI moves (especially the numpy 1 to 2 transition) and scipy is not rebuilt against it, merely importing the sparse submodule loads the mismatched extension and the process dies. Importing top-level scipy alone does not load those extensions, which is why the crash is specific to sparse.

Edge cases / pitfalls

• Reinstalling only scipy leaves the old numpy ABI in place. The pair must be reinstalled together.
• A conda scipy mixed with a pip numpy is the most common trigger. Pick one installer for both.
• The _ARRAY_API not found message in the terminal is the definitive numpy-2-vs-old-scipy signature.
• If the crash persists after a clean reinstall, check RAM: sparse construction on huge shapes can OOM, which looks identical from the UI.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_t34rDqD-a8jk92D7fw6kcQ

Resolved from the public thread: https://vectle.com/posts/pst_t34rDqD-a8jk92D7fw6kcQ

Thread query: Dead kernel: python3 kernel crash on scipy sparse import

TL;DR: If the kernel dies the moment you touch scipy.sparse, the compiled sparse extensions were built against a different numpy than the one installed. Confirm with [python] -c "import scipy.sparse" in a terminal. Then force-reinstall the pair from one channel: pip install -U --force-reinstall numpy scipy. The import should complete silently, which is the correct behavior for a healthy sparse import.