jupyter kernelspec list shows the kernel but the notebook cannot find it

TL;DR

jupyter kernelspec list shows specs from every search path on that machine, but the notebook server only sees the paths visible to its own process and environment. Run jupyter --paths as the server user and compare the data directories against the spec's location; the spec is in a directory the server never scans. Move or reinstall the kernelspec into a directory the server actually reads, then restart the server so it rescans.

jupyter kernelspec list shows the kernel but the notebook cannot find it

Use this when

• The CLI lists a kernel name, but the notebook UI's kernel picker does not offer it.
• A kernel works when you start the server one way (for example from a terminal) but not another (for example a system service).
• An agent installs a kernelspec and the subsequent server session still cannot see it.

Not for this skill when

• The kernel is listed AND selectable but dies on start. That is a broken spec or missing ipykernel, not a search-path problem.
• The spec's argv points at a Python that no longer exists. That is the dead-interpreter case, a different fix.

Steps

1. List specs with their full paths: jupyter kernelspec list shows the name-to-directory mapping. Expected: you see the kernel name and the exact directory holding its kernel.json.
2. Show the server's search paths: run jupyter --paths in the same context that launches the server (same user, same service, same container). Expected: a data-directories list you can compare against the spec directory.
3. Identify the mismatch: check whether the spec directory sits under one of the server's data directories. Expected: you can name the directory the server scans that does NOT contain the spec.
4. Reinstall the spec where the server looks: run python -m ipykernel install --user --name mykernel --display-name "My Kernel" with the Python of the intended kernel, using --user or --sys-prefix to match the server's visible path. Expected: jupyter kernelspec list now shows the spec under a server-visible directory.
5. Restart the notebook server so it rescans kernelspecs. Expected: the kernel appears in the UI kernel picker and in jupyter kernelspec list run as the server user.
6. If a system service starts the server, repeat the path check as that service account, not as your login user. Expected: the service account's data directories match where the spec was installed.

Tool compatibility

jupyterclient 7.x and 8.x, jupytercore 5.x, notebook 6.x/7.x, JupyterLab 3.x/4.x, JupyterHub single-user servers.

Variant phrasings

kernel not showing up in jupyter notebook kernel list

The UI-side phrasing of the same search-path split. Compare jupyter --paths contexts.

kernelspec installed but not visible in jupyterhub

The multi-user version. The Hub's single-user servers read different data directories per user.

jupyter kernelspec list vs notebook kernel picker mismatch

The diagnostic phrasing. The CLI and the server can legitimately disagree when paths differ.

Why it happens

Kernelspec discovery is a filesystem scan over a list of data directories that depends on the user, the installation prefix, and environment variables at the moment the server starts. The jupyter kernelspec list command you run in your terminal uses YOUR directories, which can include a user-level or conda-env-level path that a server started by a service account, a container entrypoint, or JupyterHub never sees. Both sides are telling the truth about what they can see; the bug is the assumption that there is one global kernel list. Installing with --user versus --sys-prefix chooses which directory wins, so the fix is to install the spec into a directory that is on the server's list, not just on yours.

Edge cases / pitfalls

• --sys-prefix installs into the current environment's share directory, which is invisible to a server running in a different environment. Match the flag to the server, not to your shell.
• Environment variable overrides for data directories apply at server start time. A spec installed after the server started requires a server restart to be picked up.
• Two specs with the same kernel name in different directories shadow each other. The first one found on the search path wins, which can surprise you after a reinstall.
• On JupyterHub, per-user servers each have their own user-level directory, so a spec installed by one user is correctly invisible to everyone else.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_SYiSWsW99riSaiK78pwMvA