# Jupyter kernel-environment reproducibility
**Problem:** a notebook "works in my notebook" but fails when rerun from a fresh kernel, on another machine, or via `nbconvert`. Usual causes: the notebook is attached to the wrong kernel (the server's env, not the project's), packages were installed with `!pip` into a different Python, the kernel was never restarted after an install, or nothing pins the dependency versions.
## 1. Pin the notebook to a real environment
A Jupyter *kernel* is a separate process. By default a notebook often runs on the Python that hosts the Jupyter server — not your project's venv or conda env. Register your environment as a named kernel:
```bash
# Run from the KERNEL'S environment (the venv/conda env the notebook should use),
# not necessarily the env running the Jupyter server.
python -m pip install ipykernel
python -m ipykernel install --user --name PROJECT_ENV --display-name "Python (project-env)"
```
- `--name` is the internal ID (no spaces) used by Jupyter; `--display-name` is what you see in the kernel picker.
- Select that kernel in the notebook (Kernel > Change kernel), or create the notebook from it.
- Inspect and remove stale kernelspecs:
```bash
jupyter kernelspec list
jupyter kernelspec uninstall STALE_KERNEL_NAME
```
- Sanity check from inside a cell: `import sys; print(sys.executable)` should point at your project's Python.
## 2. Install packages into the kernel's environment
Inside a notebook cell, use the `%pip` magic — never `!pip`:
```
%pip install package-name
```
- `%pip` installs into the environment backing the *running kernel*.
- `!pip` runs the shell's `pip`, which may resolve to a different Python (e.g. the notebook server's env). The install "succeeds" but `import` still fails in the notebook.
## 3. Restart the kernel after installing anything
If `import` fails right after installing, restart the kernel (Kernel > Restart) and re-run the cells. And the only real reproducibility test is **Restart & Run All** top-to-bottom — it exposes hidden state from cells you previously ran out of order.
## 4. Lock the environment so someone else can recreate it
For pip/venv projects:
```bash
pip freeze > requirements.txt # exact pins of everything installed
pip install -r requirements.txt # recreate elsewhere
```
For conda:
```bash
conda env export > environment.yml # full snapshot (versions + builds)
conda env export --no-builds > environment.yml # more portable across machines
conda env export --from-history > environment.yml # only the packages you asked for
conda env create -f environment.yml
conda activate ENV_NAME
```
- Check `requirements.txt` / `environment.yml` in next to the notebook.
- After recreating the env, re-register the kernel with the *same* `--name`, so the kernelspec stored in the notebook still matches on the new machine.
## 5. nbconvert execution gotchas when sharing
Headless execution uses the kernel recorded in the notebook's metadata:
```bash
jupyter nbconvert --to notebook --execute NOTEBOOK.ipynb
```
- The kernel name comes from the notebook metadata. If the recipient has no kernelspec with that name, execution fails — override it: `--ExecutePreprocessor.kernel_name=PROJECT_ENV`.
- Per-cell timeout defaults to 30 seconds; raise it (`--ExecutePreprocessor.timeout=600`) or disable with `-1` for long jobs.
- By default the first failing cell raises `CellExecutionError` and stops; `--ExecutePreprocessor.allow_errors=True` records errors in the output and continues.
- Executing from a clean checkout + fresh env gives the true "does it rerun" answer.