agent scheduled the notebook but the cron env had no ipykernel, execution failed
Fixes an agent scheduling a notebook whose cron env lacks ipykernel: install the kernel into the cron environment and register the spec with an absolute python path. For agent builders.
TL;DR
Cron runs with a minimal environment, so a notebook scheduled by an agent fails when the cron python has no ipykernel or resolves to the wrong interpreter. Make the schedule explicit: use the absolute path to the right python in the cron command, ensure ipykernel is installed in that python, and register the kernel spec from it. Test the exact cron command by hand before relying on the schedule.
agent scheduled the notebook but the cron env had no ipykernel, execution failedUse this when
- A scheduled notebook run fails with missing ipykernel or wrong-python errors.
- The notebook runs fine interactively.
- The schedule was set up by an agent.
Not for this skill when
- The scheduled run fails on code bugs. That is a notebook bug.
- Cron itself does not run. That is a cron problem.
Steps
- Find cron's python: add a cron line that logs
which pythonand the PATH. Verify: you see what cron actually uses. - Install ipykernel into that python:
[cron-python] -m pip install ipykernel. Verify: the import works under cron's env. - Register the spec from that python with an absolute path. Verify: kernel.json's argv is absolute.
- Write the cron command with absolute paths for python, papermill/nbconvert, and the notebook. Verify: no PATH assumptions remain.
- Run the exact command by hand in a minimal env (
env -i). Verify: it works, proving no hidden env deps.
Variant phrasings
cron notebook no ipykernel
Short form. Absolute paths, explicit env.
scheduled notebook wrong python cron
The env phrasing. Log what cron uses.
papermill cron ipykernel missing
The stack phrasing. Install into cron's python.
Compatibility: cron, papermill/nbconvert, ipykernel 6.x. The minimal-env problem is cron-specific.
Why it happens
Cron's environment is famously sparse: a minimal PATH, no conda activation, no virtualenv. An agent that schedules 'python -m papermill' gets cron's python, not the project's, and that python lacks ipykernel. Interactive testing never catches it because the interactive shell has the full environment. Absolute paths and explicit installs close the gap.
Edge cases / pitfalls
- Conda envs need activation or the env's absolute python path.
conda run -n [env]works in cron. - Secrets and env vars the notebook needs must be in the cron line or a sourced file. Cron does not inherit your shell.
- Log cron output to a file. Silent cron failures are the default.
- Test schedule changes with a short interval first, then set the real schedule.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_zd56C2d3onDQXUtsF66kNw