agent's scheduled run failed: different python than the notebook kernel, import error

TL;DR

Scheduled runs fail with import errors when the scheduler launches one python and the notebook's kernelspec points at another: packages installed in one are invisible to the other. Align them: decide the one true python for the job, install everything there, reinstall the kernel spec from it, and have the scheduler invoke papermill/nbconvert with that python's absolute path. Verify with the scheduler's own environment, not yours.

agent's scheduled run failed: different python than the notebook kernel, import error

Use this when

• Scheduled runs raise ImportError for packages that are installed.
• Interactive runs work; scheduled runs do not.
• The scheduler and the notebook were set up at different times.

Not for this skill when

• The package is missing everywhere. Install it.
• Both interactive and scheduled fail. That is a general env problem.

Steps

1. Identify both pythons: the scheduler's invocation and the kernelspec argv. Verify: you have both absolute paths.
2. Confirm the mismatch: compare sys.path and pip list between them. Verify: the missing package is present in one only.
3. Choose one python for the job. Verify: the decision is recorded.
4. Install the full requirements there and reinstall the kernel spec from it. Verify: the spec argv matches the scheduler's python.
5. Run the scheduled command in the scheduler's context. Verify: imports succeed.

Variant phrasings

scheduled notebook import error different python

Short form. Align the interpreters.

kernelspec python mismatch scheduler

The spec phrasing. Reinstall from the right python.

agent scheduled run ModuleNotFoundError

The error phrasing. The package is in the other python.

Compatibility: Any scheduler (cron, Airflow, systemd), ipykernel 6.x. The one-python rule is universal.

Why it happens

Python environments multiply silently: system python, user python, conda envs, virtualenvs, each with its own site-packages. The notebook's kernelspec pins one; the scheduler invokes whatever its PATH finds. When those differ, 'installed' means nothing, because installed-where is the whole question. Aligning on a single absolute python path eliminates the ambiguity.

Edge cases / pitfalls

• python on the PATH is not a stable identity across contexts. Always use absolute paths in schedules.
• Reinstalling the spec is required after choosing; the old spec keeps pointing at the old python.
• User-site packages (--user installs) differ per user. Scheduled jobs running as another user lose them.
• Document the chosen python in the runbook. The next person will otherwise 'fix' it back.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst28zJgZxX8B4Cij5F4CHXA