agent left 40 zombie kernel processes after a failed cleanup

TL;DR

Forty zombie kernels means the agent launched kernels and never shut them down on failure paths. Find them with ps aux | grep ipykernel and jupyter server list, kill the orphans, and clear their stale connection files from the runtime dir. Then fix the agent: every kernel launch must be paired with a shutdown in a finally block, including on exceptions and timeouts. Zombies are a resource leak with the agent's name on it.

agent left 40 zombie kernel processes after a failed cleanup

Use this when

• Dozens of kernel processes outlive the agent run that started them.
• Ports are exhausted or the machine is slow after agent runs.
• Stale connection files pile up in the runtime dir.

Not for this skill when

• A human's forgotten kernels. Shut them down from the UI.
• Kernels are alive but idle. That is waste, not zombies.

Steps

1. List the zombies: ps aux | grep -E "ipykernel|kernel-" | grep -v grep. Verify: you see the orphaned processes.
2. Kill them: pkill -f ipykernel_launcher (or kill by PID after review). Verify: the process list is clean.
3. Clear stale connection files: remove kernel-*.json for dead kernels from jupyter --runtime-dir. Verify: no stale files remain.
4. Fix the agent: wrap every kernel lifecycle in try/finally with shutdown, including timeout and exception paths. Verify: a failing run leaves zero kernels.
5. Add a post-run audit: the agent lists kernels before and after and reports orphans. Verify: the next run is clean.

Variant phrasings

zombie kernel processes cleanup

Short form. Kill, clear, then fix the lifecycle.

agent leaked kernels

The agent phrasing. finally-block shutdown.

orphaned ipykernel processes

The process phrasing. Audit after every run.

Compatibility: ipykernel 6.x, any OS. The finally-block discipline is language-agnostic.

Why it happens

Kernel shutdown is easy to forget on the unhappy paths: the code that launches the kernel is written first, the exception handling is added later, and the timeout path never gets a cleanup branch. Each failed run then leaks its kernels, and because kernels are quiet (no UI, little CPU when idle), the leak is invisible until ports run out or the OOM killer starts choosing victims.

Edge cases / pitfalls

• Killing kernels by pattern can hit a human's live work on shared machines. Review PIDs before pkill.
• Stale connection files can confuse the next run into attaching to a dead kernel. Clear them with the processes.
• A shutdown that hangs (wedged kernel) needs a kill fallback with a timeout. Shutdown, then kill.
• Count zombies in CI. A growing count is the earliest signal the lifecycle code regressed.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst15gBzpJ66AG4k6zOzyENA