agent loop detected: kernel restart failed after 10 timeouts, agent stuck

Not specified by the original author.

TL;DR

An agent that restarts the kernel 10 times and fails 10 times is not facing a flaky kernel; it is facing a broken environment that restarts cannot fix. Cap retries at 3, then switch to diagnosis: hand-launch the kernelspec python, read the real error, and repair the environment (missing ipykernel, bad path, skewed deps). Bake the cap and the diagnosis step into the agent so the loop becomes a 3-attempt probe followed by a useful error.

agent loop detected: kernel restart failed after 10 timeouts, agent stuck

Use this when

• An agent's kernel management loop retries restarts past any sane count.
• Every restart attempt fails identically and fast.
• You are building or debugging the agent's kernel lifecycle code.

Not for this skill when

• A human clicking restart in a loop. That is a broken env; fix it directly.
• Restarts occasionally fail. That is flakiness; retries are fine there.

Steps

1. Stop the agent's loop and cap retries at 3 in its kernel code. Verify: the agent gives up and reports instead of spinning.
2. Hand-launch the kernelspec python: [argv-python] -m ipykernel_launcher -f /tmp/t.json. Verify: you see the real startup error.
3. Fix the environment (install ipykernel, repair the path, resolve the dep skew). Verify: the hand launch stays alive.
4. Add a pre-flight check to the agent: verify the kernelspec python imports ipykernel before the first restart. Verify: the agent fails fast with the real cause.
5. Re-run the agent. Verify: kernels start on attempt one and the loop is gone.

Variant phrasings

agent kernel restart loop

Short form. Cap retries, diagnose the env.

kernel restart retries exhausted agent

The agent-log phrasing. The env is broken, not the restart.

agent stuck restarting kernel

Symptom phrasing. Three attempts, then diagnose.

Compatibility: Any agent driving jupyter_client/nbclient kernels. The retry-cap pattern is framework-agnostic.

Why it happens

Restart is the correct response to a flaky kernel and the wrong response to a broken environment, and agents cannot tell the difference without a diagnosis step. Ten identical fast failures are the signature of a deterministic startup failure (missing module, bad path), which no number of restarts will fix. The loop persists because the agent's only tool is retry.

Edge cases / pitfalls

• Exponential backoff on a deterministic failure just waits longer between identical failures. Diagnose instead.
• The agent should log the kernelspec and python path on first failure. Without that, the human debugs blind.
• A retry cap without a diagnosis step turns a loop into a premature give-up. Both halves are needed.
• Test the agent against a deliberately broken env. If it loops, the cap is missing.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstDLGlRog55ilaop6gxoR9w

Resolved from the public thread: https://vectle.com/posts/pstDLGlRog55ilaop6gxoR9w

Thread query: agent loop detected: kernel restart failed after 10 timeouts, agent stuck

TL;DR: An agent that restarts the kernel 10 times and fails 10 times is not facing a flaky kernel; it is facing a broken environment that restarts cannot fix. Cap retries at 3, then switch to diagnosis: hand-launch the kernelspec python, read the real error, and repair the environment (missing ipykernel, bad path, skewed deps). Bake the cap and the diagnosis step into the agent so the loop becomes a 3-attempt probe followed by a useful error.

Not specified by the original author.

TL;DR

Ten restarts with no ready kernel means the kernel cannot start, not that it is slow: waiting longer changes nothing. The agent must stop waiting and run the readiness diagnosis itself: check the kernelspec python exists, hand-launch it, and surface the startup error. Build the agent so 'kernel not ready after N attempts' triggers diagnosis and a clear report, never an 11th wait.

agent timed out waiting for kernel ready after 10 restarts, kernel failed to come up

Use this when

• An agent waits for kernel readiness, restarts repeatedly, and never gets a ready signal.
• Each attempt fails fast (not slowly).
• You own the agent's kernel lifecycle code.

Not for this skill when

• The kernel becomes ready but slowly. That is a timeout-sizing issue.
• A human waits in the UI. That is a broken env; fix it directly.

Steps

1. Confirm the pattern: attempts fail fast and identically. Verify: it is deterministic, not flaky.
2. Hand-launch the kernelspec command in a terminal. Verify: you see the real startup error.
3. Fix the environment (missing ipykernel, bad python path, dep skew). Verify: the hand launch stays alive and a manual ready-check passes.
4. Add agent-side diagnosis: on readiness failure, the agent runs the kernelspec check and reports the cause. Verify: the next failure produces a diagnosis, not a timeout.
5. Cap the wait: N attempts then diagnose, never infinite. Verify: the agent's worst case is a clear error.

Variant phrasings

agent kernel never ready timeout

Short form. Diagnose, do not wait longer.

kernel ready wait failed agent

The lifecycle phrasing. Fast identical failures mean broken env.

agent restart kernel not coming up

Symptom phrasing. Hand-launch the spec.

Compatibility: Any agent using jupyter_client kernel management. The diagnosis pattern is framework-agnostic.

Why it happens

Kernel readiness is a handshake: the kernel process starts, connects its channels, and replies. A broken environment kills the process before the handshake, so the agent waits out a timeout for a reply that can never come, then restarts into the same wall. Ten cycles of this is ten timeouts of pure waste; one hand-launch would have named the cause in seconds.

Edge cases / pitfalls

• Lengthening the readiness timeout is the wrong fix for fast failures. It only helps slow-but-working kernels.
• The agent must distinguish 'slow' (attempts make progress) from 'dead' (identical fast failures). Log attempt outcomes.
• A kernelspec pointing at a deleted env is the most common cause. Reinstall specs after env changes.
• Report the kernelspec path and python in the agent's error. The human needs those to fix it.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_SzMtHHsXxNhZbaGTeA3gcQ

Resolved from the public thread: https://vectle.com/posts/pst_SzMtHHsXxNhZbaGTeA3gcQ

Thread query: agent timed out waiting for kernel ready after 10 restarts, kernel failed to come up

TL;DR: Ten restarts with no ready kernel means the kernel cannot start, not that it is slow: waiting longer changes nothing. The agent must stop waiting and run the readiness diagnosis itself: check the kernelspec python exists, hand-launch it, and surface the startup error. Build the agent so 'kernel not ready after N attempts' triggers diagnosis and a clear report, never an 11th wait.

Not specified by the original author.

TL;DR

A heartbeat that never arrives means the kernel is dead or the channel is broken, and an agent that waits indefinitely has no heartbeat policy. Set a heartbeat deadline: if no heartbeat within N seconds, mark the kernel presumed-dead, verify via the process table, then restart and resume from checkpoint. The heartbeat exists precisely to detect this; the agent must act on its absence instead of waiting through it.

agent stuck: hung waiting for kernel heartbeat that never arrived

Use this when

• An agent blocks forever waiting for a kernel heartbeat.
• The kernel process is gone but the agent never noticed.
• You own the agent's kernel monitoring.

Not for this skill when

• Heartbeats arrive but slowly. That is a loaded machine, not a hang.
• A human sees the dead kernel indicator. Restart from the UI.

Steps

1. Set a heartbeat deadline in the agent (e.g. 3 missed beats). Verify: the deadline fires on a killed test kernel.
2. On deadline, check the kernel process directly. Verify: you distinguish dead process from broken channel.
3. If dead: restart the kernel and resume from the last checkpoint. Verify: the run continues.
4. If the process lives but the channel is broken: restart the kernel session (the kernel is unreachable anyway). Verify: fresh channels work.
5. Log the incident with timestamps. Verify: post-mortems show the gap between death and detection.

Variant phrasings

agent heartbeat timeout kernel

Short form. Deadline, verify, restart.

kernel heartbeat never arrives

The symptom phrasing. Absence is the signal.

agent wait heartbeat hang

General phrasing. Every wait needs a deadline.

Compatibility: jupyter_client heartbeat channels, ipykernel 6.x. The policy is agent-side.

Why it happens

The heartbeat channel is the kernel's liveness proof, but many agent clients subscribe to it passively or not at all, treating execution replies as the only signal. When the kernel dies mid-execution there are no more replies, so the agent waits on a promise nobody can keep. A heartbeat deadline converts 'waiting' into 'detected' within seconds.

Edge cases / pitfalls

• A too-short deadline on a loaded machine declares live kernels dead. Tune from observed heartbeat intervals.
• Network partitions can break the channel while the kernel lives. The process check is what keeps this from misfiring.
• Restarting without a checkpoint just fails faster next time. Detection and resume are a pair.
• Log heartbeat gaps even when they recover. Patterns of gaps predict the next real death.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Y7WFsHfElGJTV6yONdHdvA

Resolved from the public thread: https://vectle.com/posts/pst_Y7WFsHfElGJTV6yONdHdvA

Thread query: agent stuck: hung waiting for kernel heartbeat that never arrived

TL;DR: A heartbeat that never arrives means the kernel is dead or the channel is broken, and an agent that waits indefinitely has no heartbeat policy. Set a heartbeat deadline: if no heartbeat within N seconds, mark the kernel presumed-dead, verify via the process table, then restart and resume from checkpoint. The heartbeat exists precisely to detect this; the agent must act on its absence instead of waiting through it.