agent's top-to-bottom execution failed: outputs reference stale variables, wrong results

TL;DR

When an agent executes a notebook and outputs reference variables from an earlier, different run, the kernel had stale state: the agent executed out of order, skipped cells, or reused a dirty kernel. Make the agent always start from a fresh kernel, execute strictly top to bottom, and verify by re-running: if a second clean run differs from the first, the notebook has order dependencies to fix. Stale-variable bugs are notebook bugs the agent must surface, not silently inherit.

agent's top-to-bottom execution failed: outputs reference stale variables, wrong results

Use this when

• An agent's executed notebook has outputs inconsistent with the code.
• Re-running interactively gives different results than the agent's run.
• The agent reuses kernels across runs or executes cells selectively.

Not for this skill when

• A human's interactive run is inconsistent. Restart-and-run-all is the manual fix.
• The code is simply wrong. That is a logic bug, not staleness.

Steps

1. Force a fresh kernel per agent run; never reuse. Verify: the agent's run starts from empty state.
2. Execute strictly top to bottom with no skips. Verify: execution counts are sequential in the output.
3. Add a staleness check: run twice from clean kernels and diff key outputs. Verify: the runs agree.
4. When they disagree, find the order-dependent cell (uses a variable before its definition cell runs). Verify: you can name the cell.
5. Fix the notebook: define before use, remove hidden state. Verify: double runs agree.

Variant phrasings

agent notebook stale variables

Short form. Fresh kernel, strict order.

notebook outputs wrong agent execution

The symptom phrasing. Diff two clean runs.

top to bottom execution inconsistent

The order phrasing. Hidden dependencies are the cause.

Compatibility: nbclient/papermill-driven execution. The fresh-kernel rule applies to any runner.

Why it happens

Interactive development encourages out-of-order execution, and notebooks accumulate hidden state: a variable defined in a cell that was later deleted, or defined by running cells in a custom order. A human's kernel holds that history invisibly. An agent executing top-to-bottom on a fresh kernel sees the notebook as written, which differs from the notebook as developed, and the outputs expose the gap.

Edge cases / pitfalls

• Reusing kernels across agent runs for speed reintroduces exactly this bug. Fresh kernels are non-negotiable.
• Execution counts in the output are the audit trail. Non-sequential counts mean the run was not top-to-bottom.
• Some notebooks depend on external state (files, databases) that changes between runs. Diff with that in mind.
• Fix the notebook, not just the agent. The next human will hit the same staleness.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstt4QoVxmqcuplHdGm3cyVw