agent's nbconvert timed out on a notebook with 300 executed cells, execution failed

TL;DR

Three hundred executed cells means a giant .ipynb (outputs inline) and a long execution tail, so nbconvert times out on size and slowness together. Have the agent strip outputs before executing (--clear-output), raise the ExecutePreprocessor timeout to measured runtimes, and split the notebook if it is really three notebooks concatenated. The agent should also execute with a fresh kernel to avoid state-order bugs across 300 cells.

agent's nbconvert timed out on a notebook with 300 executed cells, execution failed

Use this when

• An agent-driven nbconvert --execute times out on a very large executed notebook.
• The .ipynb is tens of MB from saved outputs.
• The agent re-runs the same notebook repeatedly (CI, scheduled).

Not for this skill when

• A human's one-off export times out. Same fixes, but the audience differs.
• The timeout is on one slow cell. That is a per-cell problem.

Steps

1. Measure: file size and a timed trial run. Verify: you know size and runtime.
2. Strip outputs before executing: jupyter nbconvert --clear-output --inplace then execute. Verify: the file shrinks and the run speeds up.
3. Raise the timeout to the measured runtime plus headroom. Verify: the export completes.
4. Split the notebook at natural boundaries if it exceeds a few hundred cells. Verify: each piece executes independently.
5. Give the agent a pre-flight: refuse to execute notebooks over the size budget without stripping. Verify: the agent fails fast with instructions.

Variant phrasings

nbconvert timeout large notebook agent

Short form. Strip, raise, split.

agent execute 300 cell notebook timeout

The scale phrasing. Size budget first.

nbconvert huge executed notebook fails

General phrasing. Outputs are the weight.

Compatibility: nbconvert 6/7. The size/timeout tradeoff is version-independent.

Why it happens

Executed outputs bloat the JSON the exporter must parse, and 300 cells of real work take a long time to re-execute, so the agent hits both the parse cost and the execution timeout. Agents hit this more than humans because they re-execute on every run instead of incrementally, turning a notebook that opens fine into one that never finishes exporting.

Edge cases / pitfalls

• Stripping outputs before execution loses the previous run's record. Archive the executed file first.
• A 300-cell notebook usually has hidden order dependencies. Splitting requires verifying each piece runs standalone.
• The agent should log per-cell timings. Without them, the next timeout is undebuggable.
• Timeouts that scale with cell count need revisiting as the notebook grows. Budget explicitly.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_RcRzxnnWRIPTFED3tzAFqA