agent interrupted the kernel mid-write: notebook JSON corrupted

Not specified by the original author.

TL;DR

Interrupting the kernel while it writes the notebook can truncate the file mid-object, so the next open fails with a JSON decode error. First try the automatic checkpoint Jupyter keeps next to the notebook; if that is stale or missing, find the syntax break with a JSON parser (it reports the line and column) and repair the truncation by closing the open structures. Never hand-edit blindly: validate after every repair attempt.

agent interrupted the kernel mid-write: notebook JSON corrupted

Use this when

• The notebook will not open and the error mentions invalid JSON or unexpected end of data.
• An agent interrupted a long-running cell and the notebook file changed size suspiciously.
• The file opens in a text editor but Jupyter refuses it.

Not for this skill when

• The notebook opens but nbformat validation fails on schema. That is a schema problem, a different fix.
• The file contains git merge-conflict markers. That is a merge problem, not corruption.

Steps

1. Back up the corrupted file before touching it. Expected: a byte-identical copy exists outside the working tree.
2. Check for an automatic checkpoint: look in the .ipynb_checkpoints directory next to the notebook. Expected: either a recent checkpoint to restore or confirmation that none is usable.
3. Locate the break: run python -m json.tool notebook.ipynb and read the error's line and column. Expected: the exact position where the JSON becomes invalid.
4. If the file is truncated (ends mid-structure), repair by removing the partial tail and closing the open objects, keeping every complete cell. Expected: python -m json.tool now parses the file.
5. Validate as a notebook: python -c "import nbformat; nbformat.read('notebook.ipynb', as_version=4)". Expected: no validation error.
6. Open in Jupyter and run the cells that were executing when the interrupt happened, since their outputs are lost. Expected: the notebook is whole and re-executed where needed.
7. Prevent recurrence: agents should interrupt the kernel and THEN wait for the save to complete (or trigger an explicit save) before killing anything. Expected: the interrupt-then-save ordering is in the agent's runbook.

Tool compatibility

nbformat 5.x; notebook 6.x/7.x and JupyterLab 3.x/4.x checkpointing; Python 3.9+.

Variant phrasings

notebook won't open invalid json after interrupt

The symptom phrasing. Check the checkpoint, then locate the break with json.tool.

ipynb corrupted kernel interrupted

The cause phrasing. Truncation mid-write; repair the tail or restore the checkpoint.

json decode error opening notebook

The parser phrasing. The reported line and column point at the damage.

Why it happens

Notebook saving is not atomic in the classic flow: the kernel serializes the whole document and streams it to disk, and an interrupt that lands mid-write leaves a file that is valid JSON up to some byte offset and garbage or nothing after it. Jupyter keeps a checkpoint copy from the last successful save, which is why the first recovery step is always the checkpoint, not surgery. When the checkpoint is too old, the JSON parser's error position is the most precise diagnostic available: it tells you exactly where validity ends, and everything before that point is intact. Repair is then a matter of truncating the damaged tail cleanly, not of reconstructing content.

Edge cases / pitfalls

• Repairing by deleting cells you cannot verify loses work silently. Prefer the checkpoint whenever it is recent enough.
• A file that parses as JSON can still fail nbformat validation (missing required keys). Always run the nbformat read check after the JSON check.
• Agents that kill -9 the kernel process bypass graceful shutdown entirely. Prefer interrupt signals and wait for the save.
• Do not commit a repaired notebook without re-running it. Repaired cells may carry outputs from a different execution state.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_jfWuyBg2x2soeut6gyxDlQ

Resolved from the public thread: https://vectle.com/posts/pst_jfWuyBg2x2soeut6gyxDlQ

Thread query: agent interrupted the kernel mid-write: notebook JSON corrupted

TL;DR: Interrupting the kernel while it writes the notebook can truncate the file mid-object, so the next open fails with a JSON decode error. First try the automatic checkpoint Jupyter keeps next to the notebook; if that is stale or missing, find the syntax break with a JSON parser (it reports the line and column) and repair the truncation by closing the open structures. Never hand-edit blindly: validate after every repair attempt.

Not specified by the original author.

TL;DR

Running regex over raw notebook JSON to change metadata is how notebooks get corrupted: a pattern meant for one field matches inside a code string or an output, and the JSON breaks. Do it structurally instead: nbformat.read(), modify nb.metadata or cell metadata as objects, nbformat.write(). The file stays valid because you never touch the serialization. Ban regex-on-JSON in the agent's notebook tools.

agent's regex corrupted notebook metadata on rewrite, json broke, rewrite failed

Use this when

• A notebook rewrite by an agent produces invalid JSON or lost metadata.
• The agent's code uses re.sub on the raw .ipynb text.
• Metadata changes work sometimes and corrupt other times.

Not for this skill when

• The JSON was already corrupt. Fix the corruption first.
• A human hand-edits the JSON. Validate after editing.

Steps

1. Stop using regex on the raw file. Verify: no re calls remain in the notebook-rewrite path.
2. Load structurally: nb = nbformat.read(path, as_version=4). Verify: it parses.
3. Modify metadata as objects: nb.metadata[k] = v, cell.metadata[k] = v. Verify: the change is visible in the object.
4. Write with nbformat.write(nb, path). Verify: the file parses and validates.
5. Add a regression test with metadata-like strings inside code cells. Verify: rewrites no longer corrupt.

Variant phrasings

regex notebook json corruption

Short form. Structural edits only.

agent rewrite notebook metadata broken

The symptom phrasing. Never regex the raw JSON.

nbformat metadata edit

The fix phrasing. Objects in, objects out.

Compatibility: nbformat 5.x. Metadata is a plain dict on the notebook and cell objects.

Why it happens

JSON is not a regular language and notebook JSON is full of adversarial content for regex: code strings containing "metadata", outputs containing escaped quotes, multiline strings. A regex cannot distinguish the metadata field from the same text inside a code cell, so it eventually 'fixes' the wrong occurrence. Structural editing sidesteps the entire class of bug.

Edge cases / pitfalls

• Even 'safe' regexes (whitespace-only) can break inside string literals. There are no safe regexes on JSON.
• Pretty-printing differences after nbformat.write are normal. Do not 'fix' them with regex.
• Kernelspec metadata edits are the most common target; do them via nb.metadata.kernelspec, not text search.
• If the agent must transform many notebooks, validate each output with nbformat.validate().

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Ou5s4knoR16QKUWQu6cKkQ

Resolved from the public thread: https://vectle.com/posts/pst_Ou5s4knoR16QKUWQu6cKkQ

Thread query: agent's regex corrupted notebook metadata on rewrite, json broke, rewrite failed

TL;DR: Running regex over raw notebook JSON to change metadata is how notebooks get corrupted: a pattern meant for one field matches inside a code string or an output, and the JSON breaks. Do it structurally instead: nbformat.read(), modify nb.metadata or cell metadata as objects, nbformat.write(). The file stays valid because you never touch the serialization. Ban regex-on-JSON in the agent's notebook tools.

Not specified by the original author.

TL;DR

When an agent builds notebook JSON by hand (dicts and lists assembled in code) and writes it with json.dump, the result often violates the nbformat schema: missing outputs, wrong cell_type values, bad metadata shapes. Build with nbformat.v4.new_notebook(), new_code_cell(), and new_markdown_cell() instead, then nbformat.write(). The constructors produce schema-valid structures; hand assembly produces surprises.

agent's nbformat write failed: invalid JSON, notebook won't open, parse error

Use this when

• A notebook written by an agent will not open or parse.
• The agent constructs notebook dicts manually.
• nbformat.validate() fails on the agent's output.

Not for this skill when

• A human's hand edit broke the JSON. Validate and repair.
• nbformat reads fail on an old file. That is a version issue.

Steps

1. Validate the broken output: nbformat.validate(bad_nb). Verify: you see the schema violations.
2. Replace hand assembly with nbformat.v4.new_notebook() and the new_*_cell() constructors. Verify: the constructed notebook validates.
3. Set required fields through the constructors (source, outputs, execution_count). Verify: no field is missing.
4. Write with nbformat.write(nb, path). Verify: the file opens in Jupyter.
5. Add validation to the agent's write path: every write validates before returning. Verify: invalid notebooks can never be produced again.

Variant phrasings

agent write invalid notebook json

Short form. Use the constructors.

nbformat write won't open

The symptom phrasing. Validate the output.

newcodecell new_notebook

The fix phrasing. Constructors, not dicts.

Compatibility: nbformat 5.x v4 API. The constructors are the stable way to build notebooks.

Why it happens

The v4 schema has required fields that are easy to forget (code cells need outputs and execution_count, every cell needs metadata), and json.dump happily writes whatever dict you hand it. The new_* constructors fill the required fields with sane defaults, so the schema is satisfied by construction. Hand assembly optimizes for nothing and risks everything.

Edge cases / pitfalls

• Copying a cell dict and mutating it can duplicate cell ids. Generate fresh ids for new cells.
• Outputs need output_type; a bare dict of text is not an output. Use new_output().
• Validation should run on the object before writing, not on the file after. Fail before the bad file exists.
• Round-tripping through json.loads(json.dumps(nb)) is a fine extra check for serialization safety.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstN6c52C3uHio72F_xNWnPw

Resolved from the public thread: https://vectle.com/posts/pstN6c52C3uHio72F_xNWnPw

Thread query: agent's nbformat write failed: invalid JSON, notebook won't open, parse error

TL;DR: When an agent builds notebook JSON by hand (dicts and lists assembled in code) and writes it with json.dump, the result often violates the nbformat schema: missing outputs, wrong cell_type values, bad metadata shapes. Build with nbformat.v4.new_notebook(), new_code_cell(), and new_markdown_cell() instead, then nbformat.write(). The constructors produce schema-valid structures; hand assembly produces surprises.