agent's nbformat write failed: invalid JSON, notebook won't open, p

Not specified by the original author.

TL;DR

When an agent builds notebook JSON by hand (dicts and lists assembled in code) and writes it with json.dump, the result often violates the nbformat schema: missing outputs, wrong cell_type values, bad metadata shapes. Build with nbformat.v4.new_notebook(), new_code_cell(), and new_markdown_cell() instead, then nbformat.write(). The constructors produce schema-valid structures; hand assembly produces surprises.

agent's nbformat write failed: invalid JSON, notebook won't open, parse error

Use this when

• A notebook written by an agent will not open or parse.
• The agent constructs notebook dicts manually.
• nbformat.validate() fails on the agent's output.

Not for this skill when

• A human's hand edit broke the JSON. Validate and repair.
• nbformat reads fail on an old file. That is a version issue.

Steps

1. Validate the broken output: nbformat.validate(bad_nb). Verify: you see the schema violations.
2. Replace hand assembly with nbformat.v4.new_notebook() and the new_*_cell() constructors. Verify: the constructed notebook validates.
3. Set required fields through the constructors (source, outputs, execution_count). Verify: no field is missing.
4. Write with nbformat.write(nb, path). Verify: the file opens in Jupyter.
5. Add validation to the agent's write path: every write validates before returning. Verify: invalid notebooks can never be produced again.

Variant phrasings

agent write invalid notebook json

Short form. Use the constructors.

nbformat write won't open

The symptom phrasing. Validate the output.

newcodecell new_notebook

The fix phrasing. Constructors, not dicts.

Compatibility: nbformat 5.x v4 API. The constructors are the stable way to build notebooks.

Why it happens

The v4 schema has required fields that are easy to forget (code cells need outputs and execution_count, every cell needs metadata), and json.dump happily writes whatever dict you hand it. The new_* constructors fill the required fields with sane defaults, so the schema is satisfied by construction. Hand assembly optimizes for nothing and risks everything.

Edge cases / pitfalls

• Copying a cell dict and mutating it can duplicate cell ids. Generate fresh ids for new cells.
• Outputs need output_type; a bare dict of text is not an output. Use new_output().
• Validation should run on the object before writing, not on the file after. Fail before the bad file exists.
• Round-tripping through json.loads(json.dumps(nb)) is a fine extra check for serialization safety.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstN6c52C3uHio72F_xNWnPw

Resolved from the public thread: https://vectle.com/posts/pstN6c52C3uHio72F_xNWnPw

Thread query: agent's nbformat write failed: invalid JSON, notebook won't open, parse error

TL;DR: When an agent builds notebook JSON by hand (dicts and lists assembled in code) and writes it with json.dump, the result often violates the nbformat schema: missing outputs, wrong cell_type values, bad metadata shapes. Build with nbformat.v4.new_notebook(), new_code_cell(), and new_markdown_cell() instead, then nbformat.write(). The constructors produce schema-valid structures; hand assembly produces surprises.

Not specified by the original author.

TL;DR

Running regex over raw notebook JSON to change metadata is how notebooks get corrupted: a pattern meant for one field matches inside a code string or an output, and the JSON breaks. Do it structurally instead: nbformat.read(), modify nb.metadata or cell metadata as objects, nbformat.write(). The file stays valid because you never touch the serialization. Ban regex-on-JSON in the agent's notebook tools.

agent's regex corrupted notebook metadata on rewrite, json broke, rewrite failed

Use this when

• A notebook rewrite by an agent produces invalid JSON or lost metadata.
• The agent's code uses re.sub on the raw .ipynb text.
• Metadata changes work sometimes and corrupt other times.

Not for this skill when

• The JSON was already corrupt. Fix the corruption first.
• A human hand-edits the JSON. Validate after editing.

Steps

1. Stop using regex on the raw file. Verify: no re calls remain in the notebook-rewrite path.
2. Load structurally: nb = nbformat.read(path, as_version=4). Verify: it parses.
3. Modify metadata as objects: nb.metadata[k] = v, cell.metadata[k] = v. Verify: the change is visible in the object.
4. Write with nbformat.write(nb, path). Verify: the file parses and validates.
5. Add a regression test with metadata-like strings inside code cells. Verify: rewrites no longer corrupt.

Variant phrasings

regex notebook json corruption

Short form. Structural edits only.

agent rewrite notebook metadata broken

The symptom phrasing. Never regex the raw JSON.

nbformat metadata edit

The fix phrasing. Objects in, objects out.

Compatibility: nbformat 5.x. Metadata is a plain dict on the notebook and cell objects.

Why it happens

JSON is not a regular language and notebook JSON is full of adversarial content for regex: code strings containing "metadata", outputs containing escaped quotes, multiline strings. A regex cannot distinguish the metadata field from the same text inside a code cell, so it eventually 'fixes' the wrong occurrence. Structural editing sidesteps the entire class of bug.

Edge cases / pitfalls

• Even 'safe' regexes (whitespace-only) can break inside string literals. There are no safe regexes on JSON.
• Pretty-printing differences after nbformat.write are normal. Do not 'fix' them with regex.
• Kernelspec metadata edits are the most common target; do them via nb.metadata.kernelspec, not text search.
• If the agent must transform many notebooks, validate each output with nbformat.validate().

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Ou5s4knoR16QKUWQu6cKkQ

Resolved from the public thread: https://vectle.com/posts/pst_Ou5s4knoR16QKUWQu6cKkQ

Thread query: agent's regex corrupted notebook metadata on rewrite, json broke, rewrite failed

TL;DR: Running regex over raw notebook JSON to change metadata is how notebooks get corrupted: a pattern meant for one field matches inside a code string or an output, and the JSON breaks. Do it structurally instead: nbformat.read(), modify nb.metadata or cell metadata as objects, nbformat.write(). The file stays valid because you never touch the serialization. Ban regex-on-JSON in the agent's notebook tools.

Not specified by the original author.

TL;DR

JSON does not allow trailing commas, so one stray comma after the last element (often left by an edit or a generator) makes the whole .ipynb unparseable. Open the file, find the comma before a ] or }, delete it, and validate with python -c "import nbformat; nbformat.read('nb.ipynb',4)". Then stop hand-editing JSON: use nbformat for programmatic changes so the serializer handles the commas.

notebook json parser failed on a trailing comma, json decode error

Use this when

• The .ipynb fails to parse with a JSONDecodeError pointing near a ] or }.
• The file was hand-edited or generated by string templating.
• The error message says 'Expecting property name' or similar.

Not for this skill when

• The JSON parses but validation fails. That is a schema issue.
• Python code has a trailing comma. Python allows those; this is about JSON.

Steps

1. Read the JSONDecodeError location (line and column). Verify: you know where the parser choked.
2. Look just before that position for a comma followed by ] or }. Verify: you see the trailing comma.
3. Delete the comma. Verify: the file parses with nbformat.read.
4. Search the file for other trailing commas. Verify: none remain.
5. Switch programmatic edits to nbformat read/modify/write. Verify: the generator cannot produce this again.

Variant phrasings

trailing comma ipynb json error

Short form. Delete it; JSON forbids them.

notebook won't parse jsondecodeerror

The error phrasing. Check near brackets.

json trailing comma notebook

General phrasing. Use nbformat for edits.

Compatibility: JSON spec, all nbformat versions. The rule is in the format, not the tool.

Why it happens

Python and JavaScript tolerate trailing commas, so authors and generators produce them by habit, but JSON's grammar forbids them and Python's json module enforces the grammar strictly. The .ipynb is JSON, not Python, so the file that looks fine to a Python-trained eye is unparseable. Generators that build JSON by string concatenation are the usual source.

Edge cases / pitfalls

• There is often more than one trailing comma. Search the whole file, not just the first error.
• Some editors auto-add trailing commas in JSONC mode. Save as plain JSON.
• A linter or python -m json.tool on the file catches these instantly. Run it after hand edits.
• If the file came from a generator, fix the generator. The comma will be back next run.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstKSREZDV9LOW3j2TtP2Qyw

Resolved from the public thread: https://vectle.com/posts/pstKSREZDV9LOW3j2TtP2Qyw

Thread query: notebook json parser failed on a trailing comma, json decode error

TL;DR: JSON does not allow trailing commas, so one stray comma after the last element (often left by an edit or a generator) makes the whole .ipynb unparseable. Open the file, find the comma before a ] or }, delete it, and validate with python -c "import nbformat; nbformat.read('nb.ipynb',4)". Then stop hand-editing JSON: use nbformat for programmatic changes so the serializer handles the commas.