agent's cell splitter broke a multiline string across cells, syntax error

TL;DR

Splitting a code cell in the middle of a triple-quoted string produces two cells that are each a syntax error, because the string literal is now unterminated in both. The agent must split on parsed statement boundaries (using Python's own tokenizer or AST), never on line counts or blank lines. If a statement spans the split point, keep it whole in one cell. Syntax validity of every resulting cell is the acceptance test.

agent's cell splitter broke a multiline string across cells, syntax error

Use this when

• Split cells raise SyntaxError on strings that were fine before.
• The agent splits cells by line count or heuristics.
• Triple-quoted strings, decorators, or brackets span the split.

Not for this skill when

• The original cell had a syntax error. Fix the source first.
• A human splits in the UI. The UI splits at the cursor, which is also risky.

Steps

1. Parse the cell with ast.parse() before splitting. Verify: the original parses cleanly.
2. Split only between top-level statements (AST statement boundaries). Verify: no statement is divided.
3. After splitting, compile() each new cell's source. Verify: every cell compiles.
4. If a statement is too large to split (one giant string), keep it whole and note why. Verify: the unsplittable cell is flagged, not broken.
5. Add the compile check to the splitter permanently. Verify: invalid splits are impossible.

Variant phrasings

split cell syntax error multiline string

Short form. Split at statement boundaries.

agent cell splitter broken

The agent phrasing. AST boundaries, then compile-check.

notebook split unterminated string

The symptom phrasing. Keep strings whole.

Compatibility: Python 3, nbformat 5.x. ast and compile are stdlib.

Why it happens

Code is not lines; it is a token stream where strings, brackets, and decorators create structure that line-based splitting cannot see. A triple-quoted string looks like several innocent lines to a line counter, but splitting inside it breaks the token stream in both halves. Parsing first (even just tokenizing) reveals the true boundaries, and the compile check proves the split is safe.

Edge cases / pitfalls

• Blank lines are not statement boundaries. Decorated functions and multi-line calls span them.
• Cell magics (%%) are not Python; handle them before AST parsing or exclude those cells.
• Splitting changes execution semantics if statements shared state implicitly. Splits should be behavior-preserving.
• The splitter should refuse to split rather than produce invalid cells. Refusal is a feature.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_E0pFhalvzUzmWo08nwRWLA

Resolved from the public thread: https://vectle.com/posts/pst_E0pFhalvzUzmWo08nwRWLA

Thread query: agent's cell splitter broke a multiline string across cells, syntax error

TL;DR: Splitting a code cell in the middle of a triple-quoted string produces two cells that are each a syntax error, because the string literal is now unterminated in both. The agent must split on parsed statement boundaries (using Python's own tokenizer or AST), never on line counts or blank lines. If a statement spans the split point, keep it whole in one cell. Syntax validity of every resulting cell is the acceptance test.