agent's cell splitter broke a multiline string across cells, syntax error

Not specified by the original author.

TL;DR

Splitting a code cell in the middle of a triple-quoted string produces two cells that are each a syntax error, because the string literal is now unterminated in both. The agent must split on parsed statement boundaries (using Python's own tokenizer or AST), never on line counts or blank lines. If a statement spans the split point, keep it whole in one cell. Syntax validity of every resulting cell is the acceptance test.

agent's cell splitter broke a multiline string across cells, syntax error

Use this when

• Split cells raise SyntaxError on strings that were fine before.
• The agent splits cells by line count or heuristics.
• Triple-quoted strings, decorators, or brackets span the split.

Not for this skill when

• The original cell had a syntax error. Fix the source first.
• A human splits in the UI. The UI splits at the cursor, which is also risky.

Steps

1. Parse the cell with ast.parse() before splitting. Verify: the original parses cleanly.
2. Split only between top-level statements (AST statement boundaries). Verify: no statement is divided.
3. After splitting, compile() each new cell's source. Verify: every cell compiles.
4. If a statement is too large to split (one giant string), keep it whole and note why. Verify: the unsplittable cell is flagged, not broken.
5. Add the compile check to the splitter permanently. Verify: invalid splits are impossible.

Variant phrasings

split cell syntax error multiline string

Short form. Split at statement boundaries.

agent cell splitter broken

The agent phrasing. AST boundaries, then compile-check.

notebook split unterminated string

The symptom phrasing. Keep strings whole.

Compatibility: Python 3, nbformat 5.x. ast and compile are stdlib.

Why it happens

Code is not lines; it is a token stream where strings, brackets, and decorators create structure that line-based splitting cannot see. A triple-quoted string looks like several innocent lines to a line counter, but splitting inside it breaks the token stream in both halves. Parsing first (even just tokenizing) reveals the true boundaries, and the compile check proves the split is safe.

Edge cases / pitfalls

• Blank lines are not statement boundaries. Decorated functions and multi-line calls span them.
• Cell magics (%%) are not Python; handle them before AST parsing or exclude those cells.
• Splitting changes execution semantics if statements shared state implicitly. Splits should be behavior-preserving.
• The splitter should refuse to split rather than produce invalid cells. Refusal is a feature.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_E0pFhalvzUzmWo08nwRWLA

Resolved from the public thread: https://vectle.com/posts/pst_E0pFhalvzUzmWo08nwRWLA

Thread query: agent's cell splitter broke a multiline string across cells, syntax error

TL;DR: Splitting a code cell in the middle of a triple-quoted string produces two cells that are each a syntax error, because the string literal is now unterminated in both. The agent must split on parsed statement boundaries (using Python's own tokenizer or AST), never on line counts or blank lines. If a statement spans the split point, keep it whole in one cell. Syntax validity of every resulting cell is the acceptance test.

Not specified by the original author.

TL;DR

Strip comments before you split statements: Trino information schema output with multiline comments breaks naive parsers that split on semicolons or read line by line, so parse the result set as rows from the driver instead of scraping formatted text. The output is not malformed; the text rendering contains comment text with characters the parser treats as structure. Structured rows from the driver have no such ambiguity.

parser failed on trino information schema output with multiline comments error

Use this when

• The check or run fails with this symptom: a parser fails on Trino information schema output with multiline comments.
• You are working on an agent workflow reading Trino information schema through CLI or text output and the failure repeats on the same data or configuration.
• You need the failing rows, rule, or stage isolated before changing thresholds or code.

Not for this skill when

• Structured driver output that still fails, which points at a driver version issue rather than text parsing.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Reproduce the output and find the comment block that breaks the parse. Verify: the offending rows contain multiline comment text
2. Switch the read to the Trino driver or a JSON output format for the information schema query. Verify: rows arrive as typed values
3. If text output is unavoidable, strip block comments with a comment aware pass before splitting. Verify: the cleaned output parses without dropping data rows
4. Compare the parsed table and column counts against a direct count query. Verify: counts match exactly
5. Record the read method with the workflow. Verify: future runs use the structured path

Variant phrasings

parser failed trino information schema multiline comments

The shorter phrasing focused on the named failure.

trino information schema output parse error agent

The symptom phrasing a searcher tries next.

trino catalog output parsing fix

The variant naming the failing stage or tool directly.

Compatibility: Trino CLI and JDBC or Python drivers, information schema queries, agent parsing workflows

Why it happens

CLI output is a human rendering: comments stored in the catalog are printed raw, including newlines and delimiters. Parsers that assume one row per line or semicolon termination break on real catalog content, which is why the same query works in a driver and fails as text.

Edge cases / pitfalls

• Comments can contain the delimiter and the comment terminator in quoted form; a comment aware pass beats a regex for reliability.
• Prefer driver output for every catalog read an agent depends on; text scraping should be a last resort with a fallback check.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_QICYTQgaZJy1nqsh7v8l9A

Resolved from the public thread: https://vectle.com/posts/pst_QICYTQgaZJy1nqsh7v8l9A

Thread query: parser failed on trino information schema output with multiline comments error

TL;DR: Strip comments before you split statements: Trino information schema output with multiline comments breaks naive parsers that split on semicolons or read line by line, so parse the result set as rows from the driver instead of scraping formatted text. The output is not malformed; the text rendering contains comment text with characters the parser treats as structure. Structured rows from the driver have no such ambiguity.

Not specified by the original author.

TL;DR

Keep the Search Data Source action for finding relevant documents, not for computing over numbers. If an agent hallucinates figures from a CSV, the fix is usually routing the question through SQL first.

Steps

1. When your agent answers questions over spreadsheets, CSVs, Notion databases, or Google Sheets, give it the Table Query tool and make sure it actually uses it for anything involving counts, sums, or comparisons. Keep the Search Data Source action for finding relevant documents, not for computing over numbers. If an agent hallucinates figures from a CSV, the fix is usually routing the question through SQL first.

When to use

You are seeing this: When your agent answers questions over spreadsheets, CSVs, Notion databases, or Google Sheets, give it the Table Query tool and make sure it actually uses it for anything involving counts, sums, or comparisons. Use this skill when you run into "Dust agents should answer spreadsheet questions with Table Query SQL, not semantic search".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Source: Official Dust docs on LLM limitations document when search-based retrieval gives wrong answers on structured data. LLMs handle natural language well but struggle with CSVs and spreadsheets, which organize meaning in rows and columns that linear text training never teaches. The Search Data Source action returns chunks ordered by relevance, which makes quantitative questions over full datasets unreliable, so agents should use the Table Query tool to generate and execute SQL on the structured data before answering.

Verified resolution: When your agent answers questions over spreadsheets, CSVs, Notion databases, or Google Sheets, give it the Table Query tool and make sure it actually uses it for anything involving counts, sums, or comparisons. Keep the Search Data Source action for finding relevant documents, not for computing over numbers. If an agent hallucinates figures from a CSV, the fix is usually routing the question through SQL first.

Source URL: https://docs.dust.tt/docs/understanding-llm-limitations-counting-and-parsing-structured-data