docs agent truncated api examples mid-json, shipped invalid code blocks

TL;DR

Detect truncated examples by parsing every JSON code block; raise the agent's output limit for example generation or split large examples. Add a validation step that fails the run on any unparseable code block.

The exact error

api examples truncated mid-json in docs

Steps

1. Find the broken blocks Extract every JSON code block from the generated docs and parse each one; collect the failures. Success check: You have the list of truncated examples.

2. Raise the output budget Increase the agent's max output tokens for the example-generation step, or split large examples across multiple calls. Success check: Examples complete within the budget.

3. Regenerate the broken ones Regenerate each truncated example with the larger budget. Success check: Every JSON code block parses cleanly.

4. Validate every block Add a post-generation step that parses all JSON code blocks and fails the run on any failure. Success check: No invalid code block can ship again.

When to use

• Use when generated docs contain cut-off JSON examples.
• Use when an agent writes API examples near token limits.

When not to use

• Not for examples that are wrong but complete (accuracy problem).
• Not for non-JSON code blocks (different validation).

Why it happens

Language models stop mid-structure when they hit output limits, and JSON is unforgiving: one missing brace invalidates the whole block. An agent generating long examples will truncate them silently unless the budget accounts for the full example.

Edge cases

Parsing every code block is cheap and catches truncation deterministically, unlike re-reading. Splitting large examples (request and response separately) also keeps each generation well under the limit.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstdcEdmtx0jmkSe86Yw-eLQ