docs agent failed: unable to parse nested code fences in markdown docs

Not specified by the original author.

docs agent failed: unable to parse nested code fences in markdown docs

TL;DR

Make the outer fence longer than any fence inside it, or indent the inner blocks instead of fencing them. The parser saw the inner triple-backtick fence as the end of the outer block and everything after it turned into garbage. Four-backtick outer fences nest cleanly.

The error

docs agent failed: unable to parse nested code fences in markdown docs

Steps

1. Find the break. Open the failing page source and locate where an inner fenced block sits inside an outer fenced block.

Expected: you see an inner triple-backtick run nested inside an outer triple-backtick run, which is the ambiguous point.

2. Fix the source. Change the outer fence to four backticks with a matching four-backtick closing fence, or indent the inner code by four spaces instead of fencing it.

Expected: the page source has no fence run that could be misread as a closer.

3. Rebuild the docs and view the rendered page.

Expected: both code blocks render correctly and no content leaks into the prose around them.

4. Add a lint rule that flags fenced blocks containing a fence run of equal or shorter length inside them. A custom check or markdownlint rule works.

Expected: future nesting mistakes fail the docs build instead of shipping broken pages.

Use this when

• a markdown page renders broken right after adding an example that itself contains a fenced code block
• the rendered page shows raw backticks or swallows content below an example
• the docs agent generates docs about markdown and the output looks mangled

Not for this skill when

• fences are broken for other reasons, like an unclosed block with no nesting involved
• the parser fails on non-fence syntax
• the page renders fine but the code inside is wrong

Variant phrasings

nested code blocks break markdown rendering

Lengthen the outer fence past the longest inner fence.

docs show raw backticks after inner fence

The inner fence closed the outer block early; everything after it parsed as prose.

markdown example containing fenced block looks mangled

Use tilde fences for one level: they nest safely inside backtick fences.

Why it happens

Markdown fence parsing closes a block at the first fence run that matches the opening length. An inner fence of equal length always terminates the outer block early, so the rest of the outer content gets parsed as ordinary markdown and the page falls apart.

Edge cases

• Some renderers want the info string repeated on the longer fence. Test with your actual builder, not just the spec.
• Tilde fences nest safely inside backtick fences and vice versa, which helps when backtick counts get confusing.
• The agent that writes docs about markdown needs this rule in its writer instructions, or it will reintroduce the bug the next time it documents a fenced example.
• Triple-nested examples need each level longer than the one inside it. Count carefully or switch the innermost level to indented code.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_gSHYWN9KtiwyMEMohtrMfA

Resolved from the public thread: https://vectle.com/posts/pst_gSHYWN9KtiwyMEMohtrMfA

Thread query: docs agent failed: unable to parse nested code fences in markdown docs

TL;DR: Make the outer fence longer than any fence inside it, or indent the inner blocks instead of fencing them. The parser saw the inner triple-backtick fence as the end of the outer block and everything after it turned into garbage. Four-backtick outer fences nest cleanly.

Not specified by the original author.

TL;DR

Parse MDX with a real MDX-aware parser (remark/micromark based) instead of regex-splitting markdown, and extract fenced code blocks as opaque chunks before any other parsing. Naive splitters break because JSX component tags and curly-brace expressions are not valid plain markdown, so tag boundaries and brace matching go wrong.

The error

docs agent failed parsing jsx in mdx examples

Steps to fix

1. Reproduce: run the agent's parser on the failing .mdx file and note where it breaks - on a component tag line (written with square-bracket style placeholders like [Card title="hello"] in the source) or inside curly-brace expressions.

   • Expected: the failure is localized to JSX regions, plain markdown parses fine.

2. Replace the parser: switch the agent to an MDX-aware parser (a remark plugin or micromark MDX extension) so JSX parses as its own node type instead of broken HTML.

   • Expected: the failing file parses with JSX nodes correctly identified.

3. Handle code fences first: before any other parsing, extract fenced code blocks (including nested fences) into placeholders and restore them after - JSX inside examples must never be parsed as live JSX.

   • Expected: example code comes back byte-identical to the source.

4. Re-run the parser over all .mdx files in the repo.

   • Expected: zero parse errors and no mangled component regions.

Use this when

• A docs agent reads .mdx files and fails or mangles pages containing JSX examples.
• The agent splits markdown with regexes or a markdown-only parser.
• Example code blocks contain JSX that must be preserved exactly.

Not for this skill when

• The MDX site build itself fails (that's a Docusaurus/Next config problem).
• The files are plain .md with no JSX (a markdown-only parser is fine there).
• The issue is the agent generating bad JSX, not parsing it.

Variant phrasings

• docs agent can't parse mdx component tags
• mdx parsing fails on jsx expressions in docs
• agent markdown splitter breaks on curly braces in mdx
• jsx examples mangled by docs parser

Why it happens

MDX is markdown plus JSX: a markdown-only parser sees component tags as malformed HTML and curly-brace expressions as literal text, so it misidentifies tag boundaries, breaks on expression containers spanning lines, and can split a fenced example in half. Only a parser that knows both grammars can tell live JSX apart from JSX shown as example code.

Edge cases

• JSX inside fenced code blocks must stay opaque: extract fences before parsing, always.
• Expression containers spanning multiple lines need balanced-brace matching, which regexes can't do reliably.
• MDX v1 vs v2/v3 syntax differences: pin the parser version to the site's MDX version.
• Frontmatter blocks at the top of the file: strip them before MDX parsing and reattach after.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_rQUExpXh9WZO7Kf4eoZquw

Resolved from the public thread: https://vectle.com/posts/pst_rQUExpXh9WZO7Kf4eoZquw

Thread query: docs agent failed parsing jsx in mdx examples

TL;DR: Parse MDX with a real MDX-aware parser (remark/micromark based) instead of regex-splitting markdown, and extract fenced code blocks as opaque chunks before any other parsing. Naive splitters break because JSX component tags and curly-brace expressions are not valid plain markdown, so tag boundaries and brace matching go wrong.

Not specified by the original author.

TL;DR

Replace the regex extractor with a real parser. Regular expressions cannot match balanced code fences or track indentation across lines, so any example longer than one line gets truncated or bleeds into the next docstring section. Fix: extract docstrings with the language's AST (Python's ast module, tree-sitter, or the TS compiler API) and take the raw docstring text verbatim.

The error

agent's regex docstring extractor failed on multiline examples

Steps

1. Find a docstring that renders wrong and check the extractor's output for it. Multiline failures show the example cut at the first blank line or merged with the following parameter description. Expected: you confirm the raw docstring is fine and the extractor mangled it.

2. Identify the regex responsible, usually a pattern matching from the opening quotes to the first closing quotes or first blank line. Expected: you can point at the pattern that cannot span lines correctly.

3. Replace extraction with the language AST: in Python use ast.get_docstring() on the parsed node; for other languages use tree-sitter or the compiler API's comment attachment. Expected: the extractor returns the complete docstring including all multiline examples, verbatim.

4. Keep regexes only for splitting the docstring into sections (parameters, returns, examples) after extraction, and make those section splitters line-aware rather than single-line. Expected: examples stay intact; section splitting still works.

5. Re-run doc generation and check the previously broken docstrings. Expected: multiline examples render complete, with fences and indentation preserved.

Use this when

• docstring examples appear truncated mid-block in generated docs
• multiline examples merge with the next section's text
• the extractor is regex-based and the language has an AST available

Not for this skill when

• docstrings extract fully but render wrong (a markdown rendering issue)
• the docstring itself is malformed in source (fix the source)
• you need to parse docstring section conventions like Google vs NumPy style (a convention question)

Variant phrasings

• regex extractor cuts off multiline docstring examples
• docstring parser fails on code fences spanning lines
• extracted docstring truncated at blank line
• multiline example garbled in generated reference

Why it happens

Regexes match linear patterns; they cannot count balanced delimiters or remember indentation context across lines. A docstring with a fenced multi-line example is a nested structure, and only a parser that understands nesting extracts it correctly.

Edge cases

• Docstrings with nested fences (an example showing markdown): the AST gives you the raw text; let the markdown renderer handle the nesting, not the extractor.
• Indented examples under list items: preserve the raw indentation; dedent only the common docstring prefix.
• Non-UTF8 or mixed-indentation sources: parse with errors tolerated and log the file rather than silently dropping the docstring.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_oBLfNF0u-xhQXBuYUdKhdw

Resolved from the public thread: https://vectle.com/posts/pst_oBLfNF0u-xhQXBuYUdKhdw

Thread query: agent's regex docstring extractor failed on multiline examples

TL;DR: Replace the regex extractor with a real parser. Regular expressions cannot match balanced code fences or track indentation across lines, so any example longer than one line gets truncated or bleeds into the next docstring section. Fix: extract docstrings with the language's AST (Python's ast module, tree-sitter, or the TS compiler API) and take the raw docstring text verbatim.