docs agent's llm writer produced invalid rst, sphinx build failed

TL;DR

Lint every generated .rst file with docutils before building: parse it with publish_doctree set to halt on warnings, fix what it flags, then build. LLM writers produce plausible-looking RST with broken directive indentation and missing blank lines. A 2-second lint catches what a 2-minute build failure only hints at.

docs agent's llm writer produced invalid rst, sphinx build failed

Steps

1. Find the offending file: read the sphinx-build error; it names the .rst file and line number. Open that spot. Expected: you can see the malformed markup (bad indent, directive run into text, unknown directive).
2. Lint the file directly: python -c "from docutils.core import publish_doctree; publish_doctree(open('docs/api/broken.rst').read(), settings_overrides={'report_level': 2, 'halt_level': 2})". Expected: it raises on the same problem Sphinx hit, instantly, without a full build.
3. Fix the usual LLM RST mistakes: directives need a blank line before them and their content indented consistently (three spaces is the safe convention); inline markup needs matching delimiters; unknown directives usually mean the writer invented a directive name, replace it with a real one.
4. Re-lint until clean, then run the full sphinx-build. Expected: exit 0 and the page renders.
5. Add the lint as a gate in the agent loop: every generated .rst file is parsed before the build step, and a parse failure fails the run with the file and line. Expected: invalid RST never reaches sphinx-build again.

Use this when

• sphinx-build fails on agent-generated .rst files
• the error names a file the LLM writer produced
• you want a fast pre-build check for generated RST

Not for this skill when

• hand-written RST fails (same lint helps, but the fix is editing, not agent gating)
• the build fails on valid RST (extension or config problem)
• you use Markdown sources instead (lint the Markdown, different tool)

Variant phrasings

• llm generated rst does not build
• sphinx errors on ai-written rst
• invalid restructuredtext from docs agent

Why it happens

LLMs learn RST from rendered examples, not from the spec, so they reproduce the look of directives while getting the whitespace rules wrong: missing blank lines, inconsistent indentation, invented directive names. Sphinx is strict about these, so the build fails on markup that reads fine to a human.

Edge cases

• Plain docutils does not know Sphinx-specific directives: for those, treat Sphinx's own error as the final word.
• A file can parse yet render badly (wrong nesting): spot-check the rendered page for generated files.
• Mixed line endings can confuse the parser: normalize generated files to LF.
• Very long lines from the writer are legal but unreadable: wrap them for future debuggability.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Ub2VFfDXnD1lrlIBGSCtgg