docs agent failed: truncated long module before generating reference

TL;DR

Never feed a whole long module to the model in one shot. Split it into per-symbol chunks, generate docs per chunk, then stitch. The agent truncated the file to fit the context window and silently dropped everything past the cutoff, so the reference looked complete but was missing the second half of the module. Fix: extract symbols first with a real parser, document each symbol separately, and verify coverage against the symbol list.

The error

docs agent failed: truncated long module before generating reference

Steps

1. Parse the module with a real AST parser and list every top-level class, function, and constant with its line range. Expected: a complete symbol list with counts you can check coverage against.

2. Split the source into one chunk per symbol (or small groups of related symbols), each well under the model's input budget, keeping imports and module docstring in every chunk for context. Expected: chunks that each fit comfortably, with no chunk near the limit.

3. Generate reference docs for each chunk independently. Expected: one doc section per symbol.

4. Stitch the sections in source order under the module page. Expected: the page order matches the file order.

5. Diff the documented symbols against the step-1 symbol list. Expected: zero missing symbols. Any gap means a chunk failed and must be retried.

Use this when

• generated reference docs are missing classes or functions from the bottom of a file
• the agent's output ends mid-module with no error
• docs cover small modules fully but large ones partially

Not for this skill when

• the module is fully documented but wrong (a quality issue, not truncation)
• the parser cannot read the file at all (a syntax or parser issue)
• you need to shrink context for a chat answer rather than doc generation

Variant phrasings

• docs agent dropped trailing classes in long file
• reference docs incomplete for large module
• context window cut off module during doc generation
• agent documented only the first half of the file

Why it happens

The agent sends the whole file as model input, the file exceeds the context window, and the input gets truncated at the byte or token limit. Because truncation is silent, the pipeline reports success on a partial result and nobody notices the missing symbols until a reader looks for them.

Edge cases

• Symbols that reference each other across chunks: pass a short symbol index (names plus one-line summaries) into each chunk so cross-links resolve.
• Generated files with thousands of symbols: raise the per-chunk budget instead of shrinking chunks, or document at the module level with links to per-symbol pages.
• A chunk that still fails: retry that chunk alone rather than re-running the whole module.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_8vyM83F7Qy7cURFWRVCgSg