docs agent failed to regenerate only changed pages, rebuilt full site every run
Fixes docs agents that rebuild the entire site on every run instead of incrementally. Use when every agent run triggers a full rebuild. Not for build failures.
TL;DR
Stop deleting the build cache between runs and let the tool's incremental build do its job. Full rebuilds on every run are a cache problem, not a tool limitation.
Error
(agent rebuilt all 4,000 pages on a one-line change; no error text, just wasted time)Steps
- Find where the agent wipes state: look for cache or output directory deletions at the start of its run (doctrees, .docusaurus cache, mkdocs site dir handling). Expected: the deletion step identified.
- Remove the deletion. Sphinx reuses doctrees by default; Docusaurus reuses its cache; MkDocs rebuilds are already fast enough that wiping gains nothing. Expected: the cache survives between runs.
- Verify incrementality: change one page and rebuild, then check that only affected pages were re-emitted (Sphinx prints "reading sources... [ 0%] ..."; Docusaurus logs cached modules). Expected: a one-page change produces a fast build.
- If the agent must run in a fresh container each time, persist the cache directory as a CI artifact instead of rebuilding from scratch. Expected: cache restored at run start.
When to use
- Every docs run rebuilds the full site
- The agent deletes build caches "to be safe"
- Build times grow with the site on every run
When not to use
- The build is slow for other reasons (profile first)
- You need a clean build to verify correctness (do it on a schedule, not every run)
- Incremental output is suspected stale (then investigate the staleness, not the speed)
Tool compatibility
Sphinx, Docusaurus 2.x/3.x, MkDocs. All three support incremental builds when their caches persist.
Variant phrasings
agent runs mkdocs build --clean every time
Drop --clean from the routine run; reserve it for release builds.
full rebuilds in ephemeral CI
Cache the tool's cache directory between CI runs; the speedup is usually 5-10x.
Why it happens
Agents are trained to prefer clean state, so they delete caches defensively. Doc tools are designed around persistent caches, so the "safe" deletion converts every run into a full rebuild.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_lnprYIvordOrLdWlt8XGJQ