docs agent hallucinated a migration guide for a version that does not exist
Fixes docs agents that write migration guides for nonexistent versions. Use when a migration guide references a version with no release. Not for inaccurate guides for real versions.
TL;DR
Verify the version exists in the package registry or git tags before writing, and refuse to draft migration content for unverified versions. The guide is fiction if the version is.
Error
(migration guide for v4.0.0 published; no v4.0.0 tag or registry release exists; no build error)Steps
- Check the version's existence: look for the git tag and the registry release. Expected: a definitive yes or no.
- If it does not exist, retract or delete the guide; a migration guide for a phantom version misleads every reader. Expected: the fiction removed.
- Add a verification step to the agent's writing loop: resolve the target version against tags or the registry API before drafting. Expected: unverified versions cannot reach prose.
- If the version is planned but unreleased, label the guide as a draft for the upcoming version and keep it out of the published site. Expected: no phantom versions in production docs.
When to use
- A migration guide names a version with no release
- A docs agent inferred the "next" version number
- Version numbers in docs do not match tags
When not to use
- The version exists but the guide is wrong (an accuracy problem)
- The version is a prerelease with a real tag (legitimate to document)
- The guide is explicitly marked as a proposal (then it is not a hallucination)
Tool compatibility
Any docs toolchain; the check is against git tags and the package registry.
Variant phrasings
agent invented the version from a roadmap
Roadmaps are plans, not releases. Gate on tags, not plans.
guide references a version that exists elsewhere
Confirm the version belongs to this package, not a similarly named one.
Why it happens
Agents extrapolate version sequences fluently. Without a mechanical existence check, the next plausible number becomes a documented release.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_EHPahdsDDDdTdHBiTcdvEg