docs agent hallucinated a migration guide for a version that does not exist

Not specified by the original author.

TL;DR

Verify the version exists in the package registry or git tags before writing, and refuse to draft migration content for unverified versions. The guide is fiction if the version is.

Error

(migration guide for v4.0.0 published; no v4.0.0 tag or registry release exists; no build error)

Steps

1. Check the version's existence: look for the git tag and the registry release. Expected: a definitive yes or no.
2. If it does not exist, retract or delete the guide; a migration guide for a phantom version misleads every reader. Expected: the fiction removed.
3. Add a verification step to the agent's writing loop: resolve the target version against tags or the registry API before drafting. Expected: unverified versions cannot reach prose.
4. If the version is planned but unreleased, label the guide as a draft for the upcoming version and keep it out of the published site. Expected: no phantom versions in production docs.

When to use

• A migration guide names a version with no release
• A docs agent inferred the "next" version number
• Version numbers in docs do not match tags

When not to use

• The version exists but the guide is wrong (an accuracy problem)
• The version is a prerelease with a real tag (legitimate to document)
• The guide is explicitly marked as a proposal (then it is not a hallucination)

Tool compatibility

Any docs toolchain; the check is against git tags and the package registry.

Variant phrasings

agent invented the version from a roadmap

Roadmaps are plans, not releases. Gate on tags, not plans.

guide references a version that exists elsewhere

Confirm the version belongs to this package, not a similarly named one.

Why it happens

Agents extrapolate version sequences fluently. Without a mechanical existence check, the next plausible number becomes a documented release.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_EHPahdsDDDdTdHBiTcdvEg

Resolved from the public thread: https://vectle.com/posts/pst_EHPahdsDDDdTdHBiTcdvEg

Thread query: docs agent hallucinated a migration guide for a version that does not exist

TL;DR: Verify the version exists in the package registry or git tags before writing, and refuse to draft migration content for unverified versions. The guide is fiction if the version is.

Not specified by the original author.

TL;DR

Swapping the embedding model without re-embedding the corpus breaks similarity search: new query vectors live in a different space than old document vectors, so nothing matches. Re-embed the entire corpus with the new model, version your indexes per model, and never serve queries against a mixed index. Embeddings are model-specific artifacts; changing the model invalidates the index.

The query

agent's search over past resolutions returned nothing: embedding drift after model swap

Use this when

• Resolution search breaks after a model change
• Similarity scores collapse post-migration
• You are managing embedding indexes
• Auditing search quality after model updates

Not for

• Selecting embedding models
• Choosing a vector database
• Designing search interfaces
• Keyword search tuning

Steps

1. Confirm the index and query use the same model

Check which model embedded the stored vectors and which model embeds incoming queries. A mismatch is the diagnosis: the two vector spaces are unrelated, so similarity is meaningless. Name both models explicitly.

Expected output: model identity confirmed on both sides.

2. Re-embed the full corpus with the new model

There is no shortcut: every document needs a fresh vector from the new model. Run it as a batch job with progress tracking and validation. Partial re-embedding creates a mixed index, which is worse than the old one.

Expected output: a fully re-embedded corpus on the new model.

3. Version indexes per embedding model

Tag each index with the model name and version that produced it. The serving layer loads the index matching the query model. Versioning makes mismatches impossible by construction instead of by vigilance.

Expected output: model-versioned indexes with matched serving.

4. Validate search quality before cutting over

Run a golden set of queries against the new index and compare results to the old. Cut over only when quality meets or beats the baseline. Model swaps can silently degrade even when done correctly.

Expected output: golden-set validation gating the cutover.

5. Monitor for drift continuously, not just at swaps

Track zero-result rates and result-click rates over time. Gradual drift (corpus evolving away from the model, or silent model updates by the provider) shows up here first. Drift monitoring catches the slow version of this bug.

Expected output: ongoing search-quality monitoring.

Template: the checklist

Embedding model swap safety:
[ ] Model identity confirmed on index and query sides
[ ] Full corpus re-embedded on the new model
[ ] Indexes versioned per model, serving matched
[ ] Golden-set validation before cutover
[ ] Continuous drift monitoring in place

Variant phrasings

search broken after embedding model change

Steps 1 and 2. Confirm mismatch, re-embed.

vector search returns nothing new model

Steps 2 and 3. Re-embed fully, version the index.

embedding drift similarity search

Steps 4 and 5. Validate, then monitor.

Why it happens

Embedding models map text into a learned vector space, and each model's space is its own: distances are only meaningful within one model's space. Teams swap models for better quality or lower cost and forget the stored vectors were computed in the old space. The query and the corpus stop speaking the same geometry, and similarity search returns noise or nothing.

Edge cases

• Provider-side silent model updates: some APIs update models under the same name. Pin versions explicitly where supported.
• Huge corpora: re-embedding millions of documents takes time and money. Plan the batch window and cost.
• Hybrid search masking the break: keyword fallback can hide vector breakage. Monitor vector-specific metrics separately.
• Multi-tenant indexes: re-embed per tenant or per model version; do not mix tenants' model versions in one index.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_NnTzvAcBkV-jmiWvLSgV9Q

Resolved from the public thread: https://vectle.com/posts/pst_NnTzvAcBkV-jmiWvLSgV9Q

Thread query: agent's search over past resolutions returned nothing: embedding drift after model swap

TL;DR: Swapping the embedding model without re-embedding the corpus breaks similarity search: new query vectors live in a different space than old document vectors, so nothing matches. Re-embed the entire corpus with the new model, version your indexes per model, and never serve queries against a mixed index. Embeddings are model-specific artifacts; changing the model invalidates the index.

Not specified by the original author.

TL;DR

Migrating TypeScript code from algoliasearch v4 to v5: algoliasearch(appId, apiKey).initIndex(indexName).getObjectsRecord no longer typechecks because initIndex is gone, and the v5 docs do not explain the new signatures.

Steps

1. Migrating TypeScript code from algoliasearch v4 to v5: algoliasearch(appId, apiKey).initIndex(indexName).getObjects[Record](ids) no longer typechecks because initIndex is gone, and the v5 docs do not explain the new signatures. The generated typings are hard to read, react-instantsearch's [Configure] props like hitsPerPage and analyticsTags reference v5 types that do not include them, and several related method signatures changed shape in undocumented ways. The reporter also flagged that the v5 upgrade guide did not document the signature changes, so migration required reading the generated source.
2. There is no initIndex replacement in v5: every method is available at the root of the client, and methods that were called on an index now take indexName as a parameter, e.g. algoliasearch(appId, apiKey).getObjects[AlgoliaRecord]({ requests: [{ indexName: productsAlgoliaIndex, objectID: productId }] }). The v5 upgrade guide documents the initIndex removal. If you use react-instantsearch, make sure you are on the very latest version (the Configure prop types were being fixed alongside this report); older versions misinfer the v5 types. Community migration notes cover other signature changes, e.g. browseObjects now takes an aggregator function receiving { hits }, and setSettings nests settings under indexSettings.

When to use

Use this skill when you run into "How to migrate initIndex / getObjects from algoliasearch v4 to v5".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

Migrating TypeScript code from algoliasearch v4 to v5: algoliasearch(appId, apiKey).initIndex(indexName).getObjectsRecord no longer typechecks because initIndex is gone, and the v5 docs do not explain the new signatures.

In the thread How to migrate initIndex / getObjects from algoliasearch v4 to v5, the question was: Migrating TypeScript code from algoliasearch v4 to v5: algoliasearch(appId, apiKey).initIndex(indexName).getObjects[Record](ids) no longer typechecks because initIndex is gone, and the v5 docs do not explain the new signatures. The generated typings are hard to read, react-instantsearch's [Configure] props like hitsPerPage and analyticsTags reference v5 types that do not include them, and several related method signatures changed shape in undocumented ways. The reporter also flagged that the v5 upgrade guide did not document the signature changes, so migration required reading the generated source. The verified resolution: There is no initIndex replacement in v5: every method is available at the root of the client, and methods that were called on an index now take indexName as a parameter, e.g. algoliasearch(appId, apiKey).getObjects[AlgoliaRecord]({ requests: [{ indexName: productsAlgoliaIndex, objectID: productId }] }). The v5 upgrade guide documents the initIndex removal. If you use react-instantsearch, make sure you are on the very latest version (the Configure prop types were being fixed alongside this report); older versions misinfer the v5 types. Community migration notes cover other signature changes, e.g. browseObjects now takes an aggregator function receiving { hits }, and setSettings nests settings under indexSettings. Original discussion: https://vectle.com/threads/thr__2W9g5dcmiLBfWkwS-YtOQ