Set numCandidates to 10-20x the limit or Atlas Vector Search recall suffers

TL;DR: Start with numCandidates = limit * 20 (for example limit 10 -> numCandidates 200) and tune from there against a recall benchmark. If retrieval quality is poor, raise numCandidates before blaming the embedding model. If latency is too high, lower it and measure recall so the tradeoff is explicit.

Fix

1. Start with numCandidates = limit * 20 (for example limit 10 -> numCandidates 200) and tune from there against a recall benchmark. Expected: this specific failure stops.

2. Never set numCandidates below limit; the query will not do what you expect. Expected: this specific failure stops.

Details

If retrieval quality is poor, raise numCandidates before blaming the embedding model. If latency is too high, lower it and measure recall so the tradeoff is explicit. Context: Vector search optimization notes: numCandidates is the number of nearest neighbors considered during the search, and the practical rule is 10 to 20 times the limit for good recall. limit cannot exceed numCandidates when numCandidates is specified. Setting numCandidates equal to limit gives the fastest but least accurate search, and it is the most common cause of mysteriously poor RAG retrieval quality. Higher numCandidates improves recall at the cost of latency.

When to use

You hit exactly this: Set numCandidates to 10-20x the limit or Atlas Vector Search recall suffers.

When not to use

A different error, or the same symptom in a different tool. This page only covers the failure above.

Compatibility

the tool named in the title.

Source: Vector search optimization notes: numCandidates is the number of nearest neighbors considered during the search, and the practical rule is 10 to 20 times the limit for good recall. limit cannot exceed numCandidates when numCandidates is specified. Setting numCandidates equal to limit gives the fastest but least accurate search, and it is the most common cause of mysteriously poor RAG retrieval quality. Higher numCandidates improves recall at the cost of latency.

Verified resolution: Start with numCandidates = limit * 20 (for example limit 10 -> numCandidates 200) and tune from there against a recall benchmark. If retrieval quality is poor, raise numCandidates before blaming the embedding model. If latency is too high, lower it and measure recall so the tradeoff is explicit. Never set numCandidates below limit; the query will not do what you expect.

Source URL: https://github.com/romiluz13/hybrid-search-rag/blob/HEAD/docs/cookbook/04-vector-search-optimization.md