Pinecone workflow: sparse-only vector index for keyword search

# Sparse-only indexes

## When

Keyword-heavy retrieval (product codes, names, error strings, exact terms) where you want Pinecone's infrastructure but BM25-style behavior, or as the sparse sidecar to a dense system.

## Steps

1. **Create the index** with `metric="dotproduct"` and sparse vector support. Sparse vectors are high-dimensional with mostly zero values; the cap is 2048 non-zero entries per vector.
2. **Generate sparse vectors** with a sparse embedding model (e.g. pinecone-sparse-english-v0) or BM25 term weights. The critical rule: ingest-time and query-time generation must use the same tokenizer and weighting, or nothing matches.
3. **Upsert** records with `sparse_values` (indices and values arrays).
4. **Query** with a sparse query vector built the same way.
5. **Evaluate on keyword queries.** Sparse retrieval should win on exact-term queries and lose on paraphrase queries versus dense. If it loses on exact terms, the tokenization mismatches.

## Traps

1. Different tokenizers at ingest and query: the classic silent failure. Pin one pipeline and share the code between both paths.
2. Treating sparse as a drop-in for dense: paraphrases and synonyms will not match. Sparse is keyword retrieval; pair it with dense (hybrid) for general search.
3. Exceeding 2048 non-zero values: trim or re-weight; over-long sparse vectors get rejected.