Supabase RAG pipeline: chunk, embed, store in pgvector, retrieve with a match function

RAG on Supabase: the four-stage pipeline agents get wrong

Retrieval quality is decided by the pipeline, not the model alone. Agents skip chunking, mix embedding models, or retrieve without a threshold and then blame the LLM for bad answers.

Checkable procedure

1. Chunk documents with overlap (a few hundred tokens per chunk, ~10 percent overlap). Chunks are the retrieval unit; whole documents dilute the embedding and wreck precision.
2. Embed every chunk with a single model and record the model and dimension. One model per corpus: mixing models in one table makes distances meaningless.
3. Store in a documents table with a vector column and a match_documents function doing cosine similarity with a threshold parameter. Enable pgvector and build the HNSW index after the initial load.
4. Retrieve top-k above the similarity threshold, then stuff the chunks into the prompt with source references. No chunk above threshold means "I do not know", not a low-threshold guess.
5. Version the pipeline: when the chunking or model changes, re-embed the corpus. Half the table on the old scheme silently degrades retrieval.

Ordering constraints

Extension and schema first, chunking strategy second, embedding third, index after the bulk load, match function last. Test retrieval quality before wiring the LLM; bad retrieval cannot be fixed with a better prompt.

Verification

Ask ten questions with known answers and score how often the right chunk is in the top-k. Below your bar, tune chunk size and threshold before touching anything else.

Verified against live Supabase docs 2026-09-26: https://supabase.com/docs/guides/ai