Immutable solution version
Binary vectors - querying the nearest neighbors by Hamming distance with HNSW gives wrong results
Complete solution
Practical fix from a real Neon pgvector issue: Binary vectors - querying the nearest neighbors by Hamming distance with HNSW gives wrong results
[ankane (maintainer)]: Thanks, it looks like the index is working fine. Approximate indexing won't provide perfect recall. Also, you may see better recall by splitting unique hashes into a separate table, as well as increasing ef_construction even further (I'd try 512 next).
Context: GitHub issue pgvector/pgvector#972 (closed, 8 comments): Hi. So, I have pgvector 0.8.1, postgres 17.7, and a table like this with ~20kk records: ``ruby create_table :image_hashes do |t| t.column :external_id, :integer, null: false t.column :vector_binary, :bit, limit: 576 t.timestamps end add_index :image_hashes, :external_id, unique: true execute 'CREATE INDEX ON image_hashes USING hnsw (vector_binary bit_hamming_ops);' I clearly see that the needed record is in the table: SELECT (vector_binary <~> '[576 bits omitted]') AS distance FROM image_hashes WHERE external_id = 608810322 LIMIT 1; distance ---------- 0 (1 row) `` But querying the n
Evidence
GitHub issue pgvector/pgvector#972 (closed, 8 comments): Hi. So, I have pgvector 0.8.1, postgres 17.7, and a table like this with ~20kk records: ``ruby create_table :image_hashes do |t| t.column :external_id, :integer, null: false t.column :vector_binary, :bit, limit: 576 t.timestamps end add_index :image_hashes, :external_id, unique: true execute 'CREATE INDEX ON image_hashes USING hnsw (vector_binary bit_hamming_ops);' I clearly see that the needed record is in the table: SELECT (vector_binary <~> '[576 bits omitted]') AS distance FROM image_hashes WHERE external_id = 608810322 LIMIT 1; distance ---------- 0 (1 row) But querying the n Verified answer: [ankane (maintainer)]: Thanks, it looks like the index is working fine. Approximate indexing won't provide perfect recall. Also, you may see better recall by splitting unique hashes into a separate table, as well as increasing ef_construction` even further (I'd try 512 next). Source: https://github.com/pgvector/pgvector/issues/972
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_OuhDJMtg0g8iVylmH37nbQ" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Binary vectors - querying the nearest neighbors by Hamming distance with HNSW gives wrong results <span>— Vectle</span></a>