Skip to content
← Back to activity

Immutable solution version

Bug(rust): Merge insert doesn't work for table with index

Version 1Sign in to save

Complete solution

Merge insert failing on a LanceDB table that has an index: an IVFPQ index cannot be built without a minimum number of rows, and creating the index at table-creation time breaks merge insert. Reporter-verified workaround: do not create the index when creating the table.

LanceDB - Cargo.toml: `` [package] name = "lancedb-bug" version = "0.1.0" edition = "2024" [dependencies] anyhow = "1.0.97" arrow-array = "54.1" half = "2.4.1" lancedb = "0.18.2" rand = "0.9.0" tokio = "1.44.1" src/main.rs: `` use std::sync::Arc; use arrowarray::{ FixedSizeListArray, RecordBatch, RecordBatchIterator, builder::StringBuilder, types::Float16Type, }; use lancedb::{ arrow::arrowschema::{DataType, Field, Schema}, index::Index, }; use rand::Rng as ; const VECDIMS: usize = 64; const TBL_NAME: &str = "events";

The fix

Merge insert failing on a LanceDB table that has an index: an IVFPQ index cannot be built without a minimum number of rows, and creating the index at table-creation time breaks merge insert. Reporter-verified workaround: do not create the index when creating the table. Create the table first, load data, then create the index once the table has enough rows.

Thread: gh:lancedb/lancedb#2285

Evidence

In gh:lancedb/lancedb#2285 (https://github.com/lancedb/lancedb/issues/2285), the issue reported: Cargo.toml: `` [package] name = "lancedb-bug" version = "0.1.0" edition = "2024" [dependencies] anyhow = "1.0.97" arrow-array = "54.1" half = "2.4.1" lancedb = "0.18.2" rand = "0.9.0" tokio = "1.44.1" src/main.rs: `` use std::sync::Arc; use arrowarray::{ FixedSizeListArray, RecordBatch, RecordBatchIterator, builder::StringBuilder, types::Float16Type, }; use lancedb::{ arrow::arrowschema::{DataType, Field, Schema}, index::Index, }; use rand::Rng as ; const VECDIMS: usize = 64; const TBL_NAME: &str = "events"; The verified answer: Merge insert failing on a LanceDB table that has an index: an IVFPQ index cannot be built without a minimum number of rows, and creating the index at table-creation time breaks merge insert. Reporter-verified workaround: do not create the index when creating the table. Create the table first, load data, then create the index once the table has enough rows.

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_4wrZcaf0Hakk8hAwF0YdtQ" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Bug(rust): Merge insert doesn&#39;t work for table with index <span>— Vectle</span></a>