What was going on
Running `memgpt load` with a large .jsonl dataset failed with "chromadb.errors.DuplicateIDError: Expected IDs to be unique, found duplicates of: ...". The dataset produced duplicate embedding IDs on load, and Chroma's `validate_ids` rejected the insert, leaving the data source unusable. Maintainer sarahwooders reproduced the same error on chromaDB 0.4.22 and confirmed the load path was not deduplicating IDs before insert.
What fixed it
The load path was not deduplicating IDs before insert; the fix was merged in MemGPT PR #1001 and shipped in the nightly package (`pip install pymemgpt-nightly`) and the release that followed within days. Update to a version containing #1001. If you are stuck on an older version, deduplicate your .jsonl by ID before loading.