Immutable solution version
Crash with many objects with geoCoordinates props: "panic: runtime error: makeslice: len out of range"
Complete solution
If Weaviate crashes with 'panic: runtime error: makeslice: len out of range' while importing objects with geoCoordinates props, upgrade to v1.17.2 or later. The HNSW index struggled with duplicate geo coordinates, which made a race condition much more likely.
What was reported: Issue weaviate/weaviate#2539 (closed, 17 comments): I have had an issue where the weaviate server crashes (and is unable to recover) when I attempt an import of a postcode file that is publicly accessible at https://www.getthedata.com/open-postcode-geo
On my system, their error occurs after around 88,000 records have been processed, but this varies. I have had it work fine once (when my docker volume wasn't mapped) and it has failed twice (when my volume was mapped).
I attach the Jupyter notebook which contains the schema definition and file import...
What works: If Weaviate crashes with 'panic: runtime error: makeslice: len out of range' while importing objects with geoCoordinates props, upgrade to v1.17.2 or later. The HNSW index struggled with duplicate geo coordinates, which made a race condition much more likely. As a stopgap, slowing the import loop (a short sleep every few thousand records) helped.
Evidence
In the GitHub issue, the reporter asked: Crash with many objects with geoCoordinates props: "panic: runtime error: makeslice: len out of range". The working answer that came out of the thread: If Weaviate crashes with 'panic: runtime error: makeslice: len out of range' while importing objects with geoCoordinates props, upgrade to v1.17.2 or later. The HNSW index struggled with duplicate geo coordinates, which made a race condition much more likely. As a stopgap, slowing the import loop (a short sleep every few thousand records) helped. Source: https://github.com/weaviate/weaviate/issues/2539
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_Lk8OuR4_HRemovQFAwEMHQ" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Crash with many objects with geoCoordinates props: "panic: runtime error: makeslice: len out of range" <span>— Vectle</span></a>