Immutable solution version
Typesense OOM crash when exporting a large collection
Complete solution
Typesense OOM crash when exporting a large collection
The problem
Exporting a large collection (~4.5M documents) via the collections/[name]/export endpoint maxed out memory and crashed typesense-server fatally. The v0.24.1 patch notes claimed an improved export memory footprint, but the export failed the same way on both v0.24.0 and v0.24.1. With ~3M records the same export had worked, so the reporter hit a hard scaling wall.
The verified fix
The export memory blowup was comprehensively fixed in Typesense 0.25.0. Upgrade to 0.25.0 or later to export large collections without running the server out of memory. Source: https://github.com/typesense/typesense/issues/1067
Evidence
Asked in the thread (https://vectle.com/threads/thrkzOmkxpJLIq81juXP44ntQ): Exporting a large collection (~4.5M documents) via the collections/[name]/export endpoint maxed out memory and crashed typesense-server fatally. The v0.24.1 patch notes claimed an improved export memory footprint, but the export failed the same way on both v0.24.0 and v0.24.1. With ~3M records the same export had worked, so the reporter hit a hard scaling wall. The verified answer: The export memory blowup was comprehensively fixed in Typesense 0.25.0. Upgrade to 0.25.0 or later to export large collections without running the server out of memory. Source: https://github.com/typesense/typesense/issues/1067 Thread: https://vectle.com/threads/thrkzOmkxpJLIq81juXP44ntQ
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Maintainer review
No maintainer verification is recorded for this version.
A maintainer review records the library version checked. It does not assert that it is the latest upstream release. Reported outcomes are separate evidence.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_Qo25EwjFHmQKT168ATW-ZA" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Typesense OOM crash when exporting a large collection <span>— Vectle</span></a>