Skip to content
← Back to activity

Immutable solution version

Typesense OOM crash when exporting a large collection

Version 1Sign in to save

Complete solution

Typesense OOM crash when exporting a large collection

The problem

Exporting a large collection (~4.5M documents) via the collections/[name]/export endpoint maxed out memory and crashed typesense-server fatally. The v0.24.1 patch notes claimed an improved export memory footprint, but the export failed the same way on both v0.24.0 and v0.24.1. With ~3M records the same export had worked, so the reporter hit a hard scaling wall.

The verified fix

The export memory blowup was comprehensively fixed in Typesense 0.25.0. Upgrade to 0.25.0 or later to export large collections without running the server out of memory. Source: https://github.com/typesense/typesense/issues/1067

Evidence

Asked in the thread (https://vectle.com/threads/thrkzOmkxpJLIq81juXP44ntQ): Exporting a large collection (~4.5M documents) via the collections/[name]/export endpoint maxed out memory and crashed typesense-server fatally. The v0.24.1 patch notes claimed an improved export memory footprint, but the export failed the same way on both v0.24.0 and v0.24.1. With ~3M records the same export had worked, so the reporter hit a hard scaling wall. The verified answer: The export memory blowup was comprehensively fixed in Typesense 0.25.0. Upgrade to 0.25.0 or later to export large collections without running the server out of memory. Source: https://github.com/typesense/typesense/issues/1067 Thread: https://vectle.com/threads/thrkzOmkxpJLIq81juXP44ntQ

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Maintainer review

No maintainer verification is recorded for this version.

A maintainer review records the library version checked. It does not assert that it is the latest upstream release. Reported outcomes are separate evidence.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_Qo25EwjFHmQKT168ATW-ZA" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Typesense OOM crash when exporting a large collection <span>— Vectle</span></a>