Elasticsearch OutOfMemory on a small VPS: heap sizing and mlockall

Elasticsearch OutOfMemory on a small VPS: heap sizing and mlockall

TL;DR

[accepted answer]: this is a heap space issue. Rules of thumb: 1. Give Elasticsearch about half the box's RAM as heap, e.g.

Use this when you run into the situation in the title.

When not to use this skill: unrelated tasks. It covers only the procedure above.

Compatibility

The steps above apply to the commands named in them. This skill does not pin a version, so if a flag looks different on your machine, check your installed version's docs first.

Details

export ESHEAPSIZE=2g on a 4GB machine. Never set heap above ~50% of RAM (the OS needs the rest for filesystem cache), and never above ~32GB or you lose compressed object pointers. 2. Lock the memory for the JVM: set bootstrap.mlockall to true in the config so the heap never gets swapped out. 3. Investigate the disk: with only 30 small articles a 26GB data folder means something is off. Run GET cat/indices to find which index is huge (old log indexes that never got deleted are the usual suspect), and GET /nodes/stats for detailed node info. Tiny VPS + defaults is the classic combo: the default heap is small and shards/indexes pile up silently until the node falls over.

Context: Stack Overflow #34601697 (accepted answer, 17 votes): a small blog on a 4GB VPS runs fine for two months, then Elasticsearch dies with an Out of Memory (heap space) error. The asker also notices the data folder is 26GB for just 30 small articles.

Source: Stack Overflow #34601697 (accepted answer, 17 votes): a small blog on a 4GB VPS runs fine for two months, then Elasticsearch dies with an Out of Memory (heap space) error. The asker also notices the data folder is 26GB for just 30 small articles.

Verified resolution: [accepted answer]: this is a heap space issue. Rules of thumb: 1. Give Elasticsearch about half the box's RAM as heap, e.g. export ESHEAPSIZE=2g on a 4GB machine. Never set heap above ~50% of RAM (the OS needs the rest for filesystem cache), and never above ~32GB or you lose compressed object pointers. 2. Lock the memory for the JVM: set bootstrap.mlockall to true in the config so the heap never gets swapped out. 3. Investigate the disk: with only 30 small articles a 26GB data folder means something is off. Run GET cat/indices to find which index is huge (old log indexes that never got deleted are the usual suspect), and GET /nodes/stats for detailed node info. Tiny VPS + defaults is the classic combo: the default heap is small and shards/indexes pile up silently until the node falls over.

Source URL: https://stackoverflow.com/questions/34601697