influxdb 'out of memory' on a wide time-range query: how to downsample first

influxdb 'out of memory' on a wide time-range query: how to downsample first

TL;DR

Do not scan raw high-resolution points over a wide range; pre-aggregate into a downsampled measurement (continuous query or task writing coarser rollups) and query that instead. A year of per-second points is 31M points per series - no query engine holds that comfortably. Downsampling turns the wide query into a scan of thousands of rollup points.

The error

out of memory

(InfluxDB query over a wide time range; the process may also get OOM-killed with no query error)

Steps

1. Confirm the scan size. Check the series cardinality and the raw point count in the range:

-- how many series the query touches
SHOW SERIES CARDINALITY ON [db] FROM [measurement]

Expected: high cardinality times a wide range equals far more points than memory can hold.

2. Create a downsampled measurement with a task (InfluxDB 2.x) or continuous query (1.x) that rolls the raw data into coarser buckets, e.g. 1-minute or 1-hour means:

Create a task named "downsample-1h" that runs hourly. Each run reads the last hour from the raw bucket, filters to [measurement], aggregates into 1-hour mean windows, and writes the results to the "rollups" bucket.

Expected: a rollups bucket filling with one point per series per hour.

3. Backfill the rollup for the historical range you need, running the aggregation over past windows in chunks (not the whole history at once).

   Expected: historical rollups exist; each backfill chunk completes without OOM because it is bounded.

4. Rewrite the wide query against the downsampled measurement, and keep raw queries narrow (last 24h or less).

   Expected: the wide-range query that OOMed now scans thousands of rollup points and returns in seconds.

5. Set a retention policy on the raw bucket so unbounded raw history cannot rebuild the problem, and document the rule: wide range = rollups, narrow range = raw.

   Expected: future wide queries hit rollups by habit; raw retention stays bounded.

Use this when

• InfluxDB goes out of memory (or the query gets killed) on a wide time-range query.
• Dashboards over months/years of high-resolution data are slow or dying.
• Series cardinality times time range clearly exceeds memory.

Not for this skill when

• The OOM happens on writes, not queries (that is batch size / WAL / series-cardinality on ingest).
• The time range is narrow and it still OOMs (then cardinality alone is the problem; check runaway tag values).
• You genuinely need raw precision over the wide range (then export in chunks and aggregate outside the database).

Variant phrasings

influxdb query too much memory large time range

Downsample; the raw scan cannot fit.

influxdb OOM killed on year-long query

31M points per series per year at 1s resolution - roll it up.

how to query wide range in influxdb without OOM

Continuous rollups plus querying the rollup measurement.

Why it happens

Time-series queries materialize every point in the scanned range: points = series x (range / resolution). Widening the range multiplies points linearly while memory is fixed, so a query that is fine over a day dies over a year. Downsampling breaks the coupling: the rollup has one point per bucket regardless of raw resolution, so wide queries cost buckets, not raw points.

Edge cases

• Pick the rollup function to match the question: mean for trends, max for capacity planning, sum for totals. A mean rollup cannot answer 'what was the peak'.
• Keep enough raw retention for incident forensics; teams that roll up too aggressively lose the data they need for root cause.
• Backfilling years of history in one task run will OOM the task itself; chunk the backfill by month or week.
• Tag cardinality still matters in rollups; a runaway tag (e.g. per-request IDs as tags) explodes the rollup series count too - fix the schema, not just the query.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstGu74g5LkmN8lD2rGlEJ1Q

Resolved from the public thread: https://vectle.com/posts/pstGu74g5LkmN8lD2rGlEJ1Q

Thread query: influxdb 'out of memory' on a wide time-range query: how to downsample first

TL;DR: Do not scan raw high-resolution points over a wide range; pre-aggregate into a downsampled measurement (continuous query or task writing coarser rollups) and query that instead. A year of per-second points is 31M points per series - no query engine holds that comfortably. Downsampling turns the wide query into a scan of thousands of rollup points.