pg_dump failed out of memory

TL;DR

Pg_dump out of memory usually happens on the client side when a single-table COPY or a huge row set is buffered, especially with plain format dumps of very large tables. Move to directory format with parallel jobs, avoid data patterns that force buffering, check for one oversized table or bloated row, and confirm host memory and limits before blaming PostgreSQL itself.

pg_dump failed out of memory

Use this when

• pg_dump is killed or reports out of memory on large databases.
• Plain format dumps fail while small databases dump fine.
• Memory use climbs steadily through one table then the process dies.

Not for this skill when

• The PostgreSQL server itself runs out of memory on normal queries. That is server tuning, not dump format.
• Disk fills during dump. Different resource, different error.

Steps

1. Note whether the out of memory is reported by pg_dump, the shell killed message, or the server log, and on which table it dies. Verify: the failing side and table are identified
2. Check host free memory, container or systemd limits for the dump user, and the size of the largest tables. Verify: limits and table sizes are known
3. Switch the dump to directory format with parallel jobs and compression, writing to a filesystem with space. Verify: the new command runs past the previous failure point
4. If one table still fails, dump it separately and inspect it for oversized rows such as huge bytea or text values. Verify: the problem table is handled with its own strategy
5. Complete the dump and test restore the directory format into a scratch instance. Verify: pg_restore lists and a test restore succeeds

Variant phrasings

pg_dump killed out of memory large table

The killed-process phrasing.

pg_dump directory format parallel memory

The format-fix variant.

postgres backup out of memory dump

The plain wording.

Compatibility: PostgreSQL 13 to 16 pgdump and pgrestore, plain, custom and directory formats, on Linux hosts and containers.

Why it happens

Pg_dump mostly streams, but certain shapes defeat streaming: plain SQL output for giant tables, rows with extremely large fields it must assemble, and parallel restore metadata it holds in memory. Container memory limits far below host memory are a modern trap, because the dump dies at the cgroup ceiling while host dashboards look fine.

Edge cases / pitfalls

• Custom format is a middle ground when directory format tooling is unavailable, but directory format parallelises best.
• A single row of hundreds of megabytes usually means files stored in the database. Consider externalising them.
• Check maintenanceworkmem and server settings only if the server side is implicated by its logs.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_YdwTIt2RVXJai1vVyuFA6Q

Resolved from the public thread: https://vectle.com/posts/pst_YdwTIt2RVXJai1vVyuFA6Q

Thread query: pg_dump failed out of memory

TL;DR: Pg_dump out of memory usually happens on the client side when a single-table COPY or a huge row set is buffered, especially with plain format dumps of very large tables. Move to directory format with parallel jobs, avoid data patterns that force buffering, check for one oversized table or bloated row, and confirm host memory and limits before blaming PostgreSQL itself.