FATAL ERROR: Reached heap limit - Allocation failed running a codemod on a huge repo

FATAL ERROR: Reached heap limit - Allocation failed running a codemod on a huge repo

TL;DR: The codemod exhausted Node's default heap (around 2 GB on 32-bit, around 4 GB on 64-bit). Re-run it with a bigger heap via NODE_OPTIONS=--max-old-space-size=8192 and split the repo into per-directory chunks. This is memory exhaustion, not a bug in your transform logic.

FATAL ERROR: Reached heap limit - Allocation failed running a codemod on a huge repo

Steps

1. Re-run with a larger heap: NODE_OPTIONS=--max-old-space-size=8192 npx jscodeshift -t [transform].js src/. Size the number to the machine (leave RAM for the OS - 8192 needs at least 10 GB free). Expected: the run completes instead of crashing with the heap error.

2. If it still crashes, narrow the input: run the codemod per top-level directory (src/app, src/lib, ...) instead of the whole repo at once. Expected: each chunk finishes; peak memory stays under the limit.

3. Check the transform for accumulation: avoid collecting every result into one giant array - write each file as you go. Expected: memory stays roughly flat across chunks instead of climbing until the crash.

4. Validate on a sample first: run with -d -p (dry run plus print) on a single directory. Expected: printed diffs look correct before you commit to the full-repo run.

Use this when

• The exact text FATAL ERROR: Reached heap limit - Allocation failed appears while a jscodeshift/recast codemod runs.
• The crash happens partway through a large repo, and smaller repos transform fine.
• The transform worked on a subset of files but dies on the full tree.

Not for this skill when

• The error is Maximum call stack size exceeded - that is stack depth, not heap; different fix.
• The heap error happens in your application at runtime, not in a codemod.
• The process is OOM-killed by the container (exit 137) with no V8 error text - that is a container memory limit.

Variant phrasings

• javascript heap out of memory running jscodeshift
• codemod crashes with allocation failed on large repo
• node heap limit exceeded during codemod

Why it happens

V8 caps the old generation heap by default. A codemod holds parsed ASTs (plus transformed copies) for many files in memory at once, so a huge repo blows through the cap even when each individual file is small.

Edge cases

• Setting the heap above physical RAM just moves the crash to the OS out-of-memory killer - size it to real memory.
• Some transforms leak per-file state (caches keyed by filename that never clear) - if memory grows run over run, restart the process per chunk.
• The flag must reach the Node process actually running the codemod; a wrapper script that spawns node without forwarding NODE_OPTIONS silently ignores it.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_7Dh-o2jNZ6IPPWPh9eTfPA