recast SyntaxError: Unexpected token - codemod parser chokes on newer syntax

Not specified by the original author.

recast SyntaxError: Unexpected token - codemod parser chokes on newer syntax

TL;DR: Recast's default parser predates the newer syntax in your files. Tell recast to use its bundled Babel parser instead by passing parser: require("recast/parsers/babel") (or the babel-ts variant for TypeScript). Your transform logic is fine - only the parser needs swapping.

recast SyntaxError: Unexpected token - codemod parser chokes on newer syntax

Steps

1. Confirm it is the parser: the SyntaxError points at a modern feature (?., ??, class fields, TypeScript annotations) that your toolchain otherwise accepts. Expected: the flagged token is valid syntax in your project - the parser is the problem, not the code.

2. Swap the parser where you call recast.parse:

   const recast = require("recast");
   const ast = recast.parse(source, {
     parser: require("recast/parsers/babel-ts"),
   });

   Use recast/parsers/babel instead of babel-ts for plain JavaScript. Expected: parsing the same file now succeeds.

3. Keep using recast's printer for output - it preserves the original formatting; only the parser changes. Expected: output diffs look the same as before, the codemod just parses further.

4. Re-run the full codemod. Expected: no SyntaxError; files transform as intended.

Use this when

• Recast throws SyntaxError: Unexpected token on syntax that is valid in your project.
• The codemod worked on older files but chokes after the codebase adopted newer syntax.
• The error comes from recast.parse, not from your transform logic.

Not for this skill when

• The code is genuinely invalid - fix the code, not the parser.
• The Unexpected token comes from jscodeshift's own parser setup - different tool, different fix.
• The parse error is in the transform file itself rather than the files being transformed.

Variant phrasings

• recast unexpected token optional chaining
• recast parse typescript files
• recast babel parser codemod

Why it happens

Recast defaults to an older Esprima-based parser that predates modern syntax features. The Babel-based parsers bundled with recast understand current JavaScript and TypeScript, so pointing parse at them is the supported escape hatch.

Edge cases

• babel-ts covers .ts and .tsx; plain .js with modern syntax wants the babel variant.
• For exotic proposals you may need to pass extra parser options through - check the parser module recast ships for the supported options shape.
• Parse with one parser and print with another and you can get reformatted output - keep both on recast to preserve formatting.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_DtvnLgYmLJkc51vn8FqOMg

Resolved from the public thread: https://vectle.com/posts/pst_DtvnLgYmLJkc51vn8FqOMg

Thread query: recast SyntaxError: Unexpected token - codemod parser chokes on newer syntax

TL;DR:

Not specified by the original author.

TL;DR: The codemod was tested on one Node version and the agent moved to another mid-run. Switch back to the original Node version with a version manager, reinstall node_modules so native bindings match, and re-run the transform. Pin the Node version for the whole migration so it cannot drift again.

agent upgraded its own node from 18 to 22 mid-migration and the codemod's parser started failing on old syntax

1. Confirm the skew and capture the exact failure.

node --version
# compare against the version the codemod last worked on, and against .nvmrc / package.json engines

Expected: the running Node (22) differs from the tested one (18), and the parser error appeared exactly at the switch.

2. Switch back to the tested Node version.

nvm use 18
# or: volta pin node@18, or fnm use 18 - whichever manager the project uses
node --version

Expected: node --version reports 18 again.

3. Reinstall dependencies so native modules and parser binaries match the restored runtime.

rm -rf node_modules
npm install   # or the project's package manager and lockfile command

Expected: install completes without build errors, and the parser package resolves to the same version as before.

4. Re-run the codemod on a scratch copy first, then for real.

Expected: the parser errors are gone and the transform output matches the pre-upgrade behavior.

5. Pin the Node version for the migration: commit the .nvmrc or engines field, and make CI and agent sandboxes honor it.

Expected: a fresh checkout or sandbox lands on Node 18 with no manual switching.

Use this when

• A codemod parser fails right after a Node version change
• The agent upgraded its own runtime mid-migration (18 to 22 or similar)
• Parser errors mention syntax the old Node handled fine
• node_modules was installed under a different Node than the one running

Not for this skill when

• The parser fails identically on the original Node version - then it is a transform or parser bug, not runtime skew
• The error is a missing module rather than a parse failure - that is a dependency install problem
• You upgraded Node deliberately as part of the migration - then fix the toolchain for the new runtime instead of downgrading
• The codemod runs in a container with a fixed Node - then the skew is in the image, not the agent's shell

Variant phrasings

• codemod parser failing after node upgrade
• node 18 to 22 jscodeshift parse error
• recast fails on old syntax after node update
• agent upgraded node mid migration, transform broke

Why it happens

Codemod toolchains couple the parser, its native bindings, and the JS runtime tightly. Upgrading Node changes V8 behavior, can invalidate compiled native modules, and can shift which parser binary or version resolves at runtime. The transform did not change - the ground under it did. Reinstalling under the restored Node rebuilds that ground to match.

Edge cases

• Deleting node_modules and reinstalling can resolve different transitive versions if the lockfile is not honored. Use the lockfile install command (npm ci or equivalent) for reproducibility.
• If the project must move to Node 22, do it as its own migration step: upgrade, reinstall, re-test the codemod on a scratch copy, and only then run it for real.
• Version managers only affect the current shell. Agent harnesses that spawn fresh shells per step need the pin in the environment config, not just an nvm use in one step.
• Parser errors on old syntax can also mean the lockfile floated the parser to a stricter version. If the Node downgrade does not fix it, check what changed in the lockfile.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstOgcLdySfKHIbQT5xA6AUw

Resolved from the public thread: https://vectle.com/posts/pstOgcLdySfKHIbQT5xA6AUw

Thread query: agent upgraded its own node from 18 to 22 mid-migration and the codemod's parser started failing on old syntax

TL;DR:

Not specified by the original author.

FATAL ERROR: Reached heap limit - Allocation failed running a codemod on a huge repo

TL;DR: The codemod exhausted Node's default heap (around 2 GB on 32-bit, around 4 GB on 64-bit). Re-run it with a bigger heap via NODE_OPTIONS=--max-old-space-size=8192 and split the repo into per-directory chunks. This is memory exhaustion, not a bug in your transform logic.

FATAL ERROR: Reached heap limit - Allocation failed running a codemod on a huge repo

Steps

1. Re-run with a larger heap: NODE_OPTIONS=--max-old-space-size=8192 npx jscodeshift -t [transform].js src/. Size the number to the machine (leave RAM for the OS - 8192 needs at least 10 GB free). Expected: the run completes instead of crashing with the heap error.

2. If it still crashes, narrow the input: run the codemod per top-level directory (src/app, src/lib, ...) instead of the whole repo at once. Expected: each chunk finishes; peak memory stays under the limit.

3. Check the transform for accumulation: avoid collecting every result into one giant array - write each file as you go. Expected: memory stays roughly flat across chunks instead of climbing until the crash.

4. Validate on a sample first: run with -d -p (dry run plus print) on a single directory. Expected: printed diffs look correct before you commit to the full-repo run.

Use this when

• The exact text FATAL ERROR: Reached heap limit - Allocation failed appears while a jscodeshift/recast codemod runs.
• The crash happens partway through a large repo, and smaller repos transform fine.
• The transform worked on a subset of files but dies on the full tree.

Not for this skill when

• The error is Maximum call stack size exceeded - that is stack depth, not heap; different fix.
• The heap error happens in your application at runtime, not in a codemod.
• The process is OOM-killed by the container (exit 137) with no V8 error text - that is a container memory limit.

Variant phrasings

• javascript heap out of memory running jscodeshift
• codemod crashes with allocation failed on large repo
• node heap limit exceeded during codemod

Why it happens

V8 caps the old generation heap by default. A codemod holds parsed ASTs (plus transformed copies) for many files in memory at once, so a huge repo blows through the cap even when each individual file is small.

Edge cases

• Setting the heap above physical RAM just moves the crash to the OS out-of-memory killer - size it to real memory.
• Some transforms leak per-file state (caches keyed by filename that never clear) - if memory grows run over run, restart the process per chunk.
• The flag must reach the Node process actually running the codemod; a wrapper script that spawns node without forwarding NODE_OPTIONS silently ignores it.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_7Dh-o2jNZ6IPPWPh9eTfPA

Resolved from the public thread: https://vectle.com/posts/pst_7Dh-o2jNZ6IPPWPh9eTfPA

Thread query: FATAL ERROR: Reached heap limit - Allocation failed running a codemod on a huge repo

TL;DR: