agent ran the codemod over test fixtures too and now the golden files dont match the new behavior

TL;DR

The codemod treated fixtures as source code and rewrote them, so golden-file tests now fail. Decide per fixture set: if the fixtures encode behavior the codemod intentionally changed, regenerate the goldens deliberately and review the diff; if the fixtures should not have changed, revert them. Then add ignore patterns so the next run skips fixtures, and make the agent list its target files before running.

The query

agent ran the codemod over test fixtures too and now the golden files dont match the new behavior

Use this when

• Codemod rewrote test fixtures and golden files fail
• Need to exclude fixtures from a codemod run
• Golden files do not match after a codemod
• An agent's codemod touched files outside its scope

Not for

• Golden files failing because the new behavior is wrong (fix the codemod)
• Fixtures that intentionally encode the migrated behavior (regenerate deliberately)
• Test failures unrelated to file rewriting (run the tests, read the error)

Steps

1. Scope the damage

git status --short | head -40
git diff --stat | tail -5

Expected output: the list of rewritten files, split into intended targets and fixtures. Knowing the exact fixture set bounds the recovery.

2. Decide per fixture set: revert or regenerate

If the fixtures represent inputs the codemod should never transform (sample data, third-party snapshots): revert them with git checkout. If the fixtures are golden outputs of the transformed code and the new output is correct: regenerate them with the project's blessed command and review the full diff.

Expected output: a deliberate decision per set, not a blanket revert or blanket accept. The diff review is where you catch the codemod changing behavior it should not have.

3. Revert or regenerate

git checkout -- [fixture paths to revert]

or run the project's golden-update command for the sets you accept. Expected output: git status shows only intended changes plus deliberately regenerated goldens.

4. Exclude fixtures from the codemod

Add ignore patterns for fixture directories to the codemod invocation or config (jscodeshift --ignore-pattern, ast-grep --globs with negation, or the tool's config file). Expected output: a dry run of the codemod lists zero fixture files in its targets.

5. Make the agent preview its file list

The agent's codemod procedure: run with --dry-run first, print the file list, and confirm no fixture, snapshot, or generated directory appears before the real run.

Expected output: the next out-of-scope rewrite is caught at the preview, not after the fact.

Tool compatibility

Tool-agnostic: jscodeshift, ast-grep, libcst, and regex codemods all need explicit scoping.

Variant phrasings

codemod rewrote test fixtures golden files fail

Steps 1 through 3: scope, decide revert vs regenerate, execute.

exclude fixtures from jscodeshift run

Step 4: --ignore-pattern for fixture directories, verified with a dry run.

golden files dont match after codemod

Step 2: the mismatch is either correct-new-behavior (regenerate deliberately) or out-of-scope rewriting (revert).

Why it happens

Codemods match by pattern, and fixtures are designed to look like real code: that is their job. Without explicit scoping, the transform cannot distinguish "code to migrate" from "data shaped like code". Golden files then fail because the expected outputs were rewritten alongside the inputs, destroying the test's ability to judge the transform.

Edge cases

• Regenerating goldens from a buggy codemod bakes the bug into the test suite: review the regenerated diff as skeptically as a code change.
• Some fixtures SHOULD be transformed (e.g. fixtures of the new API): excluding everything fixture-shaped is as wrong as including everything. Decide per directory.
• Snapshot files of the codemod's own output are a special case: they must be regenerated, never hand-edited, or they stop testing anything.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_KFvEATMinU-jeBAi52Wekw