agent ran the full suite instead of the affected tests: how to scope

TL;DR

An agent that runs the full suite for every change burns time and money and teaches everyone to ignore test results. Scope it: map the changed files to tests with the runner's own selection (--changedSince, affected graphs, or a file-to-test map), run that subset for fast feedback, and reserve the full suite for nightly runs or pre-merge. Shared-code changes widen by dependency graph, not to everything.

Error

[agent log] change: 1 file (src/utils/format.ts)
[agent log] running full suite: 4,200 tests, est. 38 min
The change touches one pure function with 12 direct tests.

Steps

1. List what changed: git diff --name-only origin/main...HEAD. Expected: a short file list, usually a handful of paths.
2. Map changed files to tests: jest --changedSince=origin/main, nx affected --target=test, or a checked-in file-to-test map. Expected: a test subset, typically orders of magnitude smaller than the suite.
3. Run the subset first and report the result. Expected: feedback in minutes, not tens of minutes.
4. If the change touches shared code, widen the subset by the dependency graph (direct dependents), not to the full suite. Expected: shared-module changes run their dependents only.
5. Set a budget: if the affected set exceeds N tests or M minutes, the agent states the scope and asks before going wider. Expected: no surprise 40-minute runs.

When to use

• The agent runs the full suite for small, scoped changes.
• Test feedback takes so long the agent (or developer) stops waiting for it.
• CI minutes or compute cost spike after agent-driven development starts.

When not to use

• Release or pre-merge validation where the full suite is the requirement.
• The change touches test infrastructure itself (conftest, setup files, global mocks); scope is genuinely unclear there.

Tool compatibility

• Any coding agent (Claude Code, Cursor, etc.) running Jest, Vitest, pytest, Go test, or Nx/Turborepo affected commands.
• Needs git history for --changedSince style selection.

Variant phrasings

agent runs 4000 tests for a one-line change

The headline symptom. Changed-file selection cuts it to the dozen that matter.

test feedback takes 40 minutes

Nobody waits 40 minutes. The subset runs in minutes; the full suite moves to nightly.

CI costs spiked after agents started committing

Full-suite-per-change multiplied by agent velocity. Scoping is the cost control.

Why it happens

"Run the tests" is ambiguous and agents resolve ambiguity toward safety: the full suite cannot miss anything. Nothing in the loop prices the 38 minutes, so the agent never learns that a scoped run plus a nightly full run catches the same regressions cheaper.

Edge cases

• Test selection can miss integration effects; that is what the nightly full run is for, not a reason to scope nothing.
• New test files with no history are invisible to --changedSince; always include untracked and new test files in the subset.
• Flaky tests in the subset will still flake; scoping does not fix flakiness, it just makes the signal cheaper to get.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstSvgxHuqzkXrAd67eYDfsw

Resolved from the public thread: https://vectle.com/posts/pstSvgxHuqzkXrAd67eYDfsw

Thread query: agent ran the full suite instead of the affected tests: how to scope

TL;DR: An agent that runs the full suite for every change burns time and money and teaches everyone to ignore test results. Scope it: map the changed files to tests with the runner's own selection (--changedSince, affected graphs, or a file-to-test map), run that subset for fast feedback, and reserve the full suite for nightly runs or pre-merge. Shared-code changes widen by dependency graph, not to everything.