review agent's test run on the PR timed out: 30-minute review budget blown: how to fix

TL;DR

Do not run the full suite in the review path: run only the tests covering the changed files, wrap every invocation in a hard timeout, parallelize, and report honestly when tests were skipped. A review that dies waiting for tests delivers nothing; a review that reports partial test results still delivers the review.

The problem

review agent's test run on the PR timed out: 30-minute review budget blown: how to fix

When to use

• The test step exceeds the review job's timeout
• Review jobs are killed at the time limit during tests
• Slow suites make every review late
• A hung test hangs the entire review

When not to use

• Tests fail fast (a test-quality issue, not a time issue)
• The suite is fast and the timeout came from elsewhere (profile first)
• Policy requires the full suite per PR (then raise the budget; do not pretend it fits)

Steps

1. Run affected tests only, not the whole suite

Map changed files to the tests that cover them (per-language tooling: jest changed-since, pytest test selection, go test on changed packages) and run that subset.

npx jest --changedSince=origin/main --passWithNoTests

Expected output: a test run proportional to the PR's size instead of the repo's size.

2. Wrap every test invocation in a hard timeout

Use the timeout command or the runner's timeout-minutes so a hung test fails fast instead of eating the budget.

timeout 600 npx jest --changedSince=origin/main --passWithNoTests
echo "exit: $?"

Expected output: exit 124 on timeout instead of a hung job; the review step survives to report.

3. Parallelize where the suite allows

Use pytest-xdist, jest workers, or go test parallelism, and shard large suites across runners when the budget is tight.

Expected output: wall-clock test time drops by roughly the parallelism factor on multi-core runners.

4. On budget breach, report partial results honestly

If the targeted run still exceeds the budget, stop, report which tests ran and their result, and mark the rest as not-run. Never let the review die silently with the tests.

Expected output: reviewers see "12/15 test files passed, 3 not run (budget)" instead of a dead job.

5. Move the slow full suite out of the review path

Keep the full suite as a separate scheduled or merge-queue job, not a per-PR review step. Expected output: reviews stay fast; full-suite signal still exists where it belongs.

Tool compatibility

• timeout from coreutils: available on Linux and macOS runners.
• jest --changedSince, pytest with selection plugins, go test on changed packages: per-language affected-test mechanisms.
• CI timeout-minutes: the outer guard on GitHub Actions and equivalents.

Variant phrasings

test suite too slow for PR review

Same fix. Step 1 is the highest-leverage change; most "too slow" suites are running 10x more tests than the PR needs.

review job killed while running tests

Add step 2's hard timeout inside the job so the kill becomes a controlled failure with partial results instead of a dead job.

hung test blocked the code review

Timeouts (step 2) convert hangs into failures; step 4 converts failures into reported partial results. The review itself is never the casualty.

Why it happens

Review pipelines bolt the repo's standard test command onto the review job, and the standard command is tuned for CI thoroughness, not review latency. As the repo grows, the suite outgrows the review budget, and the first symptom is reviews dying inside the test step.

Edge cases

• Integration tests that genuinely need the full budget: move them out of the review path (step 5); do not stretch the review budget to fit them.
• Flaky tests that hang intermittently: timeouts plus retries-with-backoff; track flake rate separately.
• Tests requiring external services: stub or skip in review; run for real in the merge-queue job.
• Zero tests selected: --passWithNoTests (or the equivalent) keeps docs-only PRs green instead of failing on an empty selection.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_76-Zwm-5jZhdudYOpxeoYQ