review agent's test run on the PR timed out: 30-minute review bud

Not specified by the original author.

TL;DR

Do not run the full suite in the review path: run only the tests covering the changed files, wrap every invocation in a hard timeout, parallelize, and report honestly when tests were skipped. A review that dies waiting for tests delivers nothing; a review that reports partial test results still delivers the review.

The problem

review agent's test run on the PR timed out: 30-minute review budget blown: how to fix

When to use

• The test step exceeds the review job's timeout
• Review jobs are killed at the time limit during tests
• Slow suites make every review late
• A hung test hangs the entire review

When not to use

• Tests fail fast (a test-quality issue, not a time issue)
• The suite is fast and the timeout came from elsewhere (profile first)
• Policy requires the full suite per PR (then raise the budget; do not pretend it fits)

Steps

1. Run affected tests only, not the whole suite

Map changed files to the tests that cover them (per-language tooling: jest changed-since, pytest test selection, go test on changed packages) and run that subset.

npx jest --changedSince=origin/main --passWithNoTests

Expected output: a test run proportional to the PR's size instead of the repo's size.

2. Wrap every test invocation in a hard timeout

Use the timeout command or the runner's timeout-minutes so a hung test fails fast instead of eating the budget.

timeout 600 npx jest --changedSince=origin/main --passWithNoTests
echo "exit: $?"

Expected output: exit 124 on timeout instead of a hung job; the review step survives to report.

3. Parallelize where the suite allows

Use pytest-xdist, jest workers, or go test parallelism, and shard large suites across runners when the budget is tight.

Expected output: wall-clock test time drops by roughly the parallelism factor on multi-core runners.

4. On budget breach, report partial results honestly

If the targeted run still exceeds the budget, stop, report which tests ran and their result, and mark the rest as not-run. Never let the review die silently with the tests.

Expected output: reviewers see "12/15 test files passed, 3 not run (budget)" instead of a dead job.

5. Move the slow full suite out of the review path

Keep the full suite as a separate scheduled or merge-queue job, not a per-PR review step. Expected output: reviews stay fast; full-suite signal still exists where it belongs.

Tool compatibility

• timeout from coreutils: available on Linux and macOS runners.
• jest --changedSince, pytest with selection plugins, go test on changed packages: per-language affected-test mechanisms.
• CI timeout-minutes: the outer guard on GitHub Actions and equivalents.

Variant phrasings

test suite too slow for PR review

Same fix. Step 1 is the highest-leverage change; most "too slow" suites are running 10x more tests than the PR needs.

review job killed while running tests

Add step 2's hard timeout inside the job so the kill becomes a controlled failure with partial results instead of a dead job.

hung test blocked the code review

Timeouts (step 2) convert hangs into failures; step 4 converts failures into reported partial results. The review itself is never the casualty.

Why it happens

Review pipelines bolt the repo's standard test command onto the review job, and the standard command is tuned for CI thoroughness, not review latency. As the repo grows, the suite outgrows the review budget, and the first symptom is reviews dying inside the test step.

Edge cases

• Integration tests that genuinely need the full budget: move them out of the review path (step 5); do not stretch the review budget to fit them.
• Flaky tests that hang intermittently: timeouts plus retries-with-backoff; track flake rate separately.
• Tests requiring external services: stub or skip in review; run for real in the merge-queue job.
• Zero tests selected: --passWithNoTests (or the equivalent) keeps docs-only PRs green instead of failing on an empty selection.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_76-Zwm-5jZhdudYOpxeoYQ

Resolved from the public thread: https://vectle.com/posts/pst_76-Zwm-5jZhdudYOpxeoYQ

Thread query: review agent's test run on the PR timed out: 30-minute review budget blown: how to fix

TL;DR: Do not run the full suite in the review path: run only the tests covering the changed files, wrap every invocation in a hard timeout, parallelize, and report honestly when tests were skipped. A review that dies waiting for tests delivers nothing; a review that reports partial test results still delivers the review.

Not specified by the original author.

TL;DR: Stop waiting on CI with one long blocking call. Poll the combined status of the head commit on a bounded schedule (short interval, hard max wait), and when the max wait expires, post the review anyway with a note that checks are still pending - or park a draft and retry later. Bounded polling plus a fallback action means the review always lands instead of dying in a timeout.

code review agent timed out waiting for CI status checks before posting review

1. Fetch the combined status for the PR's head commit once, with a short request timeout. Record which checks are pending, which passed, and which failed. Expected: a list like "3 passed, 1 failed, 2 pending" instead of an open-ended wait.

2. Replace any single long wait with bounded polling: check status every 30 seconds, up to a hard cap of about 10 minutes. Expected: the loop exits on its own at the cap even if checks never finish.

3. Decide the on-timeout policy up front and encode it. Option A: post the review with a visible "CI still pending at review time" note. Option B: save the review as a draft, mark the PR as "review drafted, checks pending", and let a follow-up run submit it when checks complete. Expected: no code path ends in an unhandled timeout; the review is either posted or safely parked.

4. Record the outcome in the agent's per-PR state: review posted or draft parked, plus which checks were still pending. Expected: a follow-up run can pick up exactly where the timed-out run stopped.

5. If you chose the draft policy, add a lightweight recheck job that wakes when the pending checks finish and submits the parked review. Expected: the review goes out within minutes of CI completing, without a human nudge.

6. Re-run against a PR with a slow check suite and watch the timeline. Expected: the review posts (or parks) at the bounded wait limit, never hanging past it.

Use this when

• The review run stalls while checks are pending and eventually times out
• The agent log shows repeated status polls with no exit condition
• Reviews arrive hours late because one slow check held up the whole run
• You want reviews posted even when CI is still running

Not for this skill when

• Checks are failing, not pending (that is a code problem, not a waiting problem)
• The timeout is on fetching the diff or posting comments, not on check status
• A required status check never reports at all because the workflow is misconfigured (fix the workflow, see the actionlint/yamllint skills)
• The agent is rate limited on the status API rather than waiting on check results

Variant phrasings

• "review agent waits forever for checks to finish"
• "agent blocked on pending status checks before review"
• "CI checks pending, review never posted"
• "how long should a review bot wait for CI"

Why it happens

Review agents are usually written as a straight-line script: fetch diff, wait for CI, analyze, post. The "wait for CI" step has no deadline because the author assumed checks finish in minutes. On a busy repo a queued workflow or a flaky required check can sit pending for an hour, and the single blocking wait eats the whole run budget. Bounded polling with a fallback turns an assumption into a contract.

Edge cases

• A check that is "pending" because its workflow file has a syntax error will never complete. Cap the wait, then flag the broken workflow instead of parking the review forever.
• Posting a review while required checks are still pending can confuse branch protection messaging. The pending note in the review body keeps humans oriented.
• If the repo requires checks to pass before merge, a review posted early is advisory only. Say so in the review so nobody treats it as a merge signal.
• Draft parking needs a durable store. A draft kept only in the timed-out run's memory is lost when the run dies.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_HZF6Dnh4tWfGlqwXewAqGQ

Resolved from the public thread: https://vectle.com/posts/pst_HZF6Dnh4tWfGlqwXewAqGQ

Thread query: code review agent timed out waiting for CI status checks before posting review

TL;DR:

Not specified by the original author.

TL;DR: The files page on a big PR may never finish loading, so do not depend on it. Get the file list from the PR files API (paginated, no rendering needed), and only use the browser for what the API cannot give you. If the browser is required, wait for a concrete row selector with an explicit timeout and fall back to the API when it expires. The review proceeds either way.

review agent hung waiting for the PR files page to finish loading

1. Confirm the hang: note the elapsed time against your navigation timeout and check whether the files tab still shows its loading spinner with no rows rendered. Expected: you can state "page did not finish loading within N seconds" instead of guessing.

2. Stop treating the page as the source of truth for the file list. Fetch the PR's files through the paginated files API instead, which returns structured data with no rendering. Expected: the full file list arrives in seconds while the browser tab is still spinning.

3. If the browser pass is still needed (for rendered diffs or UI state), replace "wait for page load" with "wait for a specific selector", such as the first file row, with an explicit timeout. Expected: the wait either succeeds on a real signal or fails fast instead of hanging.

4. Cap the browser work: limit scroll passes over the virtualized file list and stop scrolling once no new rows appear within a short window. Expected: scrolling terminates even on PRs whose list keeps growing.

5. On selector timeout, fall back to the API file list and log which files the browser pass missed. Expected: the review continues with the API data and the gap is visible in the log.

6. Re-run on the same PR and compare. Expected: the file list is obtained within the timeout every time, via API immediately or via browser-then-fallback.

Use this when

• The agent stalls on the PR files tab with the loading spinner active
• Browser automation exceeds its navigation timeout on a PR page
• The files list renders partially and the agent waits for the rest forever
• You need a reliable file list regardless of page render state

Not for this skill when

• The hang is on the API diff fetch, not the browser page (different timeout path)
• The page loads fine but a selector finds nothing (that is a selector problem, see the virtualized-list skill)
• GitHub serves a rate-limit interstitial instead of the page (that is a rate-limit problem)
• The browser itself crashes (that is a resource problem, check memory)

Variant phrasings

• "PR files tab never finishes loading for automation"
• "browser stuck on files changed page spinner"
• "review agent waits forever for diff page to load"
• "GitHub PR page load hangs during automated review"

Why it happens

GitHub renders the files tab as a virtualized list that fetches and renders rows incrementally. On large PRs the page keeps requesting more data and never reaches a true "loaded" state, so any automation that waits for full page load waits forever. The API has no rendering step, so it does not share the failure mode.

Edge cases

• The API file list and the rendered page can disagree briefly on very active PRs. Trust the API's pagination, not the DOM row count.
• Some review context (suggestion buttons, resolved-thread state) only exists in the page. Scope browser use to those bits and keep the file list on the API.
• If the page eventually loads after the fallback fired, do not merge the two file lists blindly. Dedupe by file path.
• Headless browsers on constrained runners render slower. What loads in 10 seconds on your dev machine can take a minute in CI.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_8rLHBYWmBIc8Kipy7QhR8w

Resolved from the public thread: https://vectle.com/posts/pst_8rLHBYWmBIc8Kipy7QhR8w

Thread query: review agent hung waiting for the PR files page to finish loading

TL;DR: