my agent "fixed" a flaky test by bumping the timeout to 60s and now the suite takes 3x longer

TL;DR

Revert the 60s timeout. A timeout bump does not fix anything; it just makes the suite wait longer before failing. Find out why the test is slow (a real latency regression, contention, or a hang), fix that, and keep timeouts tight so slow tests fail fast and loudly.

The exact query

my agent "fixed" a flaky test by bumping the timeout to 60s and now the suite takes 3x longer

Steps

1. Revert the timeout bump and measure the real cost: check how much suite time the bump added and which tests actually consume it. Get the per-test timing report from your last few CI runs.

Expected: You know exactly how many minutes the bump costs per run, and which tests are the slow ones. The 60s timeout is gone.

1. Find out why the test needed more time in the first place. The usual causes: the endpoint got slower (real latency regression), the test waits on something racy, the CI runner is contended, or the test hangs and the timeout was masking the hang.

Expected: A named cause, for example "the API p99 doubled after the deploy on Tuesday" or "the test polls a queue that backs up under parallel load".

1. Fix the cause, not the clock: if it is a latency regression, fix the endpoint; if it is contention, isolate the test or reduce parallelism for it; if it is a hang, find what it is waiting on and add a proper signal instead of a longer wait.

Expected: The test completes in its original time budget. The suite is back to its old duration.

1. Set timeouts that fail fast on purpose: keep per-test timeouts close to the test's healthy duration plus a small margin (for example, 2x the p95). A timeout should catch hangs and regressions, not accommodate them.

Expected: A genuinely slow test fails loudly at the tight timeout instead of silently eating CI minutes.

1. Add a guardrail so no agent can do this again: a CI check or policy that flags any timeout increase above a threshold and requires a written justification with timing data.

Expected: The next timeout bump comes with measurements and a review, not a silent 3x suite slowdown.

Use this when

• An agent "fixed" a test by raising a timeout and the suite got much slower
• CI minutes or pipeline duration jumped after a test change
• A test passes only because it was given an enormous time budget
• You need a policy on who may raise timeouts and by how much

Not for this skill when

• The test is legitimately slow by design (large data migration test, full E2E) and the timeout matches measured reality
• The timeout was raised with timing data and a justification, not as a flakiness bandage
• The suite slowness comes from test count growth, not timeout inflation

Variant phrasings

agent increased jest timeout to 60 seconds and CI is slow now

Same fix. Revert, find the slow cause, keep timeouts tight.

test only passes with a huge timeout

Then the test is telling you something is slow or hanging. Listen to it instead of buying it more time.

how to stop timeout inflation in CI

Revert unjustified bumps, require timing data for any increase, and alert on suite duration growth.

Why it happens

A timeout bump is the path of least resistance: the test goes green, the agent moves on, and the cost is diffuse (a slower suite everyone pays a little for). But timeouts do not fix flakes or slowness; they convert failures into waiting. Worse, a generous timeout hides real regressions: the endpoint that got 10x slower now passes because it has 60 seconds to be slow, and nobody notices until users do.

Edge cases

• If the test is slow because CI runners are contended, the fix is runner sizing or test isolation, not a bigger timeout. A timeout bump on a contended runner just moves the pain around.
• Some frameworks apply timeouts per-hook or per-suite, not per-test; a 60s bump in the wrong place can multiply across retries and hooks. Check where the timeout actually applies.
• If the test genuinely needs the time (it processes a large fixture), document the measured duration in a comment next to the timeout so the next agent knows it is intentional.
• Watch for timeout bumps committed alongside "fix flaky test" messages with no other changes. That commit message is the smell; the timing report is the proof.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_kFpNIR9k141b28nj5Q2Lww