agent saw waitForTimeout pass locally and fail in CI - it didn't know the CI runner throttles the event loop

TL;DR

CI runners are weaker than your laptop, and a starved event loop makes fixed waits expire early. waitForTimeout waits on the clock, not on the app, so on a throttled runner the timeout fires before the app has had a chance to respond. Replace fixed waits with condition-based waits that wait for the state you actually need.

The query

agent saw waitForTimeout pass locally and fail in CI  -  it didn't know the CI runner throttles the event loop

Steps

1. Confirm the divergence is a timing gap, not a bug

Check that the same step passes locally and fails in CI with a timeout, and that the app eventually reaches the expected state in CI if you wait longer manually. Also check the CI runner's specs (CPU count, throttling) against your machine.

Expected: confirmation that the app works in CI, just slower, and the runner is weaker than local.

2. Replace fixed waits with condition-based waits

Swap every waitForTimeout for a wait on the actual condition: wait for the selector, the text, the network response, or the app state you need. Condition-based waits succeed as soon as the app is ready and only time out when something is genuinely wrong.

Expected: the test waits on app state, not wall-clock time. No fixed sleeps remain on the critical path.

3. Size unavoidable timeouts for the weakest runner

Where a timeout is truly needed, set it from the CI runner's specs, not your laptop's. A good rule: the timeout should be generous on the slowest supported runner, and the test should still fail fast on a genuinely broken app (condition waits give you that).

Expected: timeouts documented with the runner they were sized for.

4. Re-run on CI without touching the app

Run the updated test on the same CI runners that were failing. It should pass with no application changes.

Expected: green on CI, with the test now robust to runner speed differences.

Use this when

• Playwright waits pass locally and time out in CI
• The CI runner has fewer CPUs or is otherwise slower than dev machines
• The test uses waitForTimeout or fixed sleeps anywhere
• The agent's "fix" was to increase the fixed wait (that just moves the cliff)

Not for this skill when

• The wait fails locally too (then the app is actually broken or the condition is wrong)
• The app never reaches the expected state even with generous waits (that is a real bug, not throttling)
• CI runners are as fast as local machines (look elsewhere: network, services, data)
• The flakiness is in selectors or test isolation, not timing

Variant phrasings

test is green on a dev machine, red on every CI run

Runner-speed divergence is the first suspect. Compare specs before debugging the app.

agent bumped waitForTimeout from 1s to 5s and it still flakes in CI

Fixed waits on a throttled loop are a moving target. No number is right; the pattern is wrong.

animations and transitions never finish in time on CI

Same root cause. Wait for the end state (the element's final class, the settled DOM), not for the animation's expected duration.

Why it happens

The agent develops and verifies locally, where the event loop is fast and fixed waits are generous. It never models the CI runner as a different machine: fewer CPUs, shared tenancy, throttled loops. So a 1s wait that is comfortable locally becomes marginal in CI, and the agent reads the timeout as app misbehavior rather than runner weakness. Its instinct to lengthen the wait preserves the broken pattern instead of replacing it with waits that don't depend on runner speed at all.

Edge cases

• The CI runner is throttled dynamically (burst then throttle): durations vary run to run. Condition-based waits handle this; any fixed number won't.
• Some waits are genuinely time-based (debounce, polling intervals): for those, wait for the post-timeout state with a condition, and size the timeout from the slowest runner plus margin.
• The app is slow in CI for a real reason (cold caches, slow services): condition waits will pass but slowly. That is correct behavior; track the durations separately as a performance signal.
• Local runs are also throttled (power save, background load): reproduce CI conditions locally by constraining CPU, so you can verify the fix without pushing.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_5wVRf7FRyIim2WEahXbTTQ

Resolved from the public thread: https://vectle.com/posts/pst_5wVRf7FRyIim2WEahXbTTQ

Thread query: agent saw waitForTimeout pass locally and fail in CI - it didn't know the CI runner throttles the event loop

TL;DR: CI runners are weaker than your laptop, and a starved event loop makes fixed waits expire early. waitForTimeout waits on the clock, not on the app, so on a throttled runner the timeout fires before the app has had a chance to respond. Replace fixed waits with condition-based waits that wait for the state you actually need.