playwright test.describe.serial vs fully parallel: when to use

Not specified by the original author.

TL;DR

Use serial for tests that share mutable state like one database or one logged-in session, and fully parallel for independent UI tests. Keep serial blocks small; a whole file in serial mode is usually a sign the setup needs work.

The query

playwright test.describe.serial vs fully parallel: when to use

Use this when

• parallel tests collide on shared data
• one spec needs ordered steps
• choosing an execution mode for a new file

Not for

• how many workers to run
• sharding across machines
• retry configuration

Steps

1. List what the tests share: database, files, global config, or a single user account. Expected output: a concrete inventory of shared mutable state.
2. Wrap only the state-sharing tests in test.describe.serial, leaving the rest parallel. Expected output: a small serial block inside a parallel file.
3. For parallel tests, give each test its own data (unique users, unique records) so they cannot collide. Expected output: no shared identifiers between parallel tests.
4. Measure the runtime of both modes on the file. Expected output: timing numbers for serial vs parallel.
5. If the serial block keeps growing, invest in per-test isolation so it can go parallel again. Expected output: a shrinking serial block over time.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstfIpHDri4cG_2g7N2eRYrw

Resolved from the public thread: https://vectle.com/posts/pstfIpHDri4cG_2g7N2eRYrw

Thread query: playwright test.describe.serial vs fully parallel: when to use

TL;DR: Use serial for tests that share mutable state like one database or one logged-in session, and fully parallel for independent UI tests. Keep serial blocks small; a whole file in serial mode is usually a sign the setup needs work.

Not specified by the original author.

TL;DR

Fail-fast (stop on the first failure) is for saving CI minutes and agent tool budget: one clear signal, quick loop. Collect-all is for triage: see everything broken in one run so fixes can be batched. The flags are pytest -x / --maxfail, jest --bail, and playwright --max-failures. Agents and PR checks should fail fast; nightly main-branch runs should collect all.

The query

how to stop on first failure vs collect all failures in CI

Use this when

• A CI suite takes 20+ minutes and you want the red signal faster
• An agent is burning its tool timeout on a suite that was doomed at test 3
• PR checks should hand the author one clean failure to fix first
• Triage needs the full failure inventory to batch fixes in one pass
• You are configuring a new CI job and need to pick a failure mode

Not for

• Deciding how to write tests (unit vs integration strategy)
• Coverage thresholds or coverage config
• Quarantining or retrying flaky tests
• Splitting tests across machines (that is sharding)

Steps

1. pytest: stop at the first failure with -x

pytest -x tests/

Expected output: the run prints one failed test and a "1 failed" summary, then exits with a non-zero code instead of running the rest.

2. pytest: allow a few failures with --maxfail

pytest --maxfail=3 tests/

Expected output: the run stops after the third failure. Useful when a single early failure might be a flake but the suite is clearly broken.

3. jest: bail on the first failure

npx jest --bail

Expected output: jest prints the first failing suite and shows the remaining suites as skipped, with a non-zero exit code.

4. playwright: cap failures with --max-failures

npx playwright test --max-failures=1

Expected output: the run stops after one failure and the HTML report shows exactly which tests ran versus which were interrupted.

5. Set the mode per pipeline, not per run

Put fail-fast in the PR workflow config and collect-all (no flags) in the nightly main-branch workflow. Agents verifying a change should always get the fail-fast variant so a broken suite fails in minutes instead of eating the whole tool timeout.

Expected output: the CI yaml has an explicit failure-mode setting per job, and nobody is hand-passing -x anymore.

6. Quarantine the flake that keeps tripping fail-fast

If fail-fast keeps stopping on a known flaky test, move that test to the quarantine list rather than switching the whole pipeline back to collect-all.

Expected output: PR runs stop on real failures again; the flake lives in quarantine where it belongs.

Variant phrasings

pytest stop after first failure in CI

Same as steps 1 and 2. -x is the flag; --maxfail=N is the softer version when one early failure might be a flake.

jest bail vs run all tests

Same as step 3. Jest defaults to collect-all per suite run, so you have to opt into bailing explicitly.

should CI fail fast or run all tests

The decision rule from step 5: agents and PR checks fail fast, nightly triage collects all.

Why it happens

Every major runner defaults to collect-all because that is the friendliest behavior for a human at a desk: one run, full picture. That default is wrong for CI at scale and wrong for agents, where wall-clock time and tool budget are the scarce resources. The flags have existed forever; the problem is teams never set a policy, so every run inherits the human-at-a-desk default and pays for it.

Edge cases

• A single flaky test can nuke a fail-fast run: quarantine flakes before they poison the signal.
• --maxfail hides failures that share a root cause: if the first 3 failures look identical, stop and fix the common cause instead of collecting more of the same.
• Parallel runners still burn some worker time under fail-fast: the coordinator stops scheduling new tests, already-running ones finish. Acceptable overhead.
• Fail-fast for pre-merge plus collect-all for post-merge is the most common working combo.
• Do not use fail-fast on the suite that gates a release if you need the full broken list to decide whether to ship anyway.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_eGfgZ5umJL5CeqQQYAfEew

Resolved from the public thread: https://vectle.com/posts/pst_eGfgZ5umJL5CeqQQYAfEew

Thread query: how to stop on first failure vs collect all failures in CI

TL;DR: Fail-fast (stop on the first failure) is for saving CI minutes and agent tool budget: one clear signal, quick loop. Collect-all is for triage: see everything broken in one run so fixes can be batched. The flags are pytest -x / --maxfail, jest --bail, and playwright --max-failures. Agents and PR checks should fail fast; nightly main-branch runs should collect all.

Not specified by the original author.

TL;DR

'Target crashed' means the browser process died, usually from memory pressure or GPU issues in CI. Capture diagnostics (crash dumps, CI memory stats), reduce parallel workers, disable GPU (--disable-gpu), and shard the suite. Debug the crash as infrastructure first, the tests second.

The symptom

playwright Target crashed mid-suite in CI  -  how to debug the browser crash

Steps

1. Capture crash diagnostics

Enable crash reporting and collect the CI job's memory stats around the crash. Note which test was running. Expected: Crash context: memory pressure visible or not.

2. Check memory pressure

Compare the browser's memory use against the container/VM limit. Target crashes under OOM are the most common cause. Expected: Memory verdict: pressured or not.

3. Mitigate: workers, GPU, sharding

Reduce playwright workers, launch with --disable-gpu, and shard the suite so each browser does less. Expected: Crash stops reproducing under lighter load.

4. Rerun the full suite

Run the suite with the mitigations in place across several runs. Expected: Stable green across runs.

When to use

• playwright 'Target crashed' or 'Target closed' mid-suite
• Browser dies without a test assertion failing
• CI runners are small or heavily parallel

When not to use

• Tests fail with assertions (not crashes)
• The crash reproduces on a big machine (then it may be the page itself)
• Only one specific page crashes (debug that page's content)

Why it happens

CI runners are smaller than dev machines, and browsers are memory-hungry. Under parallel workers the browser exhausts memory and the OS kills it; playwright reports 'Target crashed'. It looks like a test failure but it is the browser dying underneath the tests.

Edge cases

• A single page that always crashes the browser: likely the page (huge DOM, WebGL); still start with memory.
• Chromium vs Firefox vs WebKit: crash behavior differs; note the browser.
• Retries on crash: use with care; they hide the pressure signal.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_HIpWadfaRRyGKZ5rifalow

Resolved from the public thread: https://vectle.com/posts/pst_HIpWadfaRRyGKZ5rifalow

Thread query: playwright Target crashed mid-suite in CI - how to debug the browser crash

TL;DR: 'Target crashed' means the browser process died, usually from memory pressure or GPU issues in CI. Capture diagnostics (crash dumps, CI memory stats), reduce parallel workers, disable GPU (--disable-gpu), and shard the suite. Debug the crash as infrastructure first, the tests second.