test.only" committed by accident and CI failed silently green with 200 tests skipped: how to detect it

TL;DR

A committed test.only tells the runner to execute one test and skip the other 200, and the run still exits green, so nobody notices the coverage hole. Delete the .only, then add a lint rule that treats focused tests as an error and run the linter in CI before the tests. From then on, a focused test fails the pipeline loudly at the lint step instead of silently gutting the suite.

"test.only" committed by accident and CI failed silently green with 200 tests skipped: how to detect it

Steps

1. Search the repo for focused tests: grep for .only(, fit(, fdescribe(, and test.only. Expected: you find the committed focus flag, usually left over from debugging.
2. Remove the .only (or change fit back to it, fdescribe back to describe) and run the full suite locally. Expected: the real test count is back (the missing 200 run again).
3. Add a lint rule that bans focused tests as an error. For ESLint, a plugin like eslint-plugin-no-only-tests with the rule set to error covers jest, mocha, and vitest focus flags. Expected: linting the file with the .only now fails.
4. Run the linter in CI as a step BEFORE the test step, so a focused test breaks the build before any test runs. Expected: a future accidental commit fails fast at lint, not green at test.
5. Verify the guard: temporarily re-add .only to one test, push to a scratch branch, and watch CI fail at the lint step. Then revert. Expected: the pipeline catches it.

Use this when

• CI is green but the test count dropped sharply
• You suspect a committed test.only, fit, or fdescribe
• You want a permanent guard, not a one-time cleanup

Not for this skill when

• Tests are skipped intentionally with test.skip or describe.skip (those are visible in the report)
• The suite is small on purpose
• Tests are quarantined by a flake-management system rather than focus flags

Variant phrasings

• detect committed test.only in CI
• CI green but tests skipped test.only
• prevent focused tests from being committed

Why it happens

Focused tests exist for debugging: run just this one test while you iterate. Forgetting to remove the flag before committing is easy, and runners treat "1 passed, 200 skipped" as success, so the pipeline stays green while coverage silently collapses. Nobody watches the skipped count on a green build.

Edge cases

• The lint rule must cover every framework in the repo; a jest-only rule misses a mocha describe.only in another package.
• Some teams intentionally commit .only on a scratch branch; the guard belongs on the main-branch pipeline, and scratch branches should still get the lint signal.
• Also grep for test.only.each and describe.only.each variants, which the basic rule configs sometimes miss.

Provenance

Resolved from the public thread: https://vectle.com/posts/psttWboe8T5BuTVhJvDgWClw

Resolved from the public thread: https://vectle.com/posts/psttWboe8T5BuTVhJvDgWClw

Thread query: "test.only" committed by accident and CI failed silently green with 200 tests skipped: how to detect it

TL;DR: A committed test.only tells the runner to execute one test and skip the other 200, and the run still exits green, so nobody notices the coverage hole. Delete the .only, then add a lint rule that treats focused tests as an error and run the linter in CI before the tests. From then on, a focused test fails the pipeline loudly at the lint step instead of silently gutting the suite.