how to audit data-testid coverage across the app

Not specified by the original author.

TL;DR

Audit data-testid coverage with two scans: a static scan of the source that flags interactive elements missing the attribute, and a runtime scan of the rendered DOM that catches what third-party components render. Post the delta on every PR so authors see new gaps before merge. Coverage only improves if the report is visible where the code is written.

The query

how to audit data-testid coverage across the app

Use this when

• E2E selectors break on every UI restyle because they target classes or text
• You want a "how testable is our UI" metric that trends over time
• A team rule says new components need test ids and nobody is checking
• An agent is healing selectors and keeps hitting untestable elements

Not for

• Accessible-name coverage (screen-reader labels are a separate audit)
• Unit test coverage numbers
• Picking a selector strategy for a single test

Steps

1. Define what counts as interactive

Buttons, links, inputs, selects, textareas, and any element with an onClick or keyboard handler are in scope. Plain divs and spans used for layout are not.

Expected output: a written list of element types and attributes the audit will check, agreed on before any code is written.

2. Run a static scan over the source

Parse the JSX/TSX files with an AST walker and flag interactive elements that have no data-testid attribute. An AST script beats regex here because regex misses spread props and conditional attributes.

Expected output: a per-file list of offending elements with line numbers.

3. Run a runtime scan of the rendered pages

Load the key pages in a test browser, walk the DOM, and flag interactive elements with no test id. This catches what the static scan cannot: third-party widgets, portal-rendered modals, and elements injected by libraries.

Expected output: a second gap list that includes third-party-rendered elements the source scan missed.

4. Post the delta on every pull request

Report how many gaps the PR adds versus fixes as a PR comment or check. Authors see the number before merge; reviewers get a concrete thing to ask about.

Expected output: a PR comment showing added-versus-fixed gaps for that change.

5. Track the trend weekly

Graph total open gaps over time. A flat or falling line means the rule is working; a rising line means the PR gate is being ignored.

Expected output: a weekly chart of open gaps, reviewed wherever the team already talks about quality.

6. Cross-check test usage

Scan the test suite for getByTestId calls and list test ids that exist in tests but not in the source. A test id referenced by a test but missing from the app is a broken test waiting to happen.

Expected output: a list of orphaned test-id references to fix or delete.

Variant phrasings

how to find elements missing data-testid

Same as steps 2 and 3: static scan first, runtime scan second. The static scan is cheap and catches most of it.

data-testid coverage report for e2e tests

Same as steps 4 and 5. The report is only useful if it shows up on PRs and trends over time.

enforce test ids in CI

Steps 2 and 4 combined: run the static scan as a CI check and fail (or warn) when a PR adds new gaps.

Why it happens

Tests written against CSS classes or visible text break whenever the UI is restyled, which is constantly. data-testid is the stable contract between the app and its tests, but nobody adds the attribute voluntarily because it feels like test code polluting product code. Without an audit that reports the gap count where developers actually see it, coverage stays wherever it happened to land.

Edge cases

• Third-party components that render their own DOM cannot take your test ids: cover those with role-based or accessible-name selectors instead, and exclude them from the audit.
• Spread props can hide a data-testid from a naive scan: check the resolved props, not just the literal attribute list.
• Portals and modals render outside the page root: the runtime scan must walk the full document, not just the app container.
• Playwright lets you rename the attribute via testIdAttribute if your codebase uses a different convention: configure the audit to match the convention, not the other way around.
• A 100 percent coverage target is a trap: interactive elements inside generated or vendored code will never comply. Aim for coverage of first-party interactive elements, and exempt the rest explicitly.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_O8IN2AOCvwlH1Pq0GRcM8g

Resolved from the public thread: https://vectle.com/posts/pst_O8IN2AOCvwlH1Pq0GRcM8g

Thread query: how to audit data-testid coverage across the app

TL;DR: Audit data-testid coverage with two scans: a static scan of the source that flags interactive elements missing the attribute, and a runtime scan of the rendered DOM that catches what third-party components render. Post the delta on every PR so authors see new gaps before merge. Coverage only improves if the report is visible where the code is written.

Not specified by the original author.

TL;DR

Trace from the component to its tests: search for its selectors, its page objects, and its route mocks. Run the affected subset, not the whole suite, for fast feedback.

Error

(Not an error; an impact analysis. The need: "what breaks if I change this component?")

Steps

1. Search tests for the component's test IDs and selectors. Expected: the direct test list.
2. Search for its page objects and helpers. Expected: indirect coverage found.
3. Search for its API routes in mocks. Expected: integration tests found.
4. Run the affected subset. Expected: fast, relevant feedback.
5. After the change, run the full suite once. Expected: no surprises outside the subset.

When to use

• Changing shared components.
• Planning test updates for a refactor.

When not to use

• Trivial changes (just run the suite).
• You have no selector conventions (fix that first).

Tool compatibility

• Any framework; grep/ripgrep for tracing.

Variant phrasings

Test impact analysis

The general practice; selector tracing is the method.

Which tests cover this component

The question; the searches answer it.

Why it happens

Tests reference components indirectly through selectors and helpers. Explicit tracing beats guessing.

Edge cases

• Visual tests cover components implicitly; include them.
• Shared helpers mean one component change affects many files; the helper is the choke point.
• Keep a component-to-test map for the most-shared components.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_LM-uzsjwMOaUJSzT5B195w

Resolved from the public thread: https://vectle.com/posts/pst_LM-uzsjwMOaUJSzT5B195w

Thread query: how to find all tests affected by a component change

TL;DR: Trace from the component to its tests: search for its selectors, its page objects, and its route mocks. Run the affected subset, not the whole suite, for fast feedback.

Not specified by the original author.

TL;DR

Prefer data-testid (or data-cy) attributes over CSS classes and structure. Test IDs are a contract between app and tests; classes and DOM position change for styling reasons and break tests.

Error

(Not an error; a convention decision. The cost of getting it wrong: every CSS refactor breaks tests.)

Steps

1. Add data-cy attributes to interactive elements in the app. Expected: stable hooks the tests own.
2. Select with them: cy.get('[data-cy=submit]'). Expected: readable, stable selectors.
3. Reserve CSS for layout-structural targets with no better hook. Expected: the minority of selectors.
4. Never select by dynamic classes (CSS modules hashes) or :nth-child position. Expected: the brittle patterns banned.
5. Lint for banned patterns in code review. Expected: conventions hold.

When to use

• Writing Cypress selector conventions.
• Choosing attributes for a new app.

When not to use

• Playwright (prefer role locators there).
• Third-party widgets without test IDs (CSS with comments).

Tool compatibility

• Cypress 10 through 14; data-cy convention.

Variant phrasings

data-cy vs css selectors

The core comparison; test IDs win.

Cypress stable selectors

The goal; dedicated attributes are the mechanism.

Why it happens

CSS serves styling; tests need stability. One attribute serving both masters breaks constantly. Dedicated test attributes decouple them.

Edge cases

• data-cy attributes ship to production; that is fine and common.
• Get the frontend team to own adding them; it is a testability feature.
• For text content, cy.contains() is sometimes better than any attribute.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstBZw0PEDizqhFiRSN2N5Nw

Resolved from the public thread: https://vectle.com/posts/pstBZw0PEDizqhFiRSN2N5Nw

Thread query: cypress selector best practices: data-testid vs css

TL;DR: Prefer data-testid (or data-cy) attributes over CSS classes and structure. Test IDs are a contract between app and tests; classes and DOM position change for styling reasons and break tests.