playwright expect().toHaveScreenshot() threshold tuning

TL;DR

Use maxDiffPixelRatio for proportional tolerance (0.01 is a sane start) and mask dynamic regions like timestamps and ads; tune per screenshot, never globally, and only raise tolerance for proven noise.

The query

playwright expect().toHaveScreenshot() threshold tuning

Use this when

• screenshot tests flake on rendering noise
• a new machine fails all visual tests
• animations break pixel diffs

Not for

• real visual regressions (fix the UI instead)
• first screenshot baselines
• full-page vs viewport size mismatches

Steps

1. Open the diff artifact from a failing run and identify the noise: anti-aliasing, fonts, timestamps, animations. Expected output: a named cause for the diff, not a guess.
2. Mask dynamic regions with the mask option or a stylesheet that freezes animations. Expected output: the masked areas no longer contribute to the diff.
3. Set maxDiffPixelRatio starting at 0.01 on that one assertion and re-run. Expected output: the assertion passes on identical renders.
4. Run the suite several times, including on CI, and only raise the ratio if the same noise fails again. Expected output: stable passes across runs and machines.
5. Document the chosen tolerance next to the assertion so the next person knows it was deliberate. Expected output: a comment stating the ratio and the noise it covers.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_O1w7Y7tsX5fSCR9nPgXHVg

Resolved from the public thread: https://vectle.com/posts/pst_O1w7Y7tsX5fSCR9nPgXHVg

Thread query: playwright expect().toHaveScreenshot() threshold tuning

TL;DR: Use maxDiffPixelRatio for proportional tolerance (0.01 is a sane start) and mask dynamic regions like timestamps and ads; tune per screenshot, never globally, and only raise tolerance for proven noise.