screenshot diff flakiness from font rendering: how to fix

TL;DR

Font flakiness comes from different font files or different rasterizers on each machine, not from your app changing. Install the exact same font files in CI that developers have locally, force a fixed fallback font through fontconfig, and disable subpixel-rendering differences. If a page has real dynamic text, mask that region instead of loosening the diff threshold.

The query

screenshot diff flakiness from font rendering: how to fix

Use this when

• Screenshot diffs fail only on text edges, letter spacing, or font weight
• The same spec passes on a Mac and fails in Linux CI, or vice versa
• Diffs appear and disappear between runs with no app change

Not for

• Layout or spacing shifts (that is a CSS problem, not a font problem)
• Flickering from animations, video, or carousels
• Text that legitimately changes every run, like timestamps or ads

Steps

1. Confirm the diff is font-only. Open the failing diff image and zoom into the changed pixels. If only glyph edges differ, it is a font rasterization issue.

Expected output: a screenshot of the diff showing pixel differences limited to character edges.

1. List installed fonts on the machine that passes and the machine that fails. Compare versions, not just family names.

Expected output: two font lists where the CI machine is missing a font or has a different version.

1. Pin the font files. Check the exact font files (TTF/OTF) into your repo or fetch a fixed version in CI setup, and install them before tests run.

Expected output: a CI step that installs the same font files on every run, verified by listing the font directory.

1. Force a fallback font with fontconfig on Linux CI. Add a config that maps every missing family to one fixed font so the OS never silently substitutes something else.

Expected output: a fontconfig file in the CI image and a log line showing the fallback resolving to the expected family.

1. Neutralize rendering differences. Set the same text anti-aliasing mode everywhere and fix the device scale factor (usually 1x) in the browser launch flags.

Expected output: browser launch config with fixed scale factor and rendering flags, applied on all machines.

1. Mask genuinely dynamic text. For regions with dates, usernames, or counters, add a mask to the screenshot call instead of the whole page.

Expected output: the diff passes with only the stable regions compared, and the masked regions documented in the spec.

1. Re-baseline once and lock it. After the fonts are pinned, regenerate the baseline screenshots from CI, not from a laptop.

Expected output: a fresh baseline committed from a CI run, with the suite green on the next three runs.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_78XjotSlKejOY-7MH-LgKQ