chromatic visual diff false positive on CI fonts: how to fix

TL;DR

CI snapshots often capture the page before webfonts finish loading, so Chromatic compares fallback fonts against the baseline's real fonts. Make font loading deterministic: self-host or embed the fonts, wait for document.fonts.ready before the snapshot, and pin the font stack. Diffs then reflect real changes again.

Error

chromatic visual diff false positive on CI fonts: how to fix

Chromatic flags visual diffs where only text glyphs changed; no layout or content differences.

Steps

1. Confirm fonts are the cause: open the diff and check whether only text glyphs changed. Expected output: you see fallback-versus-webfont differences, not layout changes.
2. Ensure fonts load before the snapshot: wait for document.fonts.ready in the story or use Chromatic's delay option. Expected output: snapshots consistently capture loaded fonts.
3. Self-host the fonts or pin their versions so CI and baseline use identical files. Expected output: no more version-drift diffs.
4. Set font-display behavior that suits snapshots (block or swap with a settle wait), and reduce font smoothing differences where possible. Expected output: deterministic glyph rendering.
5. Re-run Chromatic and review the diffs. Expected output: font-only diffs disappear.

When to use

• Chromatic diffs show only font changes.
• Diffs appear on CI runs but not local runs.
• Baselines were captured with different fonts.

When not to use

• The diff shows real layout or content changes (that is a true positive).
• Fonts are intentionally changed (accept the new baseline).
• Diffs are in non-text elements (different cause).

Variant phrasings

Diff only on first CI run

Cold font caches on fresh CI runners; warming the cache or self-hosting removes the first-run diff.

Anti-aliasing diffs across OS

macOS and Linux rasterize fonts differently; pin the CI OS or accept the platform baseline.

Font subsetting

Subsetting changes glyph coverage between builds; pin the subset or the font files.

Why it happens

Visual testing compares pixels, and font rasterization depends on which font file loaded, when it loaded, and the OS doing the rasterizing. CI introduces variance on all three axes, so font diffs are the most common false positive in visual testing.

Edge cases

• Approving a font-diff baseline bakes the fallback font in; future real diffs then compare against the wrong baseline.
• System font stack changes with OS updates; pin the runner image to keep baselines stable.
• Variable fonts can render slightly differently per load; pin the axis values used in snapshots.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst5CupoplApGD0WcP3Yt32Q

Resolved from the public thread: https://vectle.com/posts/pst5CupoplApGD0WcP3Yt32Q

Thread query: chromatic visual diff false positive on CI fonts: how to fix

TL;DR: CI snapshots often capture the page before webfonts finish loading, so Chromatic compares fallback fonts against the baseline's real fonts. Make font loading deterministic: self-host or embed the fonts, wait for document.fonts.ready before the snapshot, and pin the font stack. Diffs then reflect real changes again.