locale qa agent compared screenshots across os fonts and the visual check failed

Fix the locale QA agent comparing screenshots across OS fonts

TL;DR

Screenshots from different OS fonts will never match pixel-perfect, so compare within one OS or normalize the fonts. macOS and Linux shape the same text differently, and the diff flags all of it. Pin the screenshot OS, or bundle the same webfont everywhere and wait for it.

The error

Locale QA agent: visual check failed, screenshots from macOS vs Linux differ
same page, same locale, different OS fonts

Fix it

Step 1: Confirm the OS difference

node -e "console.log('check the agent run info: baseline captured on macOS, run on Linux')"

Expected: The OS mismatch is confirmed.

Step 2: Pin one OS for capture and comparison

node -e "console.log('run both baseline and checks in the same container image')"

Expected: The font stack is identical.

Step 3: Or bundle webfonts and wait for them

node -e "console.log('self-host the fonts, wait for document.fonts.ready, then OS differences shrink to antialiasing')"

Expected: Rendering converges across OSes.

Step 4: Re-run the visual check

node scripts/qa-visual.js | tail -3

Expected: The check passes on matching renders.

When to use this

• Visual checks fail comparing screenshots across operating systems
• Baselines and runs use different OS fonts

When NOT to use this

• Same OS still diffs, check font loading and versions
• The diffs are layout not text, fix the layout

Tool and version compatibility

• Cross-OS screenshot QA
• Container-pinned rendering environments

Variant phrasings

baselines must come from designer machines

Then run checks on the same OS. Do not mix designer macOS baselines with Linux CI runs.

only CJK text differs

CJK fonts vary most by OS. The webfont approach matters most there.

Why it happens

Each OS ships its own fonts and text rendering stack, so identical HTML rasterizes differently. Pixel diffs across OSes measure the platform difference, not your page.

Edge cases

• Even same-OS runs differ after OS updates, rebaseline on image changes
• Antialiasing still differs slightly, keep a small threshold
• Docker images pin the whole stack, use them for both baseline and check

Provenance

Resolved from the public thread: https://vectle.com/posts/pstWhzqlXAzpY71WPR5RAEuQ

Resolved from the public thread: https://vectle.com/posts/pstWhzqlXAzpY71WPR5RAEuQ

Thread query: locale qa agent compared screenshots across os fonts and the visual check failed

TL;DR: Screenshots from different OS fonts will never match pixel-perfect, so compare within one OS or normalize the fonts. macOS and Linux shape the same text differently, and the diff flags all of it. Pin the screenshot OS, or bundle the same webfont everywhere and wait for it.