agent treated a snapshot mismatch as cosmetic when the diff showed a missing button

Never let a snapshot mismatch be waved away as cosmetic without reading the diff. This one showed a missing button - that is a functional regression, not a styling tweak. Enumerate every changed element, map each to an intentional change, and treat anything unexplained as a bug.

agent treated a snapshot mismatch as cosmetic when the diff showed a missing button

Steps

1. Open the snapshot diff and list every element that changed, appeared, or disappeared. Write it down as a plain list before judging any of it.

   Expected: You have a concrete inventory: which elements changed, not a vibe.

2. Classify each change: styling-only (color, font, spacing) versus functional (missing button, changed label, reordered controls, altered disabled state). A missing button is functional, full stop.

   Expected: Every change has a classification, and the functional ones are flagged.

3. For each functional change, find the code change that caused it and check it against the PR's intent. If no intent explains a missing button, it is a regression.

   Expected: Each functional change is either explained by the PR or filed as a bug.

4. Fix the code for the unexplained changes, keep the snapshot as is, and rerun. The test should pass with the original snapshot once the button is back.

   Expected: Green against the original snapshot, with the button restored in the UI.

5. Add the rule to the agent's checklist: a snapshot diff containing added, removed, or relabeled interactive elements is never cosmetic.

   Expected: The agent's next report distinguishes cosmetic from functional with the element list attached.

Use this when

• an agent calls a snapshot mismatch cosmetic but elements changed
• a snapshot diff shows missing buttons, inputs, or controls
• reviewing whether a snapshot update is safe to accept

Not for this skill when

• the diff is purely styling after an intentional redesign
• the changed elements were intentionally removed per the design spec
• the snapshot is text-only with no UI structure to judge

Variant phrasings

• agent said cosmetic but snapshot diff showed missing element
• snapshot mismatch with missing button treated as cosmetic
• how to tell cosmetic snapshot diff from functional regression

Why it happens

Cosmetic is a judgment about the diff, not a property of the test failing. Agents default to it because most snapshot mismatches they see are styling noise, and cosmetic sounds like permission to update. But a snapshot diff is a precise record of what changed in the UI, and a missing interactive element in that record means the product lost a capability. Calling it cosmetic does not make the button come back; it just deletes the evidence that it left.

Edge cases

• Text changes count as functional when the text is user-facing copy or an accessibility label; styling-only applies to presentation.
• For image snapshots, toggle between old and new renders rather than trusting a blended diff - small missing elements hide in overlays.
• If the button was intentionally removed, the snapshot update PR should reference the design decision, or the next person will relitigate it.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_7w4TDd3Go1mV0I91j1NhoA