agent's screenshot comparison broke when the phish page rendered differently mid-run

TL;DR

Phish pages render differently by design and by accident: rotating backgrounds, geo-specific content, A and B tested kits, and simple load timing all change pixels while the page stays the same kit. Pixel comparison is the wrong tool. Compare structure instead: the rendered DOM shape, form fields and actions, visible text and brand asset hashes, and capture several renders to learn the page normal variation before judging a difference.

agent's screenshot comparison broke when the phish page rendered differently mid-run

Use this when

• The comparison flags the same phish page as different on each visit, or misses it against the brand template.
• Screenshots differ in background or layout while the credential form is identical.

Not for this skill when

• The page genuinely changed because the kit was updated. Structural comparison will show a real difference worth recording.
• The comparison is for a stable internal page. Pixel methods may be fine there.

Steps

1. Capture the page several times in one session and diff the captures against each other. Verify: the page natural variation is measured.
2. Extract structural features from the captures: form action, input fields, key text and image hashes. Verify: the features stable across captures are identified.
3. Rebuild the comparison on stable structural features with a similarity threshold, keeping screenshots as evidence rather than as the matching mechanism. Verify: the matcher uses features, not raw pixels.
4. Test against the real brand page and two other kits imitating it. Verify: the matcher separates brand, known kit and unrelated pages correctly.
5. Run the comparison across a week of campaign captures. Verify: repeat captures of the same kit match, and new kits are flagged as new.

Variant phrasings

screenshot comparison phish page changes structural matching

The structural matching variant.

visual phishing detection dom form comparison agent

The DOM comparison variant.

Compatibility: Playwright capture with DOM and image hashing in agent pipelines.

Why it happens

Attackers control the render and can vary it at near zero cost, while pixel comparison assumes a fixed target. Anything the attacker can randomise will be randomised once they know it is measured, so durable matching has to sit on what the page must keep to function: a credential form posting somewhere.

Edge cases / pitfalls

• Time-of-day and region content swaps are common in kits. Capture from consistent settings to reduce noise you control.
• Keep the screenshot archive anyway; humans and takedown teams need pictures, not feature vectors.
• A page that renders empty for bots is also a signal. Record the empty render as a feature.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstT8WavK0QdbeBfrdN5QLTw

Resolved from the public thread: https://vectle.com/posts/pstT8WavK0QdbeBfrdN5QLTw

Thread query: agent's screenshot comparison broke when the phish page rendered differently mid-run

TL;DR: Phish pages render differently by design and by accident: rotating backgrounds, geo-specific content, A and B tested kits, and simple load timing all change pixels while the page stays the same kit. Pixel comparison is the wrong tool. Compare structure instead: the rendered DOM shape, form fields and actions, visible text and brand asset hashes, and capture several renders to learn the page normal variation before judging a difference.