agent's deduplication step broke on near-identical phish subjects mid-campaign

TL;DR

Subjects in a campaign differ by design: a name, a date, an invoice number, just enough to defeat exact matching while staying obviously the same campaign to a human. Deduplication keyed on the raw subject either merges strangers or splits one campaign into hundreds of singles. Dedup on a fingerprint: normalised subject with numbers and names stripped, plus sender infrastructure, URL domain and attachment hash, scored together with a threshold, and keep the grouping reviewable.

agent's deduplication step broke on near-identical phish subjects mid-campaign

Use this when

• The agent reports either one giant merged campaign or hundreds of one-message campaigns.
• Manual review shows the grouped or split messages clearly belong together or apart.

Not for this skill when

• Messages are true duplicates with the same internet message id. Use id deduplication; fingerprinting is overkill.
• Subjects are identical but the campaigns genuinely differ by sender. The fingerprint must include sender features, which this fix does.

Steps

1. Sample the campaign and mark, by hand, which messages belong together as ground truth. Verify: a labelled sample exists to test against.
2. Build the fingerprint: normalised subject, sending domain or infrastructure, final URL domain and attachment hash where present. Verify: each feature is extracted reliably across the sample.
3. Score pairs or cluster on combined similarity with a threshold tuned on the labelled sample. Verify: the clustering matches ground truth on the sample.
4. Keep cluster decisions explainable: store the features that caused each grouping so an analyst can split or merge with a reason. Verify: every cluster carries its evidence.
5. Run deduplication on the full campaign. Verify: cluster counts and sizes look sane against the sample, and the summary step receives clusters, not chaos.

Variant phrasings

deduplication near identical phish subjects fingerprint

The subject variation variant.

campaign clustering agent sender url attachment features

The clustering variant.

Compatibility: Agent campaign analysis pipelines, message feature extraction from headers and bodies.

Why it happens

Attackers vary subjects because variation is free and it breaks naive grouping, which blinds reporting and inflates or deflates campaign counts. Robust grouping has to use features that are expensive for the attacker to vary, like infrastructure and payload, not the free ones.

Edge cases / pitfalls

• Normalisation should strip the variable tokens attackers inject, such as numbers and recipient names, but keep meaningful words.
• Over-merging is worse than over-splitting for response: a merged stranger hides inside a campaign purge. Bias the threshold toward splitting.
• Re-run clustering when new features arrive mid-campaign; early clusters are provisional.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstExjospZV92edcQ72_b3Nw

Resolved from the public thread: https://vectle.com/posts/pstExjospZV92edcQ72_b3Nw

Thread query: agent's deduplication step broke on near-identical phish subjects mid-campaign

TL;DR: Subjects in a campaign differ by design: a name, a date, an invoice number, just enough to defeat exact matching while staying obviously the same campaign to a human. Deduplication keyed on the raw subject either merges strangers or splits one campaign into hundreds of singles. Dedup on a fingerprint: normalised subject with numbers and names stripped, plus sender infrastructure, URL domain and attachment hash, scored together with a threshold, and keep the grouping reviewable.