agent's dedup of findings broke: sent 500 duplicate alerts

TL;DR

Duplicates happen because each run generates new alert identities for the same findings. Compute a stable dedup key per finding, check it before alerting, and expire keys deliberately.

Error

agent's dedup of findings broke: sent 500 duplicate alerts

Steps

1. Confirm the 500 alerts map to far fewer unique findings. Expected: the duplication proven.
2. Define a stable dedup key: scanner, rule, path, and fingerprint. Expected: same finding, same key, every run.
3. Check the key against the sent-alert store before sending. Expected: repeats suppressed.
4. Persist the store durably; in-memory dedup resets on restart. Expected: dedup survives crashes.
5. Expire keys when findings are fixed so regressions re-alert. Expected: no permanent silence.

When to use

• Duplicate alert floods from agents.
• Building alert dedup.

When not to use

• Alerts never sent (different problem).
• Intentional re-alerts on escalation.

Tool compatibility

• Alert pipelines; dedup stores.

Variant phrasings

duplicate alerts from scanner

Stable keys.

dedup broke

Persist the store.

Why it happens

Without stable identities, every scan looks like all-new findings to the alerter.

Edge cases

• Fingerprint changes on re-scan break keys; prefer content-based keys.
• Clock-based keys are never stable; avoid them.
• Re-alerts on long-open findings need a separate escalation path.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_YqRnjuuqaCzbRKalje6r8g