agent's exception queue broke: sent 300 duplicate flags

TL;DR

The queue re-emitted the same flags because nothing marked them as sent. Give every flag a stable idempotency key (report ID plus rule ID), check the key before emitting, and the flood stops: each violation flags exactly once.

Error

agent's exception queue broke: sent 300 duplicate flags

Steps

1. Halt the flood: pause the queue consumer. Expected: no more duplicate flags go out.
2. Define the idempotency key for a flag: report ID plus rule ID plus expense line ID. Expected: a key that is identical for the same finding.
3. Before emitting, check whether the key was already emitted (a sent-set in durable storage). Expected: repeats are dropped.
4. Drain the backlog through the dedupe check, emitting only unseen keys. Expected: each real violation appears exactly once.
5. Clean up the duplicates already sent: notify reviewers which flags were repeats. Expected: reviewers trust the queue again.

When to use

• An exception queue emits duplicate flags at scale.
• Retries or restarts multiply the flags.

When not to use

• The flags are distinct real violations (high volume is correct).
• A single flag duplicated once (check the emit path, smaller fix).

Tool compatibility

• Custom exception queues in audit agents; reviewer dashboards.

Variant phrasings

Duplicate flags flooded reviewers

Same failure, impact-worded.

Exception queue re-emitted flags

Names the re-emission directly.

Why it happens

Queues retry, agents restart, and without idempotency every retry looks like a new flag. The sent-set is the memory that turns retries into safe no-ops.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Q-q2ue2yZACWHl6eeWM9kw