outreach agent emailed the same candidate twice in one day - its sent-state file got wiped on a container restart and...

TL;DR

Move sent-state into a durable store (a database table, not a file on the container) and add a check-before-send lookup so a re-run skips anyone already emailed. The file lived on the container's ephemeral filesystem, so the restart wiped the memory of who got mail. Durable state plus a dedupe check makes restarts harmless.

The error

Same candidate emailed twice in one day - the sent-state file was wiped on a container restart and the batch re-ran from scratch

Steps

1. Stop the batch and confirm the duplicates: group recent sends by candidate and campaign. Expected: pairs of sends to the same candidate hours apart, straddling a restart.

2. Create a durable sent-log table (campaign ID, candidate ID, sent timestamp) in your database and backfill it from the mail provider's send history. Expected: the log reflects every send, including the duplicates.

3. Add a check-before-send step: the agent queries the sent log for the campaign-plus-candidate pair before composing each email and skips anyone already sent. Expected: a re-run of the batch sends zero emails.

4. Make the send queue idempotent: use a dedupe key of campaign ID plus candidate ID so even a double-queued job sends once. Expected: duplicate queue entries collapse to a single send.

5. Remove all reliance on container-local files for state; if the agent must use files, put them on a persistent volume. Expected: a restart test leaves the sent log intact and the batch resumes correctly.

Use this when

• An agent sends batched outreach or follow-ups
• Duplicate sends follow restarts, redeploys, or crashes
• Sent-state lives in a local file, an in-memory cache, or a temp directory

Not for this skill when

• Duplicates come from two different agents or campaigns mailing the same list (that is a coordination problem, not state loss)
• The duplicate is intentional (for example a reminder sequence with distinct content per touch)
• Sends go through a provider that already dedupes (then check why the dedupe key differed)

Variant phrasings

• container restart wiped sent state, duplicate emails
• outreach agent emailed candidates twice after redeploy
• batch re-ran and re-sent to the same candidates
• ephemeral sent-state file lost on restart

Why it happens

Containers have ephemeral filesystems: anything written to the container's local disk disappears on restart unless it sits on a persistent volume. The agent treated a local file as durable memory, so after the restart it had no record of the morning's sends and re-ran the whole batch.

Edge cases

• Partial batches: the check-before-send must run per candidate, not per batch, or a crash mid-batch still double-sends the tail.
• Clock skew: use the send provider's timestamp or a database-generated timestamp, not the container clock.
• Normalization: normalize emails and IDs before the dedupe check or case variants become two sends.
• Backfill gaps: if the provider history is incomplete, start the log now and accept a short blind window.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_W0ykVMhH7HtvSNN8kpC-EA