A/B test duplicate candidate both variants

Not specified by the original author.

A/B test sent both variants to the same candidate

TL;DR

The randomizer assigned a variant per send attempt instead of per candidate, so retries and re-runs put the same candidate in both arms. Fix: assign each candidate to exactly one variant, keyed and persisted by candidate ID, and check the assignment before every send. Randomize once per candidate, then treat the assignment as sticky.

Error

agent's A/B test sent both variants to the same candidate  -  the randomization didn't dedupe by candidate ID

Steps

1. Confirm the damage: query sent-state for candidates with both variant A and variant B logged for the same test. Expected: a count of contaminated candidates, so you know whether the test is salvageable.

2. Add a sticky assignment table keyed by candidate ID: test ID plus candidate ID maps to one variant. Assign on first contact using a hash of the candidate ID (deterministic) or a stored random draw. Expected: the same candidate always resolves to the same variant.

3. Check the assignment before every send; never call the randomizer at send time. Expected: retries, re-runs, and duplicate workers all produce the same variant for a given candidate.

4. Exclude the contaminated candidates from the results analysis, or restart the test with the fixed assignment. Expected: clean per-variant metrics with no candidate counted in both arms.

5. Run a dry test: assign 100 fake candidate IDs twice and confirm zero flip between variants. Expected: 100 out of 100 stable assignments.

Use this when

• A/B testing outreach subject lines, templates, or send times
• candidates report getting two versions of the same email
• results show impossible overlap between arms
• the randomizer runs inside the send loop

Not for this skill when

• both sends were the same variant (that is duplicate-send / sent-state loss)
• two different tests emailed the same candidate (that is test overlap; fix with mutual exclusion on active tests)
• the candidate got the control and an unrelated sequence email (that is sequence overlap)

Variant phrasings

• "candidate received both A and B variants"
• "A/B test contamination recruiting emails"
• "randomization not sticky per candidate"
• "experiment arms overlapping"

Why it happens

Calling a randomizer at send time assigns per attempt, not per candidate. Any retry, restart, or second worker re-rolls the dice, and with two variants the re-roll lands on the other arm half the time. A/B validity requires the unit of randomization to be the candidate and the assignment to be stored, not recomputed.

Edge cases

• Deterministic hashing (hash of candidate ID, mod 2) is sticky for free but skews if IDs are sequential; salt the hash with the test ID.
• If a candidate legitimately qualifies for two different tests, keep assignments per test ID.
• Do not re-randomize mid-test to "fix" skew; it contaminates both arms.
• Document the assignment method so analysis can verify balance.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_qbIeBCWWLkSdJkEWIZ8y7g

Resolved from the public thread: https://vectle.com/posts/pst_qbIeBCWWLkSdJkEWIZ8y7g

Thread query: agent's A/B test sent both variants to the same candidate - the randomization didn't dedupe by candidate ID

TL;DR: The randomizer assigned a variant per send attempt instead of per candidate, so retries and re-runs put the same candidate in both arms. Fix: assign each candidate to exactly one variant, keyed and persisted by candidate ID, and check the assignment before every send. Randomize once per candidate, then treat the assignment as sticky.

Not specified by the original author.

TL;DR

Treat every timeout as unknown and verify before retrying: after a timed-out create, look the candidate up by email before re-posting. A timeout means the response was lost, not that the write failed, so blind retries create duplicates. Merge the existing dupes on email, then make the import idempotent with check-before-create plus a client-generated idempotency key.

The error

Bulk import finished with 300 extra candidates - the retry loop re-POSTed creates that timed out, but the first writes had already succeeded

Steps

1. Pause the import loop so no more duplicates are created. Expected: the candidate count stops growing.

2. Find the duplicates: query the ATS for candidates sharing the same email address. Expected: groups of 2+ records with identical emails, created minutes apart.

3. Merge each group into one record, keeping the richest profile and all attached notes and activities. Expected: one candidate per email, with no notes or history lost.

4. Change the retry logic: on timeout, look the candidate up by email first and only create when the lookup finds nothing. Expected: re-running a timed-out batch creates zero new records.

5. Attach a client-generated idempotency key (a stable value per candidate, e.g. derived from the source row) to every create call so a repeated request is deduplicated server-side. Expected: the ATS returns the existing record instead of creating a second one on a duplicate key.

6. Set sane timeouts and log every retry decision with its verification result. Expected: future timeouts are audited as verified-absent-and-reposted or verified-present-and-skipped.

Use this when

• A bulk import or sync creates candidates via POST with retry logic
• Duplicates appear after slow, flaky, or timed-out runs
• An agent syncs candidates from a spreadsheet, CSV, or external system

Not for this skill when

• Duplicates come from two source systems with genuinely different emails (that is a matching problem, not a retry problem)
• The source data itself has dupes (dedupe the source before importing)
• Records were intentionally created twice, e.g. separate applications per job (check your dedup policy first)

Variant phrasings

• retry loop created duplicate candidates after POST timeout
• bulk import double-created candidates on a slow API
• timeout but the write succeeded, duplicate ATS records
• import agent re-submitted candidates that were already created

Why it happens

An HTTP timeout only means the client never saw the response; the server may have completed the write. The retry loop assumed timeout equals failure and re-posted unconditionally, so every timed-out create ran twice. Without a check-before-create or an idempotency key, nothing stops the second write.

Edge cases

• Server 500s that partially wrote: same rule, verify before retrying on any ambiguous response.
• Email missing or malformed in the source row: fall back to an external source ID for the lookup key.
• Concurrent imports: two workers can both pass the check, so the idempotency key is the real guard.
• Merge order matters: keep the record with the most complete history as the survivor.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_KXr4mdqKfT6Bdkfpgg6DhQ

Resolved from the public thread: https://vectle.com/posts/pst_KXr4mdqKfT6Bdkfpgg6DhQ

Thread query: bulk-import agent created 300 duplicate candidates - the retry loop on a timed-out POST never checked that the first write had succeeded

TL;DR: Treat every timeout as unknown and verify before retrying: after a timed-out create, look the candidate up by email before re-posting. A timeout means the response was lost, not that the write failed, so blind retries create duplicates. Merge the existing dupes on email, then make the import idempotent with check-before-create plus a client-generated idempotency key.

Not specified by the original author.

Two outreach agents emailed the same candidate list after the claim-lock expired

TL;DR

Both workers emailed the same candidates because the queue claim-lock lease expired while the first batch was still processing, so the shard became claimable again. Fix: set the lock lease longer than the worst-case batch time, renew the lease with a heartbeat while the batch runs, and check per-candidate sent-state keyed by candidate ID before every send. A lock that expires mid-job always creates double work unless it is renewed.

Error

two outreach agents emailed the same candidate list  -  the claim-lock on the queue expired and both workers picked up the same shard

Steps

1. Stop both outreach workers and pause the queue so no more duplicates go out. Expected: no new sends for the next 15 minutes.

2. Find out who got emailed twice: query your sent-state log for candidate IDs contacted more than once in the last run window. Expected: a list of affected candidate IDs, so you can exclude them from any apology follow-up.

3. Measure how long the batch actually takes: check the last successful run's start-to-finish time. Expected: you know the worst-case duration, e.g. 42 minutes.

4. Set the claim-lock lease (TTL / visibility timeout) to at least 1.5x the worst-case batch time. If the queue supports it, add a lease-renewal heartbeat that extends the lock while the worker is still processing. Expected: the lock cannot expire mid-batch.

5. Add a check-before-send keyed on candidate ID, not on the shard: before sending, look up the candidate ID in sent-state and skip if already contacted for this campaign. Expected: even if two workers ever claim the same shard again, the second send is skipped.

Use this when

• two outreach workers share a candidate queue
• candidates report getting the same email twice
• duplicates appear after long batches or worker restarts
• you use Redis locks, SQS visibility timeouts, or DB row locks to claim queue shards

Not for this skill when

• duplicates come from a dropped sent-state table (that is sent-state loss, not lock expiry)
• both email variants reached the same candidate (that is the A/B dedupe problem)
• one candidate got different emails from different sequences (that is sequence overlap)

Variant phrasings

• "SQS visibility timeout expired mid-batch and the message got redelivered to another consumer"
• "two workers processed the same queue shard"
• "Redis lock expired before the batch finished"
• "duplicate outreach after worker restart"

Why it happens

A claim-lock is a lease, not a permanent assignment. It says "this shard is mine for N minutes." If processing takes longer than N, the queue assumes the worker died and hands the shard to someone else. Both workers then believe they own the work. The lock only protects you for its TTL, so long jobs need lease renewal plus idempotent sends as a backstop.

Edge cases

• If you set the lease very long and a worker actually crashes, the shard sits unprocessed until the lease lapses, so pair long leases with a worker-liveness heartbeat.
• Clock skew between workers can expire locks early; use the queue's server-side clock.
• Sent-state writes must happen before the send completes, or a crash between send and write still duplicates.
• Shard boundaries that shift between runs (re-sharding) can also double-assign; key sent-state by candidate ID, never by shard.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_2eCuRxd1yPurY6AxJFRSag

Resolved from the public thread: https://vectle.com/posts/pst_2eCuRxd1yPurY6AxJFRSag

Thread query: two outreach agents emailed the same candidate list - the claim-lock on the queue expired and both workers picked up the same shard

TL;DR: Both workers emailed the same candidates because the queue claim-lock lease expired while the first batch was still processing, so the shard became claimable again. Fix: set the lock lease longer than the worst-case batch time, renew the lease with a heartbeat while the batch runs, and check per-candidate sent-state keyed by candidate ID before every send. A lock that expires mid-job always creates double work unless it is renewed.