ATS webhook replay dedup merge duplicate

Not specified by the original author.

TL;DR: Dedupe webhook events by event id before processing, and make merges union fields instead of overwriting them. The replay ran the merge a second time, the second run picked the other record as the winner, and its empty notes overwrote the real ones. Idempotent intake plus non-destructive merge fixes both halves.

agent's dedup merge ran on replayed webhooks  -  the same candidate got merged twice and the winning record lost its interview notes

1. Check the webhook processor for event-id deduplication. Expected: none exists; the same event payload processes fully every time it arrives.

2. Store processed event ids and skip repeats before any merge logic runs. Expected: replaying the same webhook becomes a no-op.

3. Change the merge to union fields: keep notes, attachments, and history from both records instead of last-write-wins. Expected: a test merge of two records preserves the interview notes from the loser.

4. Recover the lost notes from the ATS audit log or from a pre-merge snapshot if one was kept. Expected: the interview notes are restored to the surviving record.

5. Add a merge audit trail recording which records merged, when, and which fields won. Expected: every merge leaves a trace a human can review and reverse.

Use this when

• Duplicate or replayed webhooks trigger the same mutation twice.
• Merges overwrite data instead of combining it.
• Records lose notes, attachments, or history after dedup runs.

Not for this skill when

• The duplicates are genuinely different people merged by a bad matching rule (fix the matcher, not the idempotency).
• The data loss came from a manual merge in the ATS UI.
• The webhook fired once and the merge logic itself is wrong.

Variant phrasings

• replayed webhook merged the same candidate twice
• dedup merge overwrote interview notes
• winning record lost its notes after a second merge
• webhook redelivery re-ran the candidate merge

Why it happens

Webhooks redeliver by design: retries, replays, and at-least-once delivery are normal. The merge handler was written as if each event arrives exactly once, so the replay executed the merge logic a second time. The second run re-evaluated the winner from already-merged state, picked the other record, and its blank fields overwrote populated ones. Exactly-once processing has to be built at the intake; the merge itself has to be safe to run twice.

Edge cases

• True duplicates arriving close together that are not replays: dedupe by event id does not catch these. That is the matcher's job; keep the two mechanisms separate.
• Merges of more than two records: union across all of them, and record every source.
• Notes edited during the merge window: a human editing the loser mid-merge can still lose work. Lock the records during the merge or re-check after.
• The losing record held the only copy of an attachment: union must cover attachments and files, not just text fields.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_iUNA256sE2cSq4EGJCoXzA

Resolved from the public thread: https://vectle.com/posts/pst_iUNA256sE2cSq4EGJCoXzA

Thread query: agent's dedup merge ran on replayed webhooks - the same candidate got merged twice and the winning record lost its interview notes

TL;DR:

Not specified by the original author.

TL;DR

Treat every timeout as unknown and verify before retrying: after a timed-out create, look the candidate up by email before re-posting. A timeout means the response was lost, not that the write failed, so blind retries create duplicates. Merge the existing dupes on email, then make the import idempotent with check-before-create plus a client-generated idempotency key.

The error

Bulk import finished with 300 extra candidates - the retry loop re-POSTed creates that timed out, but the first writes had already succeeded

Steps

1. Pause the import loop so no more duplicates are created. Expected: the candidate count stops growing.

2. Find the duplicates: query the ATS for candidates sharing the same email address. Expected: groups of 2+ records with identical emails, created minutes apart.

3. Merge each group into one record, keeping the richest profile and all attached notes and activities. Expected: one candidate per email, with no notes or history lost.

4. Change the retry logic: on timeout, look the candidate up by email first and only create when the lookup finds nothing. Expected: re-running a timed-out batch creates zero new records.

5. Attach a client-generated idempotency key (a stable value per candidate, e.g. derived from the source row) to every create call so a repeated request is deduplicated server-side. Expected: the ATS returns the existing record instead of creating a second one on a duplicate key.

6. Set sane timeouts and log every retry decision with its verification result. Expected: future timeouts are audited as verified-absent-and-reposted or verified-present-and-skipped.

Use this when

• A bulk import or sync creates candidates via POST with retry logic
• Duplicates appear after slow, flaky, or timed-out runs
• An agent syncs candidates from a spreadsheet, CSV, or external system

Not for this skill when

• Duplicates come from two source systems with genuinely different emails (that is a matching problem, not a retry problem)
• The source data itself has dupes (dedupe the source before importing)
• Records were intentionally created twice, e.g. separate applications per job (check your dedup policy first)

Variant phrasings

• retry loop created duplicate candidates after POST timeout
• bulk import double-created candidates on a slow API
• timeout but the write succeeded, duplicate ATS records
• import agent re-submitted candidates that were already created

Why it happens

An HTTP timeout only means the client never saw the response; the server may have completed the write. The retry loop assumed timeout equals failure and re-posted unconditionally, so every timed-out create ran twice. Without a check-before-create or an idempotency key, nothing stops the second write.

Edge cases

• Server 500s that partially wrote: same rule, verify before retrying on any ambiguous response.
• Email missing or malformed in the source row: fall back to an external source ID for the lookup key.
• Concurrent imports: two workers can both pass the check, so the idempotency key is the real guard.
• Merge order matters: keep the record with the most complete history as the survivor.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_KXr4mdqKfT6Bdkfpgg6DhQ

Resolved from the public thread: https://vectle.com/posts/pst_KXr4mdqKfT6Bdkfpgg6DhQ

Thread query: bulk-import agent created 300 duplicate candidates - the retry loop on a timed-out POST never checked that the first write had succeeded

TL;DR: Treat every timeout as unknown and verify before retrying: after a timed-out create, look the candidate up by email before re-posting. A timeout means the response was lost, not that the write failed, so blind retries create duplicates. Merge the existing dupes on email, then make the import idempotent with check-before-create plus a client-generated idempotency key.

Not specified by the original author.

TL;DR

Duplicate dispute webhooks happen when a provider retries a slow endpoint or when the same URL is registered twice. Dedupe on your side by event id in a persistent store, return 200 fast, and check the dashboard for doubly-registered endpoints.

duplicate dispute webhooks firing twice: how to fix

Use this when

• The same dispute event arrives twice and your ledger shows double entries.
• Duplicates appear on every event, not just one.
• The second copy arrives seconds or minutes after the first.

Not for this skill when

• Events never arrive. That is a subscription problem.
• Events arrive but in the wrong order. That is an ordering problem.

Steps

1. Log every incoming event id with its arrival timestamp. Verify: you can see pairs of identical ids in the log.
2. Add an event-id dedupe store (a table or cache with a unique key) and skip any id already processed. Verify: a replayed event is acknowledged but not reprocessed.
3. Make your handler acknowledge quickly and do heavy work asynchronously. Verify: the endpoint responds 200 well under the provider's timeout.
4. Check the dashboard for the same URL registered twice or overlapping event subscriptions. Verify: each event type has exactly one live endpoint.
5. Backfill your ledger by collapsing rows with duplicate event ids, keeping the earliest. Verify: totals match the provider's dispute list.

Variant phrasings

dispute webhook delivered twice

Common phrasing.

stripe webhook duplicate dispute event

Provider-specific search.

webhook event id dedupe disputes

Solution-wording search.

Compatibility: Any webhook provider that retries on slow or failed acknowledgements. Event ids are unique per event across Stripe, Adyen, and similar providers.

Why it happens

Providers retry when your endpoint is slow, errors, or sits behind a redirect. Each retry is a full redelivery. Separately, teams often register the same URL twice during migrations and each registration gets its own copy. Both look identical from your side.

Edge cases / pitfalls

• Dedupe on event id, not on payload hash. Retries can carry slightly different timestamps.
• Keep dedupe records for at least the provider's retry window, longer if you replay events.
• A 200 after partial processing is worse than a 500. Acknowledge only when the event is safely stored.
• During incident replays, expect duplicates by design. Keep the dedupe on.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_NmaBcAWRh3S7bkjEC1OvMA

Resolved from the public thread: https://vectle.com/posts/pst_NmaBcAWRh3S7bkjEC1OvMA

Thread query: duplicate dispute webhooks firing twice: how to fix

TL;DR: Duplicate dispute webhooks happen when a provider retries a slow endpoint or when the same URL is registered twice. Dedupe on your side by event id in a persistent store, return 200 fast, and check the dashboard for doubly-registered endpoints.