http post timeout retry duplicate idempotent: stop duplicates on retry

http post timeout retry duplicate idempotent: stop duplicates on retry

TL;DR

A POST timeout means the outcome is unknown, not that the write failed: the server may have committed before the response was lost. Never retry a POST with a fresh identity. Generate one idempotency key before the first attempt, store it with the job state, and send the same key on every retry so the server can return the first result instead of creating a duplicate. If the API has no idempotency support, check the resource state by a client reference before retrying, and rely on a server-side unique constraint as the last line of defense.

The problem

http post timeout retry duplicate idempotent
client POST times out, retry succeeds, server now has two copies of the same write

Steps

1. Generate the idempotency key once, before the first attempt. Create a random UUID for this one logical operation and save it alongside the pending job or request record before any network call. Do not generate the key inside the retry loop. Expected: restarting the process or retrying an hour later still finds the same key for the same logical operation.

2. Send the same key on the first attempt and on every retry. Most idempotent APIs accept an Idempotency-Key request header. Send that header with the stored key each time, with the identical request body. Expected: a retry that reaches a server which already processed the key returns the original response (same status and body, often marked as a replay) and creates no second record.

3. Treat a timeout as unknown and check before assuming failure. On a timeout or a dropped connection, do not mark the operation failed. Either retry with the same key (step 2) or, for APIs without key support, query by a client-supplied reference such as your own external id or request id and only re-POST when no matching record exists. Expected: when the first attempt actually committed, the check finds it and the retry is skipped or safely replayed.

4. Add a server-side unique constraint as a backstop. If you own the API, enforce uniqueness in the database on the natural key of the write (for example one row per external reference) and store the idempotency key with its response so a repeated key replays the stored response. Client discipline alone cannot stop two concurrent retries. Expected: even a buggy client that retries with a new key hits the unique constraint instead of silently duplicating, and the error surfaces for reconciliation.

5. Bound the retries and keep the key for the whole window. Retry only transient outcomes (timeouts, connection resets, 502, 503, 504, and 429 honoring Retry-After) with exponential backoff and jitter. Keep the stored key at least as long as the server keeps idempotency records, commonly 24 hours. Never retry a 4xx validation or auth response by changing the key; fix the request and treat the corrected request as a new operation with a new key. Expected: retries stop after a small bounded number of attempts, and an exhausted operation stays recoverable with the same key instead of becoming a duplicate.

6. Verify the end state by count, not by the retry response alone. After the operation completes, confirm exactly one record exists for the logical operation (one row for the external reference, one charge, one created resource). Expected: the count is 1. If it is 2, the server lacks dedupe: clean up the duplicate and add steps 2 and 4 before the next run.

Use this when

• A retried POST creates duplicates after timeouts or flaky networks
• An agent, worker, or import loop re-sends creates that may have already succeeded
• You are adding retry logic to any non-idempotent HTTP write
• The API documents an Idempotency-Key header or a client reference lookup

Not for this skill when

• The request is a safe read (GET) or a naturally idempotent write (PUT or DELETE of a named resource): plain retries are fine there
• The duplicate came from two different logical operations that legitimately share content: dedupe by key would wrongly merge them, give each operation its own key
• The server returned a definitive 4xx: the write was rejected, retrying unchanged will not help

Compatibility

Works with any HTTP client (curl, requests, fetch, httpx) and any API that accepts an idempotency key header, supports lookup by client reference, or enforces unique constraints. Server-side key storage is commonly provided by payment and platform APIs; check the API docs for how long keys are retained and which endpoints accept them.

Variant phrasings

timed out POST created two records

Same fix: the first attempt committed. Adopt one key per logical operation and verify by client reference before any further retries.

retry loop double-charged or double-created

Same root cause with money or resources attached: stop the loop, reconcile the duplicate with the provider, then make every money-moving retry check-then-act with a stable key.

should I retry a POST after a timeout

Yes, but only with the same idempotency key, or after a state check proves the first attempt did not commit. A blind retry is a second write.

Why it happens

HTTP gives the client no signal when a response is lost after the server commits. A timeout collapses three different states (never received, received and failed, received and succeeded) into one observation. A retry written as a brand-new POST is, to the server, indistinguishable from a new intentional write, so it dutifully creates a second copy. The idempotency key restores the missing identity: it tells the server that all attempts carrying that key are one operation.

Edge cases

• Concurrent retries: two workers retrying at once can both pass a check-before-create test. Only the server-side unique constraint or key lock closes this race.
• Key reuse with a different body: never reuse a key for a changed payload. Servers that validate this reject the mismatch; treat a changed request as a new operation.
• Long server processing: if the first attempt is still in flight, a keyed retry usually waits or returns an in-progress response rather than starting a second execution. Back off instead of spawning parallel attempts.
• Partial success in batch POSTs: dedupe per item, not per batch, so retrying the failed items does not recreate the successful ones.
• Clock and expiry: if the retry window can outlive the server key retention, persist your own record of the created resource id so late retries resolve by lookup instead of re-creating.