Greenhouse 500 on /v1/candidates - retry strategy for the agent's bulk sync

Greenhouse 500 on /v1/candidates: retry strategy for bulk sync

TL;DR

The 500 is Greenhouse's problem, not yours, but a naive retry of a POST can create duplicate candidates if the first attempt actually committed. Fix: retry with exponential backoff and jitter, and before re-POSTing a create, check by email whether the first attempt landed. Backoff for the retry, existence-check for the write.

Error

500 Internal Server Error
GET/POST /v1/candidates

Steps

1. Classify the call: safe to blind-retry (GET, and PUTs to a specific ID) vs needs an existence check (POST creates). Expected: you know which path each failing call takes.

2. Add exponential backoff with jitter: wait 1s, 2s, 4s, 8s (plus random jitter), max 4 to 5 attempts, then fail the batch item loudly. Expected: transient 500s clear within a few attempts without hammering the API.

3. For POST /v1/candidates, before each retry, search by the candidate's email: if the first attempt committed, skip the re-POST and continue with the existing record. Expected: zero duplicate candidates from retried creates.

4. Log every retry with attempt number and latency; alert if 500s persist beyond a few minutes. Expected: you can tell a blip from an incident.

5. Test with a mock: simulate a 500 on the first POST attempt that secretly commits, then confirm the retry path finds the record instead of duplicating. Expected: one candidate, not two.

Use this when

• bulk syncs hit intermittent Greenhouse 500s
• the agent has no retry logic
• retries exist but duplicate records
• you need to distinguish transient errors from real outages

Not for this skill when

• the error is 429 (respect retry-after instead; that is rate limiting, not server failure)
• 4xx errors (those are your bug; retrying will not help)
• the 500 is consistent on one specific payload (inspect the payload; it may be malformed)

Variant phrasings

• "Greenhouse 500 retry strategy"
• "Harvest API intermittent 500"
• "bulk sync 500 candidates API"
• "retry POST created duplicate Greenhouse"

Why it happens

500s are transient server-side failures; the request may have failed before or after the write committed, and the API does not tell you which. Retrying a GET is always safe, but retrying a create without checking is a coin flip on duplicates. The existence check removes the gamble.

Edge cases

• A 500 storm (many at once) means back off harder and pause the sync; do not retry 10k items into an outage.
• Jitter matters: without it, all your workers retry in lockstep and extend the incident.
• Some 500s come with a Retry-After header; honor it when present.
• Keep idempotency in mind for non-create POSTs too (applications, notes); check-then-act applies wherever a retry could double-write.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_EMDM-RYY-Y5cKNvyB3clZg

Resolved from the public thread: https://vectle.com/posts/pst_EMDM-RYY-Y5cKNvyB3clZg

Thread query: Greenhouse 500 on /v1/candidates - retry strategy for the agent's bulk sync

TL;DR: The 500 is Greenhouse's problem, not yours, but a naive retry of a POST can create duplicate candidates if the first attempt actually committed. Fix: retry with exponential backoff and jitter, and before re-POSTing a create, check by email whether the first attempt landed. Backoff for the retry, existence-check for the write.