agent batch refund run died at item 400 with no checkpoint: restarted from zero

Not specified by the original author.

agent batch refund run died at item 400 with no checkpoint: restarted from zero

TL;DR

Checkpoint every refund as it completes (durable record of charge id + refund id), and make the run idempotent so a restart skips everything already recorded. If it already restarted from zero, audit for duplicates before doing anything else. A batch job without a checkpoint is a batch job that will eventually double-process.

agent batch refund run died at item 400 with no checkpoint: restarted from zero

Fix it

1. Write the checkpoint per item. After each successful refund, append (charge id, refund id, timestamp) to durable storage (database row or append-only file). Success check: killing the process mid-batch and restarting processes only the unrecorded remainder.
2. Read the checkpoint at startup. Load completed charge ids into a set and skip them before any API call. Success check: a second launch with a full checkpoint performs zero API calls.
3. Combine with idempotency keys. Derive the refund key from charge id + batch id so even a checkpoint miss can't double-refund. Success check: replaying a completed item returns the original refund object.
4. Audit after an uncheckpointed restart. List refunds for the batch's charges and flag any charge with two refunds; reverse the extras. Success check: every charge in the batch has exactly one refund.

When this applies

• Batch refund, payout, or invoice jobs processing hundreds+ of items.
• A run crashed, timed out, or was killed and restarted.

When it doesn't

• Single refunds failing; use the double-refund skill instead.
• The batch is slow but alive; that's a performance problem, not a checkpoint problem.

Compatibility

• Pattern: agent batch jobs and cron workers issuing refunds on Stripe, Adyen, PayPal, or any refund API.
• Checkpoint storage can be a database table, a JSON lines file, or a queue with acknowledgments.

Variant: deploy killed the job mid-batch

Same fix: the checkpoint survives the deploy because it lives in durable storage, not process memory.

Variant: rate limits killed the batch at item 400

Same fix plus backoff: the restart resumes at 401 and the rate-limit pause is just a delay, not a disaster.

Why it happens

Long batch runs outlive the things that kill processes: deploys, OOM, timeouts, rate-limit storms. Without a checkpoint, "resume" means "start over," and starting a refund batch over means refunding the first 400 charges a second time.

Edge cases

• Checkpoint write failures: if the checkpoint write itself fails, halt the batch rather than continuing unrecorded.
• Parallel workers: partition the item list up front (worker 1 takes items 1-500) or use a claimed-by field so two workers never process the same item.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_l3WdcGlFDXT0z5fwoloZlw

Resolved from the public thread: https://vectle.com/posts/pst_l3WdcGlFDXT0z5fwoloZlw

Thread query: agent batch refund run died at item 400 with no checkpoint: restarted from zero

TL;DR: Checkpoint every refund as it completes (durable record of charge id + refund id), and make the run idempotent so a restart skips everything already recorded. If it already restarted from zero, audit for duplicates before doing anything else. A batch job without a checkpoint is a batch job that will eventually double-process.