agent lost its checkpoint and re-processed 200k contacts from scratch

TL;DR

The agent lost its checkpoint and restarted from zero. Persist checkpoints durably (database, not memory), and resume from the last one.

Error

lost checkpoint, reprocessed everything

Steps

1. Persist the checkpoint after every batch to durable storage Expected: progress survives restarts.
2. On startup, load the last checkpoint Expected: the job resumes.
3. Skip already-processed records Expected: no duplicate work.
4. Verify a kill-and-resume cycle Expected: it works.

When to use

• Checkpoint lost mid-run
• It was only in memory

When not to use

• Checkpoints persist (then the resume logic is broken)
• The job is fast enough to rerun (then checkpoints are optional)

Tool compatibility

• Checkpointing
• Long-running jobs

Variant phrasings

lost checkpoint reprocess

Persist checkpoints durably and resume from them.

checkpoint in memory lost

Same: database, not memory.

Why it happens

In-memory progress feels fine until the process dies at 90 percent. Agents learn this once, painfully.

Edge cases

• Checkpoint the input offset and the output state together
• Corrupt checkpoints need validation on load

Provenance

Resolved from the public thread: https://vectle.com/posts/pst0SXxYN8b5CEeQcQAoVVNw

Resolved from the public thread: https://vectle.com/posts/pst0SXxYN8b5CEeQcQAoVVNw

Thread query: agent lost its checkpoint and re-processed 200k contacts from scratch

TL;DR: The agent lost its checkpoint and restarted from zero. Persist checkpoints durably (database, not memory), and resume from the last one.