onboarding agent lost checklist progress after context compaction

TL;DR

Long onboarding runs compact the agent's context, the checklist state evaporates, and the agent redoubles finished steps or skips unfinished ones. The fix: keep the checklist outside the model's memory. Persist every step's state to a durable store as it completes, and have the agent reload the checklist from that store whenever it resumes. Context becomes a cache; the store is the truth.

The error

Agent (after compaction): "Starting onboarding: step 1 of 8..."
Reality: steps 1-5 already completed an hour ago

Steps

1. Confirm the loss pattern. Compare the agent's transcript before and after compaction against the real world. Expected output: completed steps reappear as pending after the compact.
2. Move checklist state to durable storage. Write each step's status (pending, done, failed) to a database or file as it completes. Expected output: the store shows 5 of 8 done, independent of the transcript.
3. Make the agent load state on resume. The first action after any resume is reading the checklist store. Expected output: the agent announces "resuming at step 6" instead of "starting."
4. Make steps idempotent anyway. Even with the store, a retried step must be safe. Expected output: re-running a completed step changes nothing.
5. Run a long onboarding past a compaction point. Expected output: the agent resumes at the correct step with no repeats.

When to use

• The agent repeats steps after long runs
• Progress resets visibly after context compaction or summarization
• Onboarding has enough steps to outlast one context window

When not to use

• The agent fails on step one: a prompt or tool problem, not state loss
• Steps fail for real errors: fix the errors, the checklist is fine
• The run is short enough to fit in context: keep it simple

Tool compatibility

Any agent framework with context compaction or summarization (long-running onboarding agents, multi-step provisioning bots). The external-state pattern works with any durable store: a database row, a JSON file, or a task tracker API.

Variant phrasings

agent forgot progress after compaction

Persist the checklist outside the transcript.

onboarding agent repeats steps

Load state from the store on every resume.

context window lost checklist

The store is the truth; context is a cache.

agent restarts onboarding from scratch

Resume from durable state, not from memory.

Why it happens

Summarization keeps the gist and drops the details, and a checklist is all details: which of 8 steps are done is exactly the kind of thing a summary compresses into "onboarding in progress." The agent wakes up with a vague sense of the task and no record of position, so it starts over or guesses. This is not a reasoning failure; it is an architecture failure. State that must survive summarization must live outside the summarization boundary, in storage the summarizer cannot touch.

Edge cases

• The store write fails silently: the agent thinks it saved progress and did not. Confirm writes or the store is as unreliable as memory.
• Two agent instances share one checklist: add a lock or owner field so they do not both resume step 6.
• The summary itself is wrong about the goal: persist the goal and constraints alongside the checklist, not just the step states.
• Compaction happens mid-step: record "in progress" with a timestamp so a resume can detect and safely retry the interrupted step.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_nt9HSf4pMY2qSMyokooJDQ