context window overflowed mid-account-review: agent failed to save the at-risk list

TL;DR

The agent reads an entire book of business into one context and drowns in it: notes, tickets, and usage for hundreds of accounts exceed any window. Work account by account or in small groups, write each account's risk verdict to storage as it is decided, and keep only a compact running summary in context. The final list is assembled from storage, never from memory.

context window overflowed mid-account-review: agent failed to save the at-risk list

Use this when

• The review fails or truncates on large account sets
• Findings from early accounts vanish from the final output
• The agent loads full ticket and note histories per account

Not for this skill when

• The model in use has a window large enough and the failure is a provider error instead
• The task genuinely needs cross-account reasoning over full raw data

Steps

1. Measure where the context goes: tokens per account by data type (notes, tickets, usage). Verify: the dominant context consumer is identified
2. Restructure the review into per-account passes with a strict input budget per account (summaries and key fields, not raw histories). Verify: each pass fits with wide margin
3. Persist each account's verdict, evidence pointers, and confidence to a results store as it completes. Verify: progress survives any later failure and is inspectable mid-run
4. Assemble the at-risk list from the store with a final ranking pass over the compact verdicts only. Verify: the final list is complete and traceable to per-account evidence
5. Add a context-budget check that refuses or chunks inputs above the per-account budget. Verify: overflow becomes impossible by construction, not by luck

Variant phrasings

context overflow account review

A related phrasing for the same underlying problem.

agent lost at risk list

A related phrasing for the same underlying problem.

chunk large book review

A related phrasing for the same underlying problem.

Compatibility: Compatibility: LLM agent frameworks with tool-based storage (LangChain, custom orchestrators) and any model context size, current versions.

Why it happens

Context is working memory, not a database. Reviews that treat it as storage fail at scale and lose work silently because truncation drops the earliest findings first. Externalising verdicts as you go is the agent equivalent of writing results to disk instead of holding them in RAM.

Edge cases / pitfalls

• Per-account passes lose cross-account patterns like shared incidents. Add a cheap portfolio-level pass over metadata to catch those separately.
• Evidence pointers must resolve later; store ids and timestamps, not paraphrases, for the audit trail.
• Ranking over verdicts needs the verdict schema kept small and consistent, or the ranking pass inherits a parsing problem.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_FDQWf3Ap4zPAgGWGK8iI9Q