context window overflow crashed the agent's campaign-summary step; the batch failed

TL;DR

A campaign summary that loads every message into the context is a design with a built-in ceiling: the campaign only has to outgrow the window once. Summarise hierarchically instead: per-message structured verdicts stored in a table, per-cluster summaries over those rows, and a final summary over cluster summaries and counts computed by code. The model should read aggregates and samples, never the raw campaign.

context window overflow crashed the agent's campaign-summary step; the batch failed

Use this when

• The summary step fails on large campaigns but works on small ones.
• The error names context length or the run dies at the final step after successful triage.

Not for this skill when

• The failure is a model outage error, not a length error. Check the error text before redesigning.
• The campaign is small and the summary includes huge raw headers per message. Trimming inputs may be enough.

Steps

1. Measure the token size of the current summary input on a failing campaign. Verify: the overflow is quantified, not assumed.
2. Persist per-message verdicts as structured rows during triage, with counts and groupings computed by code. Verify: campaign statistics exist without the model.
3. Build cluster summaries from the rows for each sender, kit or URL group, each summarised separately. Verify: no single summary call approaches the context limit.
4. Compose the final report from cluster summaries plus code-computed counts, with a few exemplar messages quoted. Verify: the final call input is a small fraction of the window.
5. Run the summary on the previously failing campaign. Verify: it completes and its counts match the triage table exactly.

Variant phrasings

context window overflow campaign summary agent batch

The overflow variant.

hierarchical summarisation large campaign triage report

The hierarchical variant.

Compatibility: LLM agent frameworks with structured storage and staged summarisation.

Why it happens

Context windows grow, but campaigns grow faster, and raw evidence is the bulkiest possible input. The fix is architectural: evidence lives in storage, the model works on reductions, and arithmetic is done by code that does not hallucinate totals.

Edge cases / pitfalls

• Never ask the model to count. Counts from code, narrative from the model.
• Exemplar selection matters: choose median and extreme examples deliberately, not the first rows.
• Keep cluster boundaries stable across runs so summaries are comparable over a campaign lifetime.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_w5ZZVqhbtymKNsf1yiyTvw