# Intel agent hit the context limit, summarization failed mid 10-K run

## TL;DR
Hitting the context limit mid 10-K means the agent fed too much filing into one prompt. The fix is hierarchical summarization: summarize each ITEM section on its own, then summarize the section summaries. No prompt ever holds more than one section, so the limit becomes unreachable by construction.

## The error
```text
(agent failed)
intel agent hit context limit, summarization failed mid 10-K run; prompt exceeded max tokens
```

## When this helps
- an agent hits context limits on 10-Ks
- summarization fails mid filing
- designing hierarchical summarization
- budgeting prompt sizes

## When it doesn't
- the filing is short; hierarchy adds needless passes
- you need verbatim quotes; summaries lose them, extract quotes separately
- the limit is hit by tool output, not the filing; trim the tools

## Works with
python 3.8+ with re and json. Any model with a context limit.

## Steps
### 1. Split the filing into ITEM sections
```python
import re
htm = open("filing.htm").read()
parts = re.split(r"(?i)(ITEM\s+\d+[A-Z]?\.)", htm)
print("sections:", len(parts) // 2)
for p in parts[1::2][:6]:
    print(p[:40])
```
Expected: Section headings. The 10-K's own structure is the chunking scheme.

### 2. Summarize each section independently
```python
import json
sections = {"ITEM 1": "business summary", "ITEM 7": "mdna summary"}
open("section_summaries.json", "w").write(json.dumps(sections, indent=2))
print("one summary per section; each prompt held one section")
```
Expected: Per-section summaries. The model never sees more than a section at a time.

### 3. Summarize the summaries for the final brief
```python
import json
sections = json.load(open("section_summaries.json"))
combined = " ".join(sections.values())
print("synthesis input chars:", len(combined))
print("final prompt is summaries only, far under the limit")
```
Expected: A small synthesis input. Two levels of summarization keep every prompt bounded.

### 4. Enforce a per-section character budget
```python
import json
sections = json.load(open("section_summaries.json"))
budget = 20000
for k, v in sections.items():
    sections[k] = v[:budget]
print("sections truncated to budget; no prompt can overflow")
```
Expected: A hard budget. Truncation at the section level guarantees the limit holds.

## Other ways people phrase this
### context limit 10-k summarization failed
Summarize sections, then summarize the summaries. Two levels, bounded prompts.

### agent context overflow filing
Per-section budgets make overflow structurally impossible.

### prompt too long 10-k agent
The filing's ITEM structure is the natural chunking.

## Why it happens
10-Ks exceed every model's context window, so single-prompt summarization is doomed. Hierarchical summarization bounds each prompt to one section, and the synthesis prompt to the summaries. The context limit stops being a runtime surprise and becomes a design constant.

## Edge cases
- Section summaries lose cross-section nuance; keep key figures verbatim in the notes.
- Very long ITEM 7s may need sub-chunking; apply the budget recursively.
- Quote extraction should run on sections, not on summaries.
- Track per-section token estimates; budgets need measuring, not guessing.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Rw-Lk25t0xNEzdGQmUcbMw