# Briefing agent crashed parsing a 200-page 10-K overnight

## TL;DR
A 200-page 10-K crashes parsers because the agent tries to hold the whole filing in memory and context at once. The fix is chunked processing: split the filing into sections, parse and summarize each chunk independently, then combine the section summaries. No single step ever sees the whole document, which is also how you stay inside model context limits.

## The error
```text
(agent crash)
briefing agent crashed parsing 200-page 10-K overnight; out of memory / context overflow
```

## When this helps
- a briefing agent crashes on huge filings
- overnight runs die on 200-page 10-Ks
- designing chunked document processing
- staying inside model context limits

## When it doesn't
- the filing is small; chunking adds needless complexity
- the crash is a parse bug on a specific section; fix the parser
- you need cross-section coreference; chunked summaries lose some of that

## Works with
python 3.8+ with re. Filing structure follows SEC 10-K item conventions.

## Steps
### 1. Split the filing into sections before parsing
```python
import re
htm = open("filing.htm").read()
parts = re.split(r"(?i)(ITEM\s+\d+[A-Z]?\.)", htm)
print("sections found:", len(parts) // 2)
open("sections.txt", "w").write("\n".join(p[:60] for p in parts[1::2]))
print(open("sections.txt").read()[:300])
```
Expected: Section boundaries like ITEM 1., ITEM 7. Splitting on 10-K structure gives natural chunks.

### 2. Process one section at a time with bounded memory
```python
import re
htm = open("filing.htm").read()
parts = re.split(r"(?i)(ITEM\s+\d+[A-Z]?\.)", htm)
for i in range(1, min(len(parts), 6), 2):
    chunk = (parts[i] + parts[i+1])[:20000]
    print("chunk", i, len(chunk), "chars")
print("streaming pattern: one chunk in memory at a time")
```
Expected: Per-chunk processing. Memory stays flat regardless of filing size.

### 3. Summarize sections independently, then combine
```python
import json
section_notes = {"ITEM 1": "business overview", "ITEM 7": "mdna summary"}
combined = " ".join(section_notes.values())
open("brief.json", "w").write(json.dumps({"sections": section_notes, "combined": combined[:200]}))
print("map-reduce pattern: sections then synthesis")
```
Expected: A combined brief from section notes. The synthesis step sees summaries, never the full filing.

### 4. Set a per-section timeout so one bad section cannot kill the run
```python
import signal
def handler(signum, frame):
    raise TimeoutError("section timeout")
print("wrap each section parse in a 120s timeout; skip and continue on timeout")
```
Expected: A timeout pattern. Malformed sections get skipped with a note instead of crashing the overnight run.

## Other ways people phrase this
### agent crashed parsing large 10-k
Chunk by ITEM sections. No step sees the whole filing.

### out of memory 10-k parse agent
Stream sections instead of loading the document. Memory stays flat.

### overnight briefing crash huge filing
Per-section timeouts plus checkpoints make overnight runs survivable.

## Why it happens
A 200-page 10-K is tens of megabytes of HTML; loading it whole exhausts parser memory and model context alike. Agents that process linearly have no defense. Chunking by the filing's own ITEM structure bounds every step's input, and the map-reduce pattern keeps the synthesis step small.

## Edge cases
- Some 10-Ks have nonstandard item headings; fall back to page-range chunking.
- Tables split across chunk boundaries need overlap between chunks.
- Financial statements deserve their own chunk; do not merge them into MDNA.
- Checkpoint after each section so a crash loses minutes, not the night.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_zAJwF5EBKfVFMRbml4auvA