intel agent hit context overflow, summarization timed out on 50-page 10-k

Intel agent hit context overflow summarizing a 50-page 10-K

TL;DR

Context overflow on a 50-page 10-K means the agent stuffed too much filing into one prompt and the model refused. The fix is the same hierarchy that works for 200-page filings: summarize per ITEM section, then synthesize. Add a running token estimate so the agent chunks before the overflow, not after the failure.

The error

(agent failed)
intel agent hit context overflow, summarization timed out on 50-page 10-k; input exceeded context window

When this helps

• an agent overflows context on mid-size filings
• summarization fails with token limit errors
• designing chunked summarization
• budgeting prompt sizes

When it doesn't

• the filing fits comfortably; chunking adds passes for nothing
• the overflow is from tool output bloat; trim the tools first
• you need verbatim long quotes; extract them separately

Works with

python 3.8+ with re and json. Any model with a context limit.

Steps

1. Estimate tokens before sending any prompt

import re
htm = open("filing.htm").read()
chars = len(htm)
print("chars:", chars, "estimated tokens:", chars // 4)
print("budget per prompt: 30000 tokens; chunk when over")

Expected: A token estimate. Four chars per token is rough but good enough for chunking decisions.

2. Chunk by ITEM section under the budget

import re
htm = open("filing.htm").read()
parts = re.split(r"(?i)(ITEM\s+\d+[A-Z]?\.)", htm)
chunks = []
for i in range(1, len(parts), 2):
    chunk = (parts[i] + parts[i+1])[:120000]
    chunks.append(chunk)
print("chunks:", len(chunks))

Expected: Bounded chunks. Each chunk fits the per-prompt budget with room to spare.

3. Summarize chunks then synthesize

import json
notes = {"c1": "item 1 summary", "c2": "item 7 summary"}
open("notes.json", "w").write(json.dumps(notes))
print("synthesis prompt sees", sum(len(v) for v in notes.values()), "chars")

Expected: A tiny synthesis input. Two levels keep every prompt inside the window.

4. Retry the failed section with a smaller chunk on overflow

import json
print("on overflow error: halve the chunk and retry the section alone")
print("log the section; repeated overflows mean the splitter is wrong")

Expected: A recovery rule. Overflow becomes a chunking signal, not a run failure.

Other ways people phrase this

context overflow 50-page 10-k

Estimate tokens, chunk by ITEM, synthesize summaries.

summarization timed out context limit

The timeout is the overflow's symptom. Chunk first.

prompt exceeded max tokens filing

Per-chunk budgets make overflow structurally impossible.

Why it happens

A 50-page 10-K is around a megabyte of HTML, far past any context window. Agents that accumulate filing text across turns hit the wall mid run. Estimating tokens up front and chunking by the filing's ITEM structure keeps every prompt bounded.

Edge cases

• Token estimates are rough; keep a 2x safety margin under the limit.
• Tables need special handling; do not split mid-table.
• The synthesis step can still overflow with many sections; budget it too.
• Cache section summaries; reruns should not re-summarize.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_yn57xJ4dDoUubymRXIhiAg