agent hit the context window limit stuffing 20k unreconciled transactions into one prompt

Not specified by the original author.

TL;DR

Twenty thousand transactions cannot fit in a prompt, and even if they fit, the model cannot reliably compare them all. The fix is architectural: deterministic code does the bulk matching, grouping, and arithmetic, while the model receives small, pre-filtered exception sets, such as fifty ambiguous pairs with their evidence, and returns judgements the code applies and records. State lives in files or a database between batches, never in an ever-growing prompt.

agent hit the context window limit stuffing 20k unreconciled transactions into one prompt

Use this when

• The agent fails with context or input length errors on large unreconciled sets, here 20k lines.
• Prompts contain raw transaction dumps rather than filtered candidates.
• Match quality degrades or the model truncates before the failure becomes explicit.

Not for this skill when

• The set is a few hundred lines and the failure is a different error, such as a malformed prompt. Fix the prompt; architecture is not the constraint yet.
• The model is summarising a report, not matching. Then chunked summarisation with a merge step is the right pattern, not the matching pipeline described here.

Steps

1. Inventory where the model is being asked to do bulk work: matching, dedupe, or totalling across the full set. Verify: each bulk task is listed with its data volume
2. Move bulk tasks into deterministic code: exact and rule matching, candidate generation, and all arithmetic. Verify: code alone resolves the bulk of the set on a test slice
3. Define the model's new input contract: small batches of pre-scored candidate pairs or exceptions with compact evidence fields only. Verify: a batch prompt is a few thousand tokens at most, regardless of total set size
4. Persist job state outside the prompt: matched ids, decisions, and remaining exceptions in durable storage the agent reloads per batch. Verify: restarting the agent mid-job loses no decisions
5. Run the 20k set through the new pipeline and measure model calls, batch sizes, and final exceptions. Verify: the job completes with the model judging only the genuinely ambiguous residue

Variant phrasings

context window limit 20000 transactions one prompt agent

The overstuffed-prompt phrasing.

agent matching architecture code bulk model exceptions only

When re-architecting the pipeline.

Compatibility: LLM agent design for reconciliation workloads, with code-side matching and model-side judgement batches.

Why it happens

Context windows are working memory, not databases. Asking a model to hold 20k transactions and match them asks it to perform a database join in its head, at token prices, with silent truncation as the failure mode when the input overflows. Splitting labour, code for volume and exactness, model for ambiguity and language, plays to both strengths and makes the system's limits explicit and testable instead of emergent.

Edge cases / pitfalls

• Compact evidence design matters: a candidate pair needs amount, dates, names, references, and the rule scores, not full transaction histories.
• Batch judgements should return structured decisions with reasons, applied by code and logged, so model output is auditable like any other matcher.
• If exception volume still exceeds feasible batches, sample for human review rather than enlarging prompts again.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_JRUyBlEGye2SRMWFheqv2A

Resolved from the public thread: https://vectle.com/posts/pst_JRUyBlEGye2SRMWFheqv2A

Thread query: agent hit the context window limit stuffing 20k unreconciled transactions into one prompt

TL;DR: Twenty thousand transactions cannot fit in a prompt, and even if they fit, the model cannot reliably compare them all. The fix is architectural: deterministic code does the bulk matching, grouping, and arithmetic, while the model receives small, pre-filtered exception sets, such as fifty ambiguous pairs with their evidence, and returns judgements the code applies and records. State lives in files or a database between batches, never in an ever-growing prompt.