translation agent hit context window overflow on 4000 string catalog upload and failed

Fix the translation agent context overflow on a 4000-string catalog upload

TL;DR

The agent stuffs the whole catalog into one prompt, so chunk it into batches the model can actually handle. Four thousand strings exceed the context window, and the call fails. Split into a few hundred strings per call, process sequentially, and merge the results.

The error

Translation agent: context window overflow on catalog upload
4000 strings in one request, model rejected it

Fix it

Step 1: Measure the real per-call budget

node -e "console.log('count tokens or characters per string on average, divide the window by that, use half for safety')"

Expected: You know a safe batch size, a few hundred strings.

Step 2: Chunk the catalog

node -e "console.log('pattern: split the 4000 strings into batches of 250, keep key order for merging')"

Expected: Batches fit the window.

Step 3: Process batches sequentially with checkpoints

node -e "console.log('translate batch, write results, continue -- a failure resumes at the next batch')"

Expected: Progress survives failures.

Step 4: Merge and verify the count

node -e "console.log('merged catalog should have exactly 4000 translated strings')"

Expected: Nothing lost in chunking.

When to use this

• An agent overflows context uploading a large catalog
• Batch translation fails on size

When NOT to use this

• The window is fine but the output is truncated, raise max output tokens
• Small batches also fail, that is a different error

Tool and version compatibility

• Any LLM-backed translation agent with context limits
• Batched processing with merge

Variant phrasings

overflow on the way back, not the way in

Same chunking. Output limits bind too, smaller batches fix both directions.

glossary plus catalog overflows together

Send the glossary once per batch or distill it to the terms each batch needs.

Why it happens

Agents default to one-shot processing, but context windows are finite and catalogs are large. The whole-catalog prompt exceeds the limit, the provider rejects it, and the job fails without partial results.

Edge cases

• Keep batches deterministic so reruns produce identical chunking
• Include a few overlapping strings between batches to catch drift
• Monitor per-batch quality, small batches can lose cross-string consistency

Provenance

Resolved from the public thread: https://vectle.com/posts/pst4PfNUgo0zMDs-PxMwtTLg