context window overflow: review agent truncated the diff and missed the bug

TL;DR: Measure the diff in tokens before sending it, split oversized diffs into per-file or per-hunk chunks, and carry a rolling summary between chunks. Models silently drop input past their context limit, so a whole-diff review is really a review of the prefix only. Never ship a review from a truncated run.

context window overflow: review agent truncated the diff and missed the bug

1. Measure the diff size in tokens before sending it anywhere, using the same tokenizer the model uses. Expected: a token count you can compare directly against the model's limit.

2. If the diff exceeds about 70 percent of the limit, do not send it whole. Split it into per-file or per-hunk chunks that each fit comfortably. Expected: every chunk is under the limit with headroom left for the response.

3. Review each chunk independently and carry a short rolling summary forward so later chunks keep the context of earlier ones. Expected: findings in chunk 5 can reference decisions made in chunk 1.

4. After all chunks are reviewed, check coverage: every changed file must appear in at least one chunk's findings or in an explicit "no issues" note. Expected: zero changed files unaccounted for.

5. Merge and dedupe findings across chunks before posting. Expected: one clean finding list with no repeats.

6. If the model ever reports truncation mid-run, treat the whole run as invalid and re-run with smaller chunks. Never ship a partial review. Expected: partial reviews never reach the PR.

Use this when

• the agent feeds whole diffs to the model in a single call
• the PR is large enough that the tail of the diff may not fit
• reviews mysteriously miss obvious issues in later files

Not for this skill when

• the diff is small and fits easily (the misses have another cause - check the review instructions)
• the model supports the full diff size with room to spare (measure first)
• the model ignores instructions rather than input size (a truncation warning is the tell for this skill)

Variant phrasings

review agent only reviewed the first half of the diff

llm dropped the end of a large diff and missed the bug

context limit cut off the diff during code review

Why it happens

Models drop input past their context limit (silently, or with an error, depending on the API). An agent that concatenates the whole diff into one review call gets a review of the prefix only. The dropped tail is exactly where the agent then "finds nothing" - because it never saw it.

Edge cases

• Token counts from a different tokenizer mislead; always use the model's own tokenizer.
• Hunk headers and file paths consume tokens too; budget for them.
• Summaries carried between chunks can themselves grow; cap the rolling summary length.
• Some APIs truncate from the middle or the start rather than the end; know your provider's behavior before assuming the tail is what got cut.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_D162EDD3gbOaXy80cubILQ