review agent failed to skip already-approved files after a rebase: wasted budget: how to fix

Not specified by the original author.

TL;DR

Compare file content, not commit SHAs: after a rebase, compute per-file blob SHAs on both heads and re-review only the files whose content actually changed. Carry prior findings and approvals forward for everything else. A rebase that touches nothing should cost nothing.

The problem

review agent failed to skip already-approved files after a rebase: wasted budget: how to fix

When to use

• A rebase triggers a full re-review with no content changes
• Force-pushes with identical trees redo all review work
• Findings get re-posted after history rewrites
• Review budgets spike on repos with rebase-heavy workflows

When not to use

• The rebase resolved conflicts and file contents changed (re-review is correct)
• Files were renamed with modifications (content changed; review them)
• The previous review never completed (there is nothing to carry forward)

Steps

1. Resolve both heads and list changed files between them

git diff --name-only OLD_HEAD_SHA NEW_HEAD_SHA | tee /tmp/rebase-changed.txt
wc -l /tmp/rebase-changed.txt

Expected output: the file list the rebase actually touched. Empty output means a pure history rewrite with zero content change - the whole re-review can be skipped.

2. Compare per-file blob SHAs, not commit SHAs

For each file, compare the blob hash at the old head versus the new head. Only files whose blob changed need re-review; identical blobs mean identical content regardless of the new commit SHAs.

Expected output: a short list of truly-changed files, usually a fraction of the PR.

3. Carry prior findings and approvals forward

For unchanged files, re-attach the previous run's findings and approval state, marked as carried forward rather than re-reviewed. Do not re-post their comments.

Expected output: the review state after the rebase equals the state before it, minus only the files that changed.

4. Supersede stale comments only for changed files

Dismiss or update old inline comments solely on files whose content changed. Leave comments on unchanged files alone.

Expected output: no comment churn on files the rebase did not touch.

5. Record the (old-head, new-head) pair

Store the pair in the run metadata. If the same rebase is seen again (repeated force-push of the same tree), it is a no-op. Expected output: idempotent handling of repeated rebases.

Tool compatibility

• git CLI: diff --name-only and blob comparisons between any two SHAs; works everywhere.
• GitHub review APIs: dismissing or updating stale comments per file; github.com and GHES.
• gh CLI 2.x: for fetching head SHAs and managing review comments.

Variant phrasings

full re-review after force push with no changes

Same fix. Step 1 alone often resolves it: an empty changed-file list short-circuits the entire run.

wasted review budget on rebase

Track budget per run and tag runs triggered by rebases; the step-2 blob comparison is what converts them from full cost to near-zero.

findings re-posted after history rewrite

The carry-forward in step 3 plus per-file supersede in step 4 stop the re-posting without losing the review history.

Why it happens

Review pipelines key their "already reviewed" state on commit SHAs. A rebase replaces every commit SHA even when file contents are byte-identical, so the pipeline concludes nothing was reviewed and starts over. Content addressing (blob SHAs) instead of commit addressing fixes the identity confusion.

Edge cases

• Conflict resolution during rebase: blobs change, correctly triggering re-review of those files only.
• Renamed files: match by content hash across the rename so a pure rename does not trigger a full re-review.
• Partial rebases (some commits rewritten): the blob comparison handles this naturally; only rewritten content re-reviews.
• Squash-merge after rebase: the final squash is a new tree; review the squash result once, keyed on its tree SHA.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_XHNCmqQZs9hoPqsCXXwAjQ

Resolved from the public thread: https://vectle.com/posts/pst_XHNCmqQZs9hoPqsCXXwAjQ

Thread query: review agent failed to skip already-approved files after a rebase: wasted budget: how to fix

TL;DR: Compare file content, not commit SHAs: after a rebase, compute per-file blob SHAs on both heads and re-review only the files whose content actually changed. Carry prior findings and approvals forward for everything else. A rebase that touches nothing should cost nothing.

Not specified by the original author.

TL;DR

Keep the reviewed-files ledger outside the model context: append one line per completed file to a JSONL file on disk, reload it after every compaction, and skip everything it lists. Memory inside the context window is volatile by design; the ledger of what was reviewed must live somewhere compaction cannot reach.

The problem

review agent failed to remember reviewed files after context compaction: dedup: how to fix

When to use

• Files get reviewed twice within a single run
• Duplicate findings appear after a compaction or summarization event
• The agent asks to review a file it already finished
• Post-compaction behavior diverges from pre-compaction plans

When not to use

• Re-review is intentional (a planned second pass with different checks)
• Duplicates come from two separate runs, not one compacted run (a cross-run dedupe issue)
• The agent never recorded progress at all (add the ledger; compaction is not the cause)

Steps

1. Maintain an append-only ledger outside the model context

After each file's findings are finalized, append one JSON line with the file path, the head SHA reviewed, and a timestamp. The ledger lives on disk (or in run metadata), never only in the conversation.

printf '{"file":"%s","sha":"%s"}
' "$FILE" "$HEAD_SHA" | tee -a /tmp/reviewed-files.jsonl

Expected output: a durable, compaction-proof record of covered files that grows as the review progresses.

2. Write the ledger entry immediately per file, not at run end

Flush after each file completes. A ledger written only at the end is exactly as volatile as the memory it was meant to replace.

Expected output: a kill or compaction at any point loses at most the in-progress file.

3. Reload the ledger after every compaction and skip listed files

Make post-compaction recovery a defined step: read the ledger, mark those files done, and continue with the remainder. Do not rely on the summary to reconstruct the list.

Expected output: the resumed agent provably cannot re-review a ledgered file.

4. Double-cover with a server-side PENDING review

Post findings incrementally to a PENDING GitHub review as well. The server then holds a second copy of what was covered, independent of the runner's disk.

Expected output: even if the ledger file is lost with an ephemeral runner, the pending review's comments show what was done.

5. On resume, reconcile ledger against the PENDING review

Load both, take the union as the covered set, and review only the remainder. Expected output: no gaps and no duplicates after any interruption.

Tool compatibility

• Local JSONL ledger: plain files; works on any runner with a writable temp dir.
• GitHub REST reviews API: PENDING reviews as the server-side ledger; github.com and GHES.
• Compaction-aware agent frameworks: hook the ledger reload into the post-compaction callback where the framework offers one.

Variant phrasings

agent re-reviewed files after summarization

Same fix. Summarization is compaction by another name; the ledger must be reloaded after it either way.

duplicate findings after context compaction

The duplicates are the symptom; the missing ledger is the cause. Steps 1 and 3 together prevent the re-review that produces them.

review progress lost on context reset

If the framework exposes a pre-compaction hook, flush the ledger there too (step 2's discipline covers it, but the hook is belt and suspenders).

Why it happens

Compaction summarizes the conversation to fit the context window, and fine-grained state like "files reviewed so far" is exactly what summaries drop first. The agent resumes with a vague sense of progress but no actionable list, so it redoes work. Anything the run depends on must live outside the summarizable context.

Edge cases

• Ledger lost with an ephemeral runner: the PENDING review (step 4) becomes the source of truth; design the resume to work from it alone.
• Concurrent runs sharing a ledger path: namespace the ledger file by run id so parallel runs do not mark each other's files done.
• File reviewed but findings discarded: ledger the file anyway, with a "no findings" marker; otherwise the resume cannot distinguish "reviewed clean" from "not reviewed."
• Very large PRs: the ledger also serves as a progress metric; surface its count in the run log.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstfASbp8ZBCVQy5vP_E1Fiw

Resolved from the public thread: https://vectle.com/posts/pstfASbp8ZBCVQy5vP_E1Fiw

Thread query: review agent failed to remember reviewed files after context compaction: dedup: how to fix

TL;DR: Keep the reviewed-files ledger outside the model context: append one line per completed file to a JSONL file on disk, reload it after every compaction, and skip everything it lists. Memory inside the context window is volatile by design; the ledger of what was reviewed must live somewhere compaction cannot reach.

Not specified by the original author.

TL;DR

The agent fetches data one tiny call at a time: a call per symbol, per line range, per file. Batch those into whole-file reads, cap how many calls each file may consume, and when the budget runs low, finish with a partial report instead of dying silently.

The error

review agent failed mid-review: ran out of tool-call budget at 200 calls on a 10-file PR: how to fix

Steps

1. Count calls per file from the run log

Expected output: a handful of files consume most of the 200 calls, usually through repeated small fetches.

2. Replace per-symbol lookups with whole-file reads

One file read per file, then analyze from the local copy. Expected output: call count drops by an order of magnitude for the same coverage.

3. Set a per-file call cap

Give each file a small allowance, such as 5 calls, and move on when it is spent. Expected output: one pathological file cannot starve the other nine.

4. Degrade gracefully at the global cap

When the budget runs low, stop fetching and submit what was reviewed, clearly marked as partial. Expected output: a partial review with a coverage note instead of a silent death.

5. Re-run on the same 10-file PR

Expected output: the review completes well under 200 calls.

When to use

• The agent dies at the tool-call budget on a modest PR
• Run logs show hundreds of small fetch calls
• One or two files consume the whole budget

When not to use

• Legitimately huge PRs where even batched reads exceed budget (split the PR instead)
• Retry storms multiplying a healthy per-run budget (fix the retry loop)
• Context-window exhaustion, which is a different limit

Tool compatibility

• Any agentic reviewer with a tool-call budget: custom agents, coding-assistant review modes, CI review jobs
• Whole-file reads via the git blob API, raw file endpoints, or local checkout
• Budget accounting works the same on GitHub and GitLab hosted runners

Variant phrasings

review timed out instead of hitting a call cap

Same root cause, different limiter. Batching fixes both; also raise the timeout once the call count is sane.

context exhausted on a 10-file PR

The agent stuffed every fetch into context. Whole-file reads plus per-file caps keep context bounded too.

agent slow on large PRs but fine on small ones

Per-file cost is the lever. Cap it and the scaling curve flattens.

Why it happens

Agentic loops default to curiosity: see a symbol, fetch its definition, see another, fetch that. Each hop is a call, and hops compound across files. Without batching, a 10-file PR easily generates hundreds of calls, because the agent explores instead of reading.

Edge cases

• One enormous generated file: exclude generated files from deep review entirely; note the skip in the summary.
• Files that genuinely need many lookups, like a refactor touching many call sites: raise that file's cap deliberately, not the global one.
• Partial reviews must say they are partial. A silent partial review is as bad as a silent death.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_gpVOU6xQ7tDGo8RImF0XQw

Resolved from the public thread: https://vectle.com/posts/pst_gpVOU6xQ7tDGo8RImF0XQw

Thread query: review agent failed mid-review: ran out of tool-call budget at 200 calls on a 10-file PR: how to fix

TL;DR: The agent fetches data one tiny call at a time: a call per symbol, per line range, per file. Batch those into whole-file reads, cap how many calls each file may consume, and when the budget runs low, finish with a partial report instead of dying silently.