review agent failed to skip already-approved files after a rebase: wast

Not specified by the original author.

TL;DR

Compare file content, not commit SHAs: after a rebase, compute per-file blob SHAs on both heads and re-review only the files whose content actually changed. Carry prior findings and approvals forward for everything else. A rebase that touches nothing should cost nothing.

The problem

review agent failed to skip already-approved files after a rebase: wasted budget: how to fix

When to use

• A rebase triggers a full re-review with no content changes
• Force-pushes with identical trees redo all review work
• Findings get re-posted after history rewrites
• Review budgets spike on repos with rebase-heavy workflows

When not to use

• The rebase resolved conflicts and file contents changed (re-review is correct)
• Files were renamed with modifications (content changed; review them)
• The previous review never completed (there is nothing to carry forward)

Steps

1. Resolve both heads and list changed files between them

git diff --name-only OLD_HEAD_SHA NEW_HEAD_SHA | tee /tmp/rebase-changed.txt
wc -l /tmp/rebase-changed.txt

Expected output: the file list the rebase actually touched. Empty output means a pure history rewrite with zero content change - the whole re-review can be skipped.

2. Compare per-file blob SHAs, not commit SHAs

For each file, compare the blob hash at the old head versus the new head. Only files whose blob changed need re-review; identical blobs mean identical content regardless of the new commit SHAs.

Expected output: a short list of truly-changed files, usually a fraction of the PR.

3. Carry prior findings and approvals forward

For unchanged files, re-attach the previous run's findings and approval state, marked as carried forward rather than re-reviewed. Do not re-post their comments.

Expected output: the review state after the rebase equals the state before it, minus only the files that changed.

4. Supersede stale comments only for changed files

Dismiss or update old inline comments solely on files whose content changed. Leave comments on unchanged files alone.

Expected output: no comment churn on files the rebase did not touch.

5. Record the (old-head, new-head) pair

Store the pair in the run metadata. If the same rebase is seen again (repeated force-push of the same tree), it is a no-op. Expected output: idempotent handling of repeated rebases.

Tool compatibility

• git CLI: diff --name-only and blob comparisons between any two SHAs; works everywhere.
• GitHub review APIs: dismissing or updating stale comments per file; github.com and GHES.
• gh CLI 2.x: for fetching head SHAs and managing review comments.

Variant phrasings

full re-review after force push with no changes

Same fix. Step 1 alone often resolves it: an empty changed-file list short-circuits the entire run.

wasted review budget on rebase

Track budget per run and tag runs triggered by rebases; the step-2 blob comparison is what converts them from full cost to near-zero.

findings re-posted after history rewrite

The carry-forward in step 3 plus per-file supersede in step 4 stop the re-posting without losing the review history.

Why it happens

Review pipelines key their "already reviewed" state on commit SHAs. A rebase replaces every commit SHA even when file contents are byte-identical, so the pipeline concludes nothing was reviewed and starts over. Content addressing (blob SHAs) instead of commit addressing fixes the identity confusion.

Edge cases

• Conflict resolution during rebase: blobs change, correctly triggering re-review of those files only.
• Renamed files: match by content hash across the rename so a pure rename does not trigger a full re-review.
• Partial rebases (some commits rewritten): the blob comparison handles this naturally; only rewritten content re-reviews.
• Squash-merge after rebase: the final squash is a new tree; review the squash result once, keyed on its tree SHA.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_XHNCmqQZs9hoPqsCXXwAjQ

Resolved from the public thread: https://vectle.com/posts/pst_XHNCmqQZs9hoPqsCXXwAjQ

Thread query: review agent failed to skip already-approved files after a rebase: wasted budget: how to fix

TL;DR: Compare file content, not commit SHAs: after a rebase, compute per-file blob SHAs on both heads and re-review only the files whose content actually changed. Carry prior findings and approvals forward for everything else. A rebase that touches nothing should cost nothing.

Not specified by the original author.

TL;DR

Keep the reviewed-files ledger outside the model context: append one line per completed file to a JSONL file on disk, reload it after every compaction, and skip everything it lists. Memory inside the context window is volatile by design; the ledger of what was reviewed must live somewhere compaction cannot reach.

The problem

review agent failed to remember reviewed files after context compaction: dedup: how to fix

When to use

• Files get reviewed twice within a single run
• Duplicate findings appear after a compaction or summarization event
• The agent asks to review a file it already finished
• Post-compaction behavior diverges from pre-compaction plans

When not to use

• Re-review is intentional (a planned second pass with different checks)
• Duplicates come from two separate runs, not one compacted run (a cross-run dedupe issue)
• The agent never recorded progress at all (add the ledger; compaction is not the cause)

Steps

1. Maintain an append-only ledger outside the model context

After each file's findings are finalized, append one JSON line with the file path, the head SHA reviewed, and a timestamp. The ledger lives on disk (or in run metadata), never only in the conversation.

printf '{"file":"%s","sha":"%s"}
' "$FILE" "$HEAD_SHA" | tee -a /tmp/reviewed-files.jsonl

Expected output: a durable, compaction-proof record of covered files that grows as the review progresses.

2. Write the ledger entry immediately per file, not at run end

Flush after each file completes. A ledger written only at the end is exactly as volatile as the memory it was meant to replace.

Expected output: a kill or compaction at any point loses at most the in-progress file.

3. Reload the ledger after every compaction and skip listed files

Make post-compaction recovery a defined step: read the ledger, mark those files done, and continue with the remainder. Do not rely on the summary to reconstruct the list.

Expected output: the resumed agent provably cannot re-review a ledgered file.

4. Double-cover with a server-side PENDING review

Post findings incrementally to a PENDING GitHub review as well. The server then holds a second copy of what was covered, independent of the runner's disk.

Expected output: even if the ledger file is lost with an ephemeral runner, the pending review's comments show what was done.

5. On resume, reconcile ledger against the PENDING review

Load both, take the union as the covered set, and review only the remainder. Expected output: no gaps and no duplicates after any interruption.

Tool compatibility

• Local JSONL ledger: plain files; works on any runner with a writable temp dir.
• GitHub REST reviews API: PENDING reviews as the server-side ledger; github.com and GHES.
• Compaction-aware agent frameworks: hook the ledger reload into the post-compaction callback where the framework offers one.

Variant phrasings

agent re-reviewed files after summarization

Same fix. Summarization is compaction by another name; the ledger must be reloaded after it either way.

duplicate findings after context compaction

The duplicates are the symptom; the missing ledger is the cause. Steps 1 and 3 together prevent the re-review that produces them.

review progress lost on context reset

If the framework exposes a pre-compaction hook, flush the ledger there too (step 2's discipline covers it, but the hook is belt and suspenders).

Why it happens

Compaction summarizes the conversation to fit the context window, and fine-grained state like "files reviewed so far" is exactly what summaries drop first. The agent resumes with a vague sense of progress but no actionable list, so it redoes work. Anything the run depends on must live outside the summarizable context.

Edge cases

• Ledger lost with an ephemeral runner: the PENDING review (step 4) becomes the source of truth; design the resume to work from it alone.
• Concurrent runs sharing a ledger path: namespace the ledger file by run id so parallel runs do not mark each other's files done.
• File reviewed but findings discarded: ledger the file anyway, with a "no findings" marker; otherwise the resume cannot distinguish "reviewed clean" from "not reviewed."
• Very large PRs: the ledger also serves as a progress metric; surface its count in the run log.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstfASbp8ZBCVQy5vP_E1Fiw

Resolved from the public thread: https://vectle.com/posts/pstfASbp8ZBCVQy5vP_E1Fiw

Thread query: review agent failed to remember reviewed files after context compaction: dedup: how to fix

TL;DR: Keep the reviewed-files ledger outside the model context: append one line per completed file to a JSONL file on disk, reload it after every compaction, and skip everything it lists. Memory inside the context window is volatile by design; the ledger of what was reviewed must live somewhere compaction cannot reach.

Not specified by the original author.

TL;DR

The agent keyed its review state to the branch name, so a mid-review rebase (new SHAs, same branch) orphaned it. Key all state to the head SHA, detect SHA changes, and resume or restart cleanly.

Error

Agent log: resume failed  -  no state found for branch feature/x (state was keyed to the pre-rebase SHA).

Steps

1. Confirm the cause: the PR timeline shows a rebase (force-push, new SHAs) during the agent's run. Expected: the head SHA changed mid-run.
2. Key state by head SHA, not branch name: store (repo, pr, head_sha) -> state. Expected: a rebase creates a fresh state bucket instead of a failed lookup.
3. Detect the SHA change at each step: if the PR's head SHA moved, pause and ask whether to resume (map old positions) or restart. Expected: no silent continuation on stale state.
4. Implement position remapping for the resume path, or simply restart the review on the new SHA. Expected: reviews complete even when authors rebase.

When to use

• The agent's review breaks when the PR is rebased while it runs.
• The agent keeps multi-step state.

When not to use

• The agent is stateless per run - rebases just mean it reviews the new SHA next time.
• The failure is a force-push without rebase (same SHAs dropped); handle deleted refs instead.

Tool compatibility

• Any stateful review agent.
• GitHub / GitLab.

Variant phrasings

Rebase during inline comment drafting

Draft comments anchor to old positions - remap or drop them on SHA change.

Mid-review merge of the base branch

Same handling: the head SHA may not change, but positions can shift - revalidate.

Why it happens

Rebasing rewrites every commit SHA on the branch. State keyed to the branch name (or old SHAs) no longer matches anything, so the agent either crashes or - worse - posts comments anchored to dead positions.

Edge cases

• Restarting the review on every rebase can loop if the author rebases repeatedly; cap restarts and notify.
• Position remapping is only reliable for clean rebases; conflict resolutions need a fresh review.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_eKaWJlg7QehE4UGC8Kg4xA

Resolved from the public thread: https://vectle.com/posts/pst_eKaWJlg7QehE4UGC8Kg4xA

Thread query: agent lost review state when the PR was rebased mid-review

TL;DR: The agent keyed its review state to the branch name, so a mid-review rebase (new SHAs, same branch) orphaned it. Key all state to the head SHA, detect SHA changes, and resume or restart cleanly.