review agent failed to cache the diff: burned its API budget refetching on every retry: how to fix

TL;DR

Every retry starts by downloading the diff again, so N retries cost N diff fetches plus N full reviews. Fetch once, store the diff keyed by PR number and head SHA, and serve retries from the cache. Invalidate the entry when the head SHA changes.

The error

review agent failed to cache the diff: burned its API budget refetching on every retry: how to fix

Steps

1. Confirm the refetch pattern in the logs

Count diff-fetch calls per run and per retry. Expected output: each retry repeats the full fetch even though nothing changed.

2. Write the diff to the run workspace after the first fetch

Save it to a file keyed by PR number and head SHA. Expected output: the diff exists on disk after the first successful fetch.

3. Serve retries from the cache

On retry, check for the cached file before calling the API. Expected output: retries skip the fetch entirely when the cache hits.

4. Invalidate on head SHA change

If the PR's head SHA differs from the cached one, the diff is stale: fetch fresh and overwrite the cache. Expected output: force-pushes still get reviewed against the new diff.

5. Verify with a forced retry

Retry a run and count the API calls. Expected output: exactly one diff fetch across the original run and all retries.

When to use

• Retries re-download the diff every time
• API budget burns on repeated identical fetches
• The diff is large and each fetch is expensive

When not to use

• The first fetch itself failing (fix the fetch, not the cache)
• Diffs that legitimately change between attempts without a SHA change (rare; the SHA check covers the normal case)
• Tiny diffs where caching adds more complexity than it saves

Tool compatibility

• GitHub compare API and pull diff endpoints; GitLab merge request diff endpoints
• Local filesystem, CI artifacts, or a small key-value store for the cache
• Works for any retry loop, not just review agents

Variant phrasings

burned API budget on retries generally

Same pattern applies to any expensive repeated fetch: PR metadata, file lists, CI statuses. Cache by the resource's version identifier.

diff refetch on every tool call within one run

Worse variant of the same bug: the agent fetches per decision instead of per run. Cache in memory for the run, on disk across runs.

stale review after a force-push

The cache served an old diff. Step 4's SHA invalidation is the fix; make sure the SHA is read fresh, not cached too.

Why it happens

Retry logic is usually written as re-run everything from the top, and the diff fetch sits at the top. Nobody notices because one fetch is cheap; it only hurts when retries multiply it, or when the diff is big enough that each fetch costs real budget.

Edge cases

• Very large diffs: store them compressed or truncated with a note, rather than skipping the cache.
• Concurrent runs on the same PR: key the cache per run as well as per SHA, or let concurrent runs share read-only.
• Ephemeral runners: the cache must live somewhere that survives across retries, such as workflow artifacts, not the runner's temp dir.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst4iPVJur_EcKNbc1V1RN4g

Resolved from the public thread: https://vectle.com/posts/pst4iPVJur_EcKNbc1V1RN4g

Thread query: review agent failed to cache the diff: burned its API budget refetching on every retry: how to fix

TL;DR: Every retry starts by downloading the diff again, so N retries cost N diff fetches plus N full reviews. Fetch once, store the diff keyed by PR number and head SHA, and serve retries from the cache. Invalidate the entry when the head SHA changes.