GitHub returned diff too large when agent tried to read the full PR diff

TL;DR: GitHub caps how big a diff response it will serve, so stop asking for the whole thing. Page the PR's file list, fetch each file's patch individually, and skip generated files first. Per-file fetches stay under the cap while the monolithic diff does not.

GitHub returned diff too large when agent tried to read the full PR diff

1. Confirm you are hitting the cap, not a timeout: request the diff endpoint and check whether the response is an explicit diff-too-large error versus a slow connection.

Expected: a clear error naming the size limit, distinct from a network timeout.

1. Switch to the paginated PR files endpoint. Walk it page by page with a modest page size until you have the full file list.

Expected: the complete list of changed files with per-file patch content, none of it through the capped diff endpoint.

1. Shrink the set before deep fetching. Filter out generated files, lockfiles, vendored code, and minified bundles from the file list.

Expected: fewer files to process, and the worst size offenders are gone.

1. For files whose individual patch is still truncated, fetch the file's before and after blob content directly and diff them locally.

Expected: full content for the big files, computed on your side with no server cap.

1. Process and summarize per file as you go instead of reassembling one giant diff in memory.

Expected: memory stays bounded and no step re-creates the oversized payload.

1. Verify completeness: compare your fetched file count with the PR's changed-file count.

Expected: the counts match, so nothing was silently dropped by the cap.

Use this when

• The diff endpoint returns diff-too-large on a big PR
• The agent gets an empty or truncated diff for a large change
• You need full diff content, not just the file list
• Per-file patches work but the whole-PR diff does not

Not for this skill when

• The fetch times out rather than returning a size error (that is a timeout/pagination problem)
• The PR is small but the diff fails (suspect auth, a bad SHA, or a deleted ref)
• You only need the file list, not patch content (the files endpoint alone is enough)
• The cap hits on a single enormous file (go straight to blob comparison for that file)

Variant phrasings

• "GitHub diff too large error on pull request"
• "cannot get diff for large PR, response too big"
• "patch endpoint truncated on big pull request"
• "diff too large when agent reads PR"

Why it happens

Serving a whole-PR diff is expensive: GitHub has to render every changed file into one response, so it enforces a size cap to protect the API. The per-file endpoints do not have the same problem because each response is small and independently cacheable. Agents that grab the diff URL out of habit hit the cap exactly on the PRs where review matters most.

Edge cases

• The files endpoint paginates; a naive single-page fetch silently drops files past the first page. Always follow pagination to the end.
• Blob comparison for huge files can still be heavy locally. Stream the blobs to disk rather than holding both in memory.
• Renamed files carry both paths in the file list. Use the new path for any follow-up comment or annotation.
• If the PR keeps growing while you fetch, re-check the file count at the end and fetch the delta.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_SaCxZ4dOjopvzTXS84e4Rw