rate limit exceeded mid-run while agent fetched PR review comments page by page

TL;DR

Fetch bigger pages and fewer of them, then stop paginating what you already have. Set the page size to 100, use the since parameter to skip unchanged threads, and cache pages locally so a rerun never refetches. One GraphQL query can often replace the entire pagination loop.

Verbatim error

rate limit exceeded mid-run while agent fetched PR review comments page by page

Steps

1. Maximize page size. Every list call the agent makes should request per_page=100 (the maximum), cutting a 500-comment fetch from 17 calls at the default 30 per page down to 5.

Expected: the same data arrives in roughly a third of the API calls.

1. Skip what has not changed. Pass the since timestamp of the last successful fetch so GitHub returns only new or updated comments.

Expected: reruns and follow-up passes cost a handful of calls instead of a full re-walk.

1. Cache pages to disk keyed by PR number and page, with the response ETag. On rerun, send the ETag; a 304 response costs nothing against the limit.

Expected: restarting the agent reuses cached pages and only fetches what is new.

1. Consider one GraphQL query for review threads instead of REST pagination. A single pullRequest { reviewThreads(first: 100) { ... } } call returns threads with comments inline.

Expected: comment fetching drops to 1 to 2 calls total, with pagination only if threads exceed 100.

Use this when

• An agent paginates PR comments or reviews and hits the rate limit.
• You need to cut REST API calls for comment fetching.
• A rerun of the agent refetches everything from scratch.

Not for this skill when

• The limit hit is the secondary / abuse-detection limit (fix concurrency, not page size).
• Comments fail to post rather than fail to fetch (that is a permissions problem).
• The agent needs real-time comment state and caching would serve stale data (then shrink the poll interval budget instead).

Variant phrasings

• GitHub API rate limit fetching pull request comments pagination
• agent hit rate limit paginating review threads
• reduce API calls fetching PR review comments

Why it happens

Default page sizes are small (30 per page), and agents written against the happy path paginate naively: fetch page 1, then 2, then 3, with no caching and no since filter. A busy PR with hundreds of comments burns hundreds of calls just to read the conversation, and any rerun pays the full price again.

Edge cases

• since filters by update time; a comment edited after your last fetch reappears, which is correct behavior, not a duplicate bug. Dedupe by comment id.
• GraphQL review threads cap at 100 per query without nested pagination. Very large PRs still need cursor loops, but far fewer calls than REST.
• Deleted or outdated-diff comments still count in pagination. Filter by position or thread resolution state after fetching, not before.
• If the agent polls for new comments in a loop, put a floor on the poll interval (60 seconds or more) or the poller alone will eat the budget.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstXSqfRQzc0I1Mgw7LRHxug