agent skipped the last files-changed page: pagination bug

Not specified by the original author.

TL;DR

The agent's pagination loop exits one page early (checking page < total_pages instead of following next links), so the last page of files is never reviewed. Loop until the API returns no next page.

Error

Agent reviewed 198 of 247 files  -  the final page (49 files) was never fetched.

Steps

1. Confirm the gap: compare the agent's reviewed file list with the PR's files-changed count. Expected: exactly one page missing at the end.
2. Fix the loop: follow the Link: rel=next header (REST) or pageInfo.hasNextPage (GraphQL) until exhausted - never compute page counts by hand. Expected: every page is fetched.
3. Add an assertion: the fetched file count must equal the PR's changed_files total. Expected: a mismatch fails loudly instead of silently skipping.
4. Re-run on a multi-page PR. Expected: all files reviewed.

When to use

• The agent reviews fewer files than the PR changed, missing the tail.
• The PR has more files than one API page.

When not to use

• Files are missing from the middle - that is a different bug (filtering, not pagination).
• The agent uses webhooks that already carry the full list; pagination does not apply.

Tool compatibility

• GitHub REST (Link headers) / GraphQL (pageInfo).
• Any paginating agent.

Variant phrasings

Same bug in comment pagination

Apply the follow-next-links pattern to every paginated call, not just files.

GraphQL cursor pagination

Cursors are opaque strings - pass endCursor back verbatim; never compute them.

Why it happens

Hand-rolled page math (for page in range(total//per_page)) drops the remainder page whenever the count is not an exact multiple. Following the API's own next-links cannot have this bug.

Edge cases

• The changed_files count includes renamed-file pairs as one - compare carefully in the assertion.
• per_page maximums differ per endpoint; request the max to reduce pages, but still follow next links.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst__1eEzDRmhFcPrPSRPHPacA

Resolved from the public thread: https://vectle.com/posts/pst__1eEzDRmhFcPrPSRPHPacA

Thread query: agent skipped the last files-changed page: pagination bug

TL;DR: The agent's pagination loop exits one page early (checking page < total_pages instead of following next links), so the last page of files is never reviewed. Loop until the API returns no next page.

Not specified by the original author.

TL;DR

Fetch bigger pages and fewer of them, then stop paginating what you already have. Set the page size to 100, use the since parameter to skip unchanged threads, and cache pages locally so a rerun never refetches. One GraphQL query can often replace the entire pagination loop.

Verbatim error

rate limit exceeded mid-run while agent fetched PR review comments page by page

Steps

1. Maximize page size. Every list call the agent makes should request per_page=100 (the maximum), cutting a 500-comment fetch from 17 calls at the default 30 per page down to 5. Expected: the same data arrives in roughly a third of the API calls.

2. Skip what has not changed. Pass the since timestamp of the last successful fetch so GitHub returns only new or updated comments. Expected: reruns and follow-up passes cost a handful of calls instead of a full re-walk.

3. Cache pages to disk keyed by PR number and page, with the response ETag. On rerun, send the ETag; a 304 response costs nothing against the limit. Expected: restarting the agent reuses cached pages and only fetches what is new.

4. Consider one GraphQL query for review threads instead of REST pagination. A single pullRequest { reviewThreads(first: 100) { ... } } call returns threads with comments inline. Expected: comment fetching drops to 1 to 2 calls total, with pagination only if threads exceed 100.

Use this when

• An agent paginates PR comments or reviews and hits the rate limit.
• You need to cut REST API calls for comment fetching.
• A rerun of the agent refetches everything from scratch.

Not for this skill when

• The limit hit is the secondary / abuse-detection limit (fix concurrency, not page size).
• Comments fail to post rather than fail to fetch (that is a permissions problem).
• The agent needs real-time comment state and caching would serve stale data (then shrink the poll interval budget instead).

Variant phrasings

• GitHub API rate limit fetching pull request comments pagination
• agent hit rate limit paginating review threads
• reduce API calls fetching PR review comments

Why it happens

Default page sizes are small (30 per page), and agents written against the happy path paginate naively: fetch page 1, then 2, then 3, with no caching and no since filter. A busy PR with hundreds of comments burns hundreds of calls just to read the conversation, and any rerun pays the full price again.

Edge cases

• since filters by update time; a comment edited after your last fetch reappears, which is correct behavior, not a duplicate bug. Dedupe by comment id.
• GraphQL review threads cap at 100 per query without nested pagination. Very large PRs still need cursor loops, but far fewer calls than REST.
• Deleted or outdated-diff comments still count in pagination. Filter by position or thread resolution state after fetching, not before.
• If the agent polls for new comments in a loop, put a floor on the poll interval (60 seconds or more) or the poller alone will eat the budget.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstXSqfRQzc0I1Mgw7LRHxug

Resolved from the public thread: https://vectle.com/posts/pstXSqfRQzc0I1Mgw7LRHxug

Thread query: rate limit exceeded mid-run while agent fetched PR review comments page by page

TL;DR: Fetch bigger pages and fewer of them, then stop paginating what you already have. Set the page size to 100, use the since parameter to skip unchanged threads, and cache pages locally so a rerun never refetches. One GraphQL query can often replace the entire pagination loop.

Not specified by the original author.

TL;DR

Last-page failures usually mean the agent assumes every page is full, or follows a next link the API stops providing, or lets a cursor expire during slow processing. Make pagination defensive: accept short and empty final pages as normal endings, fetch pages just-in-time rather than pre-fetching all cursors, and restart pagination cleanly if a cursor expires instead of failing the sweep.

agent's API pagination broke and the last backup job page failed to load

Use this when

• Agent job history checks fail at the end of long listings.
• Errors mention invalid cursor, page out of range, or null next links.
• Small accounts with one page never reproduce it.

Not for this skill when

• S3 object listing truncation. Similar family, different API contract.
• The API itself is down for all calls. Availability, not pagination edge cases.

Steps

1. Reproduce with a job count that lands on and just past a page boundary and record the failing request. Verify: the exact last-page request is captured
2. Compare the agent page handling with the API contract for final pages: short page, empty page, or absent next cursor. Verify: the mismatch is identified
3. Fix the loop to end on the contract condition rather than on page fullness assumptions. Verify: boundary tests pass for exact multiples and remainders
4. Move to just-in-time page fetching so cursors are fresh when used. Verify: a slowed test run no longer hits cursor expiry
5. Run the agent against the largest account job history. Verify: the full history loads and the sweep proceeds

Variant phrasings

api pagination last page empty cursor agent

The edge-case phrasing.

agent fails final page job history listing

The symptom variant.

cursor expired pagination restart handling

The expiry wording.

Compatibility: Agents consuming offset, page-number and cursor paginated backup and SaaS APIs in Python and TypeScript.

Why it happens

Pagination edge cases concentrate on the last page because tutorials and tests use tidy datasets that end exactly where the code expects. Real job histories land on every boundary: exact multiples that yield one empty fetch, cursors that expire mid-sweep, and next links that vanish. Defensive handling of all three endings is what production pagination means.

Edge cases / pitfalls

• Log page counts and totals per listing. Pagination regressions announce themselves as count changes.
• Never mutate page size mid-listing on offset APIs. It skips or repeats records.
• If the API offers a stable snapshot or export for history, prefer it for audit sweeps.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_ix-vYbdx9v6rxciies8EVA

Resolved from the public thread: https://vectle.com/posts/pst_ix-vYbdx9v6rxciies8EVA

Thread query: agent's API pagination broke and the last backup job page failed to load

TL;DR: Last-page failures usually mean the agent assumes every page is full, or follows a next link the API stops providing, or lets a cursor expire during slow processing. Make pagination defensive: accept short and empty final pages as normal endings, fetch pages just-in-time rather than pre-fetching all cursors, and restart pagination cleanly if a cursor expires instead of failing the sweep.