agent's pagination loop never terminated - the API returned the same cursor on the last page instead of null

TL;DR

Stop looping on "cursor is not null" alone. Track every cursor you have already seen, break when a cursor repeats or when a page comes back empty with an unchanged cursor, and cap total iterations as a backstop. The provider's last-page cursor is a repeat, not a null, so your termination condition has to recognize repeats.

The query

agent's pagination loop never terminated  -  the API returned the same cursor on the last page instead of null

Steps

1. Reproduce with a cursor log

Run the loop against a small dataset with logging on every iteration: page number, items returned, and the cursor sent and received. Watch the last pages.

Expected: a log showing the final cursor repeating identically across iterations - proof the provider never sends null.

2. Add a seen-cursor set

Keep a set of cursors already requested. Before each request, check whether the cursor is in the set; if it is, break. This catches the repeat-cursor provider and any other provider quirk that cycles cursors.

Expected: the loop terminates the first time a cursor repeats, on any provider behavior.

3. Break on empty page with unchanged cursor

If a page returns zero items and the next cursor equals the cursor you sent, stop. An empty page plus a stagnant cursor means there is nothing left to fetch, regardless of what the cursor value is.

Expected: termination on genuinely exhausted lists even when the provider keeps handing back a cursor.

4. Cap iterations and items as a backstop

Set a maximum page count (or maximum total items) far above any legitimate list size, and abort with a loud error when it trips. A backstop turns an infinite loop into a diagnosable failure.

Expected: the loop can never run forever; a tripped cap produces an alert naming the endpoint and cursor.

5. Verify with a full-list count check

After the fix, paginate a list whose size you know and confirm the client fetched exactly that many items and then stopped. Compare against the provider's total count if the API returns one.

Expected: fetched count matches the known total and the loop exits cleanly on the first try.

Use this when

• A pagination loop runs forever or far longer than the data justifies
• The last-page cursor equals the previous cursor instead of null
• The generated client loops on while cursor is not None
• API logs show the same cursor requested dozens of times in a row

Not for this skill when

• The loop 429s or errors out (rate-limit or error-handling problem)
• The cursor is malformed or double-encoded (cursor-format problem)
• The list genuinely keeps growing during pagination (live data, not a termination bug)
• Pages return duplicates but the loop does terminate (dedupe problem)

Variant phrasings

pagination loop fetched the same page repeatedly

Same root cause. The seen-cursor set in step 2 is the direct fix.

provider never returns a null cursor

Do not wait for one. Steps 2 and 3 terminate on repeat and stagnation, which work whether the provider nulls, repeats, or omits the cursor.

Why it happens

The agent learned pagination from the textbook pattern: "loop while next_cursor is not null." The provider implements the last page differently - it echoes the final cursor back instead of nulling it. The textbook condition is never false, so the loop never ends. The agent trusted the convention without verifying the provider's actual last-page behavior, which one logged run would have revealed.

Edge cases

• Provider returns a new-but-equivalent cursor each time: the seen-set misses it. Step 3 (empty page plus stagnant data) still terminates; also compare page contents, not just cursors.
• Legitimate duplicate cursors mid-list: rare, but if a provider repeats a cursor and then continues, the seen-set would cut the list short. The iteration cap plus a fetched-count sanity check distinguishes this from the infinite case.
• Cursor is null but data continues (the inverse bug): terminate on repeated empty pages instead. Termination should consider cursor, content, and count together.
• Very large lists near the iteration cap: size the cap from the provider's documented maximum plus headroom, not from a guess.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_puMbZJjyzXa1W6ma5u2DVg