ticket-fetching agent stuck in pagination loop: cursor guard

TL;DR

A pagination loop means the cursor is not advancing: the API returns the same page (or the same cursor) repeatedly and the fetcher follows it forever. Guard the loop: track seen cursors, require forward progress each iteration, and bail out with an alert when progress stalls. Pagination needs a termination proof, not just a next-page pointer.

The query

ticket-fetching agent stuck in pagination loop: cursor guard

Use this when

• Fetch jobs never complete
• The same tickets get fetched repeatedly
• API pagination seems to loop
• You are building paginated consumers

Not for

• API authentication setup
• Designing ticket filters
• Data warehouse schema design
• Webhook versus polling choice

Steps

1. Log the cursor on every page request

Record each request's cursor and the returned next cursor. The log shows the loop directly: the same cursor repeating, or cursors cycling. Without the log you are guessing about the loop shape.

Expected output: a cursor trace showing the loop.

2. Require forward progress per iteration

Track a progress measure: new ticket ids seen, or a cursor that has never appeared before. If an iteration yields no new ids and a seen cursor, stop. Progress is the loop invariant; its absence is the exit signal.

Expected output: a progress check gating every iteration.

3. Keep a seen-cursor set and stop on repeat

Store every cursor used; if the API hands back one already seen, the pagination is cycling. Stop and alert. The seen set is cheap insurance against infinite loops.

Expected output: repeat-cursor detection with alert.

4. Cap total pages as a backstop

Set a maximum page count far above any legitimate fetch. The cap never triggers on healthy runs and always triggers on loops. Backstops catch the loop shapes your guards did not anticipate.

Expected output: a page cap that only fires on pathology.

5. Handle the empty-page-with-cursor case

Some APIs return an empty page with a next cursor (eventual consistency, filters). Decide the policy: follow a bounded number of empty pages, then stop. Unbounded empty-page following is a slow-motion infinite loop.

Expected output: an explicit empty-page policy.

Template: the checklist

Pagination loop guards:
[ ] Cursor logged per page request
[ ] Forward progress required each iteration
[ ] Seen-cursor set with stop-on-repeat
[ ] Total page cap as backstop
[ ] Explicit empty-page policy

Variant phrasings

pagination loop api fetch

Steps 2 and 3. Progress check plus seen set.

fetcher keeps getting same page

Steps 1 and 3. Log the cursor, stop on repeat.

ticket sync never finishes

Steps 2 and 4. Progress invariant plus page cap.

Why it happens

Pagination code gets written against well-behaved APIs in testing: cursors always advance, pages always have data, the last page is clearly marked. Production APIs have edge cases: eventual consistency returning the same page twice, filters that empty a page without ending the list, cursor encoding quirks. The fetcher trusts the API's "next" unconditionally, and unconditional trust loops.

Edge cases

• Data changing during fetch: new tickets arriving mid-fetch can shift pages. Use stable sort and tolerate overlap via dedup.
• Time-based pagination: clock boundaries can repeat. Prefer cursor over offset where available.
• API returning the first page on invalid cursor: a corrupted cursor restarts silently. Validate cursors before use.
• Parallel page fetching: concurrent workers must partition deterministically; overlapping partitions look like loops in the logs.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_MaBPlgjnD1kiKcUO8mIx-w