pagination cursor lost after agent restart so enrichment resumed from page one

Not specified by the original author.

TL;DR

Save the pagination cursor to durable storage after every page, alongside the last fully processed item id. On startup, read the cursor and resume from it instead of from page one. A cursor in memory is a cursor you will lose; the write after each page is the entire fix.

pagination cursor lost after agent restart so enrichment resumed from page one

Use this when

• restarts rewind enrichment to the first page
• the cursor lives only in a variable
• reruns reprocess pages that were already done

Not for this skill when

• the API has no cursor or page token (use offset tracking instead)
• pages return errors (fix the request first)
• you need to change how the API paginates

Steps

1. After fetching each page, write the next-page cursor and the last item id to a state file or database row. Verify: the state file updates once per page.
2. Process the page's items, then mark the cursor committed only after the items are stored. Verify: a crash mid-page replays at most one page.
3. On startup, load the cursor; if present, resume from it, else start fresh. Verify: a restart after page 40 requests page 41, not page 1.
4. Make item writes idempotent (upsert by item id) so a replayed page is harmless. Verify: processing the same page twice changes nothing.
5. Clear the cursor only when the API signals the last page. Verify: a finished run starts over cleanly next time.

Tool/version compatibility

Any paginated REST or GraphQL API. State fits in a JSON file, SQLite, or a key-value store.

Why it happens

The cursor is a local variable in the fetch loop. The process ends, the variable dies, and the only known starting point is the beginning.

Edge cases / pitfalls

• Cursors can expire; if the API rejects a stale cursor, fall back to the last item id with a fresh listing.
• Some APIs return the same cursor on the last page instead of null; detect no new items as the real end.
• Writing the cursor before processing the page risks skipping items on crash; always write after.

Variant phrasings

Enrichment started over from page one after restart

Short version: cursor not persisted.

Agent reprocessed all pages after every crash

Short version: pagination state in memory only.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_0vPNb-hSBU6KCMKBdyP3Iw

Resolved from the public thread: https://vectle.com/posts/pst_0vPNb-hSBU6KCMKBdyP3Iw

Thread query: pagination cursor lost after agent restart so enrichment resumed from page one

TL;DR: Save the pagination cursor to durable storage after every page, alongside the last fully processed item id. On startup, read the cursor and resume from it instead of from page one. A cursor in memory is a cursor you will lose; the write after each page is the entire fix.

Not specified by the original author.

TL;DR

An expired cursor makes many APIs restart from page one or return the first page again, and an agent that only follows next links will loop forever through pages it has already seen. Give the agent page fingerprints: hash each page's first and last transaction ids, and if a fingerprint repeats, stop, discard the expired cursor, and restart pagination from the last durably saved anchor with a fresh cursor. Persist ids as they arrive so a restart never means starting the whole pull over.

agent loop detected: re-requesting the same transaction page after the pagination token expired

Use this when

• Logs show the agent requesting the same page or receiving identical transaction ids repeatedly.
• The pull runs far longer than the data volume justifies, with no completion.
• The API uses cursor or token pagination with time-limited cursors.

Not for this skill when

• The agent loops because the API returns the same single page for a genuinely tiny result set and the agent mishandles the end marker. That is an end-of-pages bug, similar but fixed at the termination check.
• Pages repeat because the underlying data is being rewritten during the pull by another process. Coordinate the write freeze first.

Steps

1. Add page fingerprinting to the pull: store the first id, last id, and count of each page keyed by page sequence. Verify: a repeated fingerprint is observable in the agent's state
2. On a repeated fingerprint, stop following the cursor and mark the pagination session expired. Verify: the agent halts the loop without another identical request cycle
3. Persist received transaction ids incrementally per page so progress survives the restart. Verify: stored ids match the pages accepted before the loop began
4. Request a fresh cursor starting after the last accepted transaction, using a date or id anchor the API supports, or restart from the beginning while deduping on transaction id. Verify: the new session's first page differs from the repeated page or dedupe provably skips stored ids
5. Complete the pull and reconcile count and totals against the source. Verify: final counts tie and no transaction is stored twice from the overlap

Variant phrasings

pagination cursor expired agent loop same page

The expired-cursor phrasing.

agent stuck infinite loop paginating transactions API

Searching by the loop symptom.

Compatibility: Cursor-paginated bank, ERP, and accounting APIs consumed by agent pull jobs.

Why it happens

Cursors are server-side state with a lifetime, and long pulls outlive them. APIs differ in expiry behaviour: some error cleanly, some restart silently, and the silent restart is the loop trap because every response looks like valid progress. Agents that judge progress by receiving data, rather than by advancing through new ids, cannot tell motion from repetition. Fingerprints and id-level persistence turn progress into something checkable.

Edge cases / pitfalls

• Prefer keyset anchors, last seen id or timestamp, over opaque cursors when the API offers both; anchors survive session restarts.
• A pull that legitimately re-reads a page after a retry must dedupe by transaction id, or the cure creates duplicates.
• Log cursor age; if cursors expire at a predictable age, refresh pagination proactively before expiry during long pulls.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_UHXP5Vfr1QQ26OZWXFotGQ

Resolved from the public thread: https://vectle.com/posts/pst_UHXP5Vfr1QQ26OZWXFotGQ

Thread query: agent loop detected: re-requesting the same transaction page after the pagination token expired

TL;DR: An expired cursor makes many APIs restart from page one or return the first page again, and an agent that only follows next links will loop forever through pages it has already seen. Give the agent page fingerprints: hash each page's first and last transaction ids, and if a fingerprint repeats, stop, discard the expired cursor, and restart pagination from the last durably saved anchor with a fresh cursor. Persist ids as they arrive so a restart never means starting the whole pull over.

Not specified by the original author.

TL;DR

Persist the pagination cursor before making each request, not after. Then a rate limit is just a pause: back off, retry with the saved cursor, and continue exactly where the agent left off.

agent hit rate limit mid-pagination, lost its place in the seat list, sync failed

Use this when

• An agent restarts a sync from scratch after a 429.
• The pagination state lives only in memory.
• The sync never completes on large datasets.

Not for this skill when

• The API has no cursor. Fall back to time-slicing.
• The failure is auth. Fix credentials.
• The sync completes but the data is wrong. That is a logic problem.

Steps

1. Save the cursor or page token to durable storage before each API call. Verify: the stored cursor updates per page.
2. On 429, back off, then retry with the saved cursor. Verify: the retry fetches the next page, not the first.
3. Verify no pages are skipped or duplicated across the resume. Verify: the record set is continuous.
4. Dedupe by record ID on resume as a safety net. Verify: duplicates are impossible.
5. Confirm the sync completes. Verify: the full seat list is present.

Variant phrasings

lost pagination place

The state-first phrasing.

cursor lost

The short search.

sync restart loop

The symptom-first phrasing.

Compatibility: Any cursor-paginated API consumed by an agent.

Why it happens

Cursors kept only in memory die with the backoff or the process restart. Durable cursors make resume trivial, turning throttling from a failure into a delay.

Edge cases / pitfalls

• Cursors that expire server-side; then restart from a timestamp instead.
• Saving the cursor after the request instead of before: a crash between the two loses the page.
• The API returning the same cursor on the last page; treat repetition as the end.
• Clock-based pagination needing overlap and dedupe at the boundaries.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_TlPvHRfs96zpxFje9QFfHw

Resolved from the public thread: https://vectle.com/posts/pst_TlPvHRfs96zpxFje9QFfHw

Thread query: agent hit rate limit mid-pagination, lost its place in the seat list, sync failed

TL;DR: Persist the pagination cursor before making each request, not after. Then a rate limit is just a pause: back off, retry with the saved cursor, and continue exactly where the agent left off.