Fivetran list endpoints paginate with cursor plus next_cursor, not offset limit

Fivetran list endpoints paginate with cursor plus next_cursor, not offset limit

TL;DR

On every Fivetran list endpoint, start with no cursor, then pass cursor=[nextcursor] from each response until nextcursor is absent. Never pass offset expecting it to page. Write your list helper as a generator that yields items across pages and stop only when next_cursor is empty; log the page count so a missing-page bug is visible.

Use this when you run into the situation in the title.

When not to use this skill: unrelated tasks. It covers only the procedure above.

Compatibility

The steps above apply to the commands named in them. This skill does not pin a version, so if a flag looks different on your machine, check your installed version's docs first.

Details

Set limit explicitly (up to 1000) to reduce page counts on large accounts.

Context: Official Fivetran docs (REST API Pagination): documents that Fivetran uses cursor-based pagination. Each collection endpoint supports a cursor query parameter (omit it for the first page) and a limit parameter (1 to 1000 records per page, default 100). The response carries the items plus nextcursor, which is the value to pass as cursor for the next page. When nextcursor is missing, there are no more items. Agents that assume offset/limit pagination silently fetch only the first page, which corrupts any inventory or billing reconciliation built on the API.

Source: Official Fivetran docs (REST API Pagination): documents that Fivetran uses cursor-based pagination. Each collection endpoint supports a cursor query parameter (omit it for the first page) and a limit parameter (1 to 1000 records per page, default 100). The response carries the items plus nextcursor, which is the value to pass as cursor for the next page. When nextcursor is missing, there are no more items. Agents that assume offset/limit pagination silently fetch only the first page, which corrupts any inventory or billing reconciliation built on the API.

Verified resolution: On every Fivetran list endpoint, start with no cursor, then pass cursor=[nextcursor] from each response until nextcursor is absent. Never pass offset expecting it to page. Write your list helper as a generator that yields items across pages and stop only when next_cursor is empty; log the page count so a missing-page bug is visible. Set limit explicitly (up to 1000) to reduce page counts on large accounts.

Source URL: https://fivetran.com/docs/developer-resources/rest-api/getting-started/pagination