nested pagination broke the generated client - the agent paginated the parent list but dropped child items after page...

TL;DR

Every paginated list needs its own loop, including lists nested inside other lists. When the client pages through parents, it must also page through each parent's children with that parent's child cursor - not just take the first child page and move on. One loop per list level, each with its own cursor state.

The query

nested pagination broke the generated client  -  the agent paginated the parent list but dropped child items after page one

Steps

1. Identify every paginated level in the response

Map the response shape: which fields are lists, and which of those lists the provider paginates. Common shape: a paginated parent list where each parent embeds a paginated child list (orders with line items, projects with tasks).

Expected: a written map of list levels and the cursor or page parameter for each.

2. Confirm the child drop with a count

Pick a parent you know has many children. Fetch it through the generated client and count the children returned, then compare with the provider's child count for that parent. The gap is the dropped pages.

Expected: a number proving children stop after page one.

3. Add an inner pagination loop per parent

Inside the parent loop, paginate the child list for the current parent using the child cursor, collecting all child pages before moving to the next parent. Keep the child cursor state separate per parent - never reuse one parent's child cursor for another.

Expected: the client fetches every child page for every parent.

4. Watch the request volume

Nested pagination multiplies requests: 100 parents times 5 child pages is 500 calls. Add the rate-limit handling the vertical expects (backoff, concurrency cap) before running the nested loop at scale, or the fix trades missing data for 429s.

Expected: a concurrency or pacing control on the inner loop, tested against the provider's limits.

5. Verify totals at both levels

After the fix, compare parent count and total child count against the provider's numbers (or a manual spot check). Both levels must match, not just the parent level.

Expected: parent count correct AND summed child count correct.

Use this when

• Parent records arrive complete but child lists are truncated
• The generated code has one pagination loop for a response with two paginated levels
• Child counts in the client are consistently lower than the provider's counts
• The provider paginates an embedded list (check its docs for the child list's cursor)

Not for this skill when

• The child list is not paginated by the provider (then truncation is a filter or limit issue)
• Children fail with permission errors (auth/scope problem)
• The parent pagination itself is broken (fix the outer loop first)
• Children are missing because the parent request did not ask for them (field-expansion problem)

Variant phrasings

only the first page of line items came back for each order

Same fix. The inner loop in step 3 pages line items per order.

child cursor was reused across parents

A subtler version: the client paginated children but carried one cursor across parents, fetching parent B's children with parent A's cursor. Step 3's per-parent cursor state fixes it.

Why it happens

The agent generated pagination for the list it was asked about - the parents - and treated the embedded child list as a plain array. In the docs' example the parent had three children, which fit on one child page, so the truncation was invisible. Real parents have dozens of children, the provider paginates them, and the generated code never loops the inner list because nothing in the example suggested a second level of paging.

Edge cases

• Provider offers a bulk child endpoint: fetching all children in one (paginated) call and joining locally can beat N inner loops. Check before nesting.
• Child pages are large and parents are many: consider parallelizing the inner loops with a bounded worker pool, but keep the per-provider rate limit in mind.
• Child list order is unstable: collect all child pages first, then sort locally by a stable key.
• Grandchild levels: the same rule recurses. Every paginated level gets its own loop and its own cursor.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstvTQwIRAc-VewCgmCNdUmw