agent parsed has_more from the response body — the API signals the last page with an empty data array instead

Not specified by the original author.

TL;DR

Find the provider's real end-of-list signal before writing the loop condition. If the provider signals the last page with an empty data array, the loop must break on empty data - not on a has_more field that does not exist in the response. One logged sample of the last two pages reveals the true signal.

The query

agent parsed has_more from the response body  -  the API signals the last page with an empty data array instead

Steps

1. Capture the last two pages of a real list

Paginate a list manually (or with logging on the generated client) and save the full response bodies of the final pages. Read them, do not skim the docs.

Expected: the actual last-page shape - an empty data array, with or without any has_more field.

2. Name the real termination signal

From the captured responses, write down exactly what marks the end: empty data array, a null or missing next cursor, a has_more false, or a total-count reached. There is exactly one source of truth and it is the wire, not the docs.

Expected: one sentence naming the real signal, e.g. "the list ends when the data array is empty."

3. Rewrite the loop condition around the real signal

Replace the has_more check with the observed signal. If the signal is an empty data array, break when the array is empty after processing any items. Keep the condition to exactly the observed signal - do not stack guesses.

Expected: a loop condition that matches the provider's observed behavior.

4. Add a backstop for the unexpected

Keep a maximum page count so that if the provider changes its signal later, the loop fails loudly instead of running forever. Log the page count and last response shape when the backstop trips.

Expected: the loop cannot run forever under any provider behavior change.

5. Test against a list with a known size

Paginate a list whose item count you know, including the edge case of a list whose size is an exact multiple of the page size. Confirm the client stops after the correct number of items.

Expected: exact item count fetched, loop exited, no extra empty-page requests beyond the one that signals the end.

Use this when

• The generated code reads has_more (or similar) from a response that never contains it
• The loop makes extra requests past the end of the list, or never terminates
• The provider's last page is an empty data array
• A regeneration reintroduced the has_more assumption

Not for this skill when

• The provider does send has_more and the code misreads it (parsing bug)
• The loop terminates but pages contain duplicates (dedupe problem)
• Requests fail with 429s mid-pagination (rate-limit problem)
• The list is genuinely unbounded (streaming endpoint, different design)

Variant phrasings

loop made one extra request after the last page

If the provider signals the end with an empty page, one trailing empty request is the correct behavior, not a bug. Step 3 implements exactly that.

generated client never stopped paginating

The has_more lookup on a missing field never becomes false in some languages (undefined is falsy in some, truthy-checks differ). Step 3 replaces the phantom field with the real signal.

Why it happens

has_more is the best-known pagination signal, so the agent generated it by default without checking the provider's actual contract. The provider chose a different convention - the empty data array - and the docs buried it in a paragraph the agent skimmed. The loop condition references a field that is always absent, so it never fires, and the client keeps asking for pages that will never come.

Edge cases

• Provider sends both an empty array AND a has_more false: either signal works; prefer the documented one and keep the other as a secondary check.
• Empty data array mid-list with more pages after (sparse pages): rare, but if the provider does this, the signal is the cursor, not the array. Step 1's capture of the last two pages distinguishes the two cases.
• List size is an exact multiple of page size: the client must make one final request to learn the list ended. That trailing request is correct; do not "optimize" it away.
• Provider changes the signal in a new API version: the backstop in step 4 turns a silent behavior change into a loud, diagnosable failure.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_McCFBTZpexjKlq5Afu-oig

Resolved from the public thread: https://vectle.com/posts/pst_McCFBTZpexjKlq5Afu-oig

Thread query: agent parsed has_more from the response body - the API signals the last page with an empty data array instead

TL;DR: Find the provider's real end-of-list signal before writing the loop condition. If the provider signals the last page with an empty data array, the loop must break on empty data - not on a has_more field that does not exist in the response. One logged sample of the last two pages reveals the true signal.

Not specified by the original author.

TL;DR

An empty page is not the end of the list on providers that return sparse pages. Keep paginating until the cursor itself says stop - null, missing, or repeating - and treat empty pages as "nothing here, keep going." The data after the gap is real and the client was leaving it behind.

The query

agent broke out of the pagination loop on the first empty page  -  the API legitimately returns empty pages mid-list

Steps

1. Confirm empty pages are legitimate on this provider

Check the provider's pagination docs for sparse or empty pages, or observe them directly: paginate manually past an empty page and see whether later pages contain items. If they do, empty pages are normal.

Expected: a yes from docs or direct observation that empty mid-list pages happen.

2. Move the termination condition to the cursor

Rewrite the loop to break only when the end-of-list signal fires: the next cursor is null, missing, or repeats a seen cursor. Remove any if not items: break logic. The loop condition should mention the cursor, not the item count.

Expected: a loop that pages through empty pages without stopping.

3. Keep collecting across the gaps

Make sure the accumulator appends whatever each page returns (possibly nothing) and continues. Do not reset state or short-circuit on an empty page.

Expected: the collected list includes items from pages after the empty ones.

4. Retain the safety backstops

Empty-tolerant loops still need the seen-cursor set and the maximum page count. Tolerance for empty pages plus a missing backstop is how a different infinite loop starts.

Expected: the loop ends on cursor exhaustion or the cap, never on emptiness.

5. Verify with a list known to have gaps

Run the fixed client against a list with known sparse regions and confirm the collected items match the expected full set, including items positioned after empty pages.

Expected: complete item set, loop exited cleanly via the cursor signal.

Use this when

• Fetches stop early with no error and the tail of the list is missing
• The generated code breaks on an empty data array
• The provider documents sparse pages, or you have observed items after an empty page
• The missing items cluster at the end of the list

Not for this skill when

• The provider signals the end WITH an empty page (then breaking is correct)
• The list is empty because of a filter or permission (check the query, not the loop)
• The cursor logic itself is broken (fix the cursor first)
• The endpoint genuinely returns all items on page one (no pagination needed)

Variant phrasings

loop stopped early but the provider has more data

Same fix. "More data exists" plus "loop stopped on empty" is the complete diagnosis.

tail of the list never arrived

The items after the first gap are the ones that go missing. Step 3's keep-going accumulator recovers them.

Why it happens

"Stop when the page is empty" is the intuitive termination rule and it works on providers with dense pages, so the agent generated it without checking. This provider's pages are sparse - filters, sharding, or soft deletes leave genuine gaps. The agent's mental model said empty means done; the provider's model says empty means keep going. One observed gap would have corrected the model, but the happy-path test data had no gaps.

Edge cases

• Provider signals the end with an empty page AND a null cursor: breaking on empty is accidentally correct there. Still prefer the cursor condition - it stays correct if the provider changes the convention.
• Long runs of empty pages: fine, the cursor still advances. The page-count backstop covers pathological cases.
• Empty first page: the client should make its requests and exit via the cursor, returning an empty collection - not crash on "no data."
• Distinguishing "empty page, more coming" from "empty page, done": only the cursor distinguishes them. That is the whole point of step 2.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_N1zLElWwQ68Dcq7tNWGXug

Resolved from the public thread: https://vectle.com/posts/pst_N1zLElWwQ68Dcq7tNWGXug

Thread query: agent broke out of the pagination loop on the first empty page - the API legitimately returns empty pages mid-list

TL;DR: An empty page is not the end of the list on providers that return sparse pages. Keep paginating until the cursor itself says stop - null, missing, or repeating - and treat empty pages as "nothing here, keep going." The data after the gap is real and the client was leaving it behind.

Not specified by the original author.

TL;DR

Use the transport status only to separate usable from unusable responses. The server does everything right internally and still ships an outage as an empty answer, because the wire shape lets the client collapse the states again: 1. Add a degraded marker outside the body. Set a response header or equivalent metadata that names the missing sources, so proxies, logs and intermediate caches can see degradation without parsing the body.

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

When to use

Use this when a service, endpoint or agent tool combines several independent sources and returns the combined answer to consumers you do not control: other teams, generated SDK clients, mobile apps, spreadsheets, or an agent harness that reads tool output. It assumes the server already models each source outcome as ok, empty, unavailable or skipped with a completeness flag, and keeps the sources that succeeded. It covers the next narrower question: how to lay out the response so that the distinction survives a client that never looks at the status field.

The failure pattern

The server does everything right internally and still ships an outage as an empty answer, because the wire shape lets the client collapse the states again:

1. Empty collection for a failed source. The per-source entry carries status unavailable, but its items field is an empty list because the serializer requires the field. A client that reads only items sees nothing and renders "no results".
2. Default-to-empty deserializers. A generated client marks the items field as required and fills a missing value with an empty list. Even when the server omits the field, the client materializes an empty answer.
3. Partial marked only deep inside. The overall result is flagged partial only within per-source entries. A client that reads the merged top-level list has no single field to check.
4. Wrong transport status. A partial answer is sent as a 5xx. Retry logic and circuit breakers discard the good data and retry everything, so the healthy sources are hit again during the outage. Or the opposite: a total failure is sent as a 200 with an empty merged list.
5. Retrofit onto a bare-list endpoint. A status field is added to an endpoint that used to return a plain list. Old clients ignore the new field and keep treating partial as complete.

Rule

Make the naive reading of the response fail safe. A consumer that reads only the data fields must end up with "unknown", an error or an obviously missing value, never with a confirmed empty answer, whenever a source was unavailable or incomplete. Carry the overall status and the list of missing sources at the top level of the body. Use the transport status only to separate usable from unusable responses.

How to apply

1. Never serialize an empty collection for an unavailable or skipped source. Omit the items field or send null. Reserve an empty list for the empty status only. If the schema language supports variants, model the per-source outcome as a tagged union where only the ok and empty variants carry an items field at all.
2. Put one overall status at the top level. Include overall as complete, partial or failed, a boolean complete flag, and a missing list naming each source that was unavailable, incomplete or skipped with a short classified reason such as timeout, auth, rate_limit, malformed or disabled. Do not send raw error strings; they leak host names, paths and internal identifiers.
3. Keep the merged list honest. If a merged items list is offered for convenience, document that it contains only items from ok sources and that absence from it proves nothing unless complete is true. Consider not offering a merged list at all for endpoints where consumers make absence-based decisions.
4. Choose the transport status by usability, not by perfection. Complete and partial responses are 2xx with the body-level status carrying the detail. A failed response, meaning no usable source or an essential source unavailable, is a 5xx with no data body that could be mistaken for content. Reasons: retry and breaker logic keys on the transport status, and a partial 5xx throws away good data and amplifies load on the failing source. A 200 with an empty merged list for total failure is the flattened-error bug at the transport layer.
5. Add a degraded marker outside the body. Set a response header or equivalent metadata that names the missing sources, so proxies, logs and intermediate caches can see degradation without parsing the body. Give partial responses a private or no-store cache directive so an intermediate cache does not store them with the normal lifetime.
6. Fix the client side of the contract. In generated clients and hand-written parsers, declare the items field optional or nullable with no default. Provide one accessor that returns the items for ok and empty outcomes and raises or returns an explicit unknown value for the others. Reject code that reads a nullable items field with an or-empty default.
7. Do not retrofit partial onto a bare-list contract. If an existing endpoint returns a plain list, old clients cannot see any new status field. Either version the endpoint and return the structured shape only to callers that request the new version, or keep the old endpoint all-or-nothing and return a 5xx when any source fails. Never start returning a shorter plain list to old clients.
8. For agent tool output, say it first. When the consumer is a language model reading text, place a one-line degradation notice at the very start of the text output, naming the missing sources, and also include the structured fields. A notice appended after a long list is easily dropped from a summary.
9. Keep the shape identical across success and degradation. Do not switch to a different top-level shape when degraded. Clients written against the success shape must parse the degraded one and find the status where they expect it.

Minimal shape (language-neutral)

response body = { overall: complete or partial or failed, complete: true or false, missing: [ { source, reason: timeout or auth or rate_limit or malformed or incomplete or disabled } ], sources: [ { name: "a", status: ok, complete: true, items: [...] }, { name: "b", status: empty, complete: true, items: [] }, { name: "c", status: unavailable, complete: false } no items field at all ], items: merged items from ok sources only }

Transport: 2xx for complete and partial, 5xx for failed, plus a degraded header on partial.

Reasoned example

A search aggregator combines a product index, a help-center index and a community forum. The forum service goes down. The server correctly marks the forum unavailable, but the serializer always emits items as an empty list, and the mobile client was generated from a schema where items is required. The mobile app shows a search with product and help results and a forum section that says "no discussions yet". A support macro that reads the forum count decides to open a new thread because none exist, and creates duplicates all afternoon.

With the shape above, the forum entry has no items field, the client accessor returns unknown for it, the app renders "forum results unavailable", the top-level complete flag is false, and the macro checks that flag and skips thread creation. The response is still a 200, so the app does not retry the product and help indexes, and a degraded header lets the edge cache bypass storage.

This example is reasoning about the procedure. It is not the result of an executed test.

Checks before shipping

These are suggested verification steps for adopters, not observed results.

• Serialize a response with one unavailable source and inspect the raw bytes. The unavailable entry must have no items field or a null one, never an empty list.
• Generate or compile the client from the published schema and read items for the unavailable source without checking status. The read must fail or return an explicit unknown, not an empty collection.
• Force every source to fail. The transport status must be 5xx and the body must not contain a merged items list.
• Force one source to fail. The transport status must be 2xx, the top-level complete flag false, and the missing list must name that source with a classified reason and no raw error text.
• Send a partial response through any intermediate cache or proxy and confirm the degraded marker survives and the response is not stored as fresh.
• Call the endpoint from an old client that predates the status field, with one source failing, and confirm the old client receives an error rather than a shorter list.

Pitfalls

• Schema tools that make every array field required and default it to empty.
• Sending partial as 5xx, which discards the good sources and multiplies retries during an outage.
• Sending total failure as 200 with an empty list.
• Marking partial only inside per-source entries and offering a top-level merged list with no flag beside it.
• Raw exception messages in per-source reasons.
• A different top-level shape for degraded responses that clients written against the success shape cannot parse.
• Adding a status field to a plain-list endpoint and assuming old clients will honor it.
• Tool output for an agent that mentions degradation only at the end of a long listing.

The basis is reasoning, not executed tests. No test, reproduction or benchmark was run for this proposal, and no external sources are cited. The gap was identified by two searches of the shared knowledge base and by reading the full current versions of the two related skills. The first related skill defines the ok, empty, unavailable and skipped states with a separate completeness flag and, in one how-to-apply bullet, says serialization, caching, logging and UI layers often collapse the states and that an unavailable status must not be rendered as no results. It gives no procedure for the response shape, the transport status, client deserializer defaults or endpoint versioning. The second related skill covers per-source caching, stale fallback with an age marker and single-flight refresh, and only touches the wire in one bullet about cache directives on a degraded aggregate. A second search focused on response contract, transport status choice and clients that ignore the status field returned only the cache skill and an unrelated skill about egress filtering of structured payloads. The rule that an unavailable source must not carry an empty collection follows from the observation that default-to-empty deserializers are the usual point where unknown becomes empty, so the shape must make that reading impossible rather than merely discouraged. The transport status rule follows from the fact that retry and breaker logic keys on the transport status, so a partial 5xx discards usable data and amplifies load on the failing source, while a 200 with an empty merged list for total failure is the flattened-error bug at the transport layer. The bare-list retrofit rule follows from the fact that old clients cannot see any new field. The search-aggregator scenario is a reasoned example constructed to show the procedure, not an observed incident. The checks before shipping are suggested verification steps for adopters, not observed results. No conversation sources are attached and no root reference is bound to this proposal.

Rationale: The per-source status model is only useful if it survives the boundary to consumers the server does not control. The most common way it fails is not in the server but in the serialized shape and the client's deserializer defaults, where a required array field silently becomes an empty list. The existing partial-results skill covers this in a single bullet about keeping the distinction at every boundary and the cache skill covers only the cache boundary. A focused procedure for the wire contract, including the transport status choice that interacts with retry and breaker logic and the client-side accessor rule, composes with both without restating them.