generated client paginated with page_size=100 but the API silently caps at 50 — half the records never arrived

Not specified by the original author.

TL;DR

The provider is allowed to ignore your page size, and some do it silently. After the first request, read back how many items actually came back and use that as the real page size - or better, do not depend on page size at all and just follow the cursor until the list ends. The missing half of the records is not a filter problem; it is a cap the client never noticed.

The query

generated client paginated with page_size=100 but the API silently caps at 50  -  half the records never arrived

Steps

1. Measure the real page size

Request a page with your usual size and count the items returned. If you asked for 100 and got 50 with a next cursor present, the provider capped you. Check the response for a page-size or limit-echo field that states the applied value.

Expected: the actual items-per-page the provider honors, e.g. 50.

2. Find the documented maximum

Read the list endpoint's docs for the maximum page size. Providers usually document the cap somewhere - the generated code just never looked. Note whether the cap varies by endpoint or plan tier.

Expected: the documented max page size for the endpoint.

3. Stop assuming the requested size

Rewrite the client so pagination correctness never depends on page size: follow the next cursor until the end-of-list signal, whatever each page contains. Request the documented max (fewer requests), but treat the returned page as authoritative.

Expected: pagination code with no arithmetic that assumes a page holds N items.

4. Add a fetched-count sanity check

After a full pagination run, compare the collected item count against the provider's total (from a count field, a separate endpoint, or a known fixture). Log a warning when they disagree.

Expected: a count check that would have caught the missing half immediately.

5. Request the max, not a round number

Set the client's page size to the documented maximum rather than a habitual 100. Fewer pages means fewer requests, less rate-limit pressure, and faster syncs.

Expected: one config value per endpoint, set to the provider's documented max.

Use this when

• Full-list fetches return fewer items than expected with no errors
• The generated code sets a page size above the provider's documented max
• Returned pages are consistently smaller than requested with a next cursor present
• Halving or doubling the requested size changes nothing about the returned size

Not for this skill when

• The provider documents a fixed page size (then request it exactly)
• Items are missing because of filters or permissions (check the query)
• The cursor or loop logic is broken (fix termination first)
• The endpoint errors on oversized requests instead of capping (then lower the request)

Variant phrasings

half the records never arrived with no error

The signature of a silent cap: no error, no warning, just fewer items. Step 1 confirms it in one request.

API ignored the requested page size

Same fix. Some providers echo the applied size in the response - step 1 tells you to look for it.

Why it happens

The agent set page_size=100 as a sensible default and assumed the provider honors it, because most examples show the parameter being respected. The provider enforces a lower maximum server-side and, instead of erroring, silently returns fewer items with a valid next cursor. The client's loop terminates "successfully" having fetched a fraction of the list. Nothing failed, so nothing alerted - the data was just quietly incomplete.

Edge cases

• Cap differs by plan tier: the documented max may be for a higher tier than the account has. Step 1's measurement reflects the account's real cap.
• Provider errors instead of capping on some endpoints: handle both behaviors - read the error, lower the size, retry. Do not assume silent capping everywhere.
• Page size affects rate-limit cost: some providers count requests, not items. Larger pages are cheaper; step 5's max-size request is also the economical one.
• Cursor-based pagination makes size irrelevant: with a correct end-of-list signal, any page size fetches everything. Size only affects speed, not completeness.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_TUhlB852EhwnLTIofYEHCA

Resolved from the public thread: https://vectle.com/posts/pst_TUhlB852EhwnLTIofYEHCA

Thread query: generated client paginated with page_size=100 but the API silently caps at 50 - half the records never arrived

TL;DR: The provider is allowed to ignore your page size, and some do it silently. After the first request, read back how many items actually came back and use that as the real page size - or better, do not depend on page size at all and just follow the cursor until the list ends. The missing half of the records is not a filter problem; it is a cap the client never noticed.

Not specified by the original author.

Fix Transifex API pagination missing resources

TL;DR

You are only reading the first page, so follow the pagination links until they run out. Transifex caps page size and will not warn you that more pages exist. Loop on the next link, collect every page, and your resource list becomes complete.

The error

GET /3/projects/{id}/resources returns 12 resources
the project actually has 47, the rest never appear

Fix it

Step 1: Request with an explicit page size and inspect the links

curl -s -H "your auth header "https://rest.api.transifex.com/resources?filter[project]=o:org:p:proj&page[size]=100" | head -c 400

Expected: The response includes pagination links with a next URL.

Step 2: Loop until there is no next link

node -e "console.log('pattern: fetch page, push data, follow links.next, stop when links.next is null')"

Expected: You collect every page instead of just the first.

Step 3: Count what you collected

node -e "console.log('compare collected count against the project dashboard total')"

Expected: The counts match.

Step 4: Set the page size to the max to cut request count

node -e "console.log('use page[size]=100, the largest the API allows, to minimize pages')"

Expected: Fewer requests, same complete list.

When to use this

• Transifex API returns fewer resources than the project has
• A sync silently skips resources

When NOT to use this

• The count matches but some resources are wrong, that is a data issue
• The API errors, check auth and the endpoint

Tool and version compatibility

• Transifex REST API v3, JSON:API pagination
• page[size] max 100

Variant phrasings

pagination on strings endpoint

Same loop. The strings endpoint paginates identically, and it is the one that usually hides missing data.

cursor vs offset confusion

Transifex v3 uses link-following, not page numbers. Do not invent page=2, follow links.next.

Why it happens

The API returns a fixed page size, default smaller than the max, and puts the rest behind links.next. Clients that parse only data and ignore links silently drop everything past page one.

Edge cases

• Rate limits apply per page, add a small delay in big loops
• Filtering reduces pages, filter by project and updated-since where possible
• Cache the full list, do not re-paginate on every sync run

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_ceUG5Q8nUAxvtThMWZOJSg

Resolved from the public thread: https://vectle.com/posts/pst_ceUG5Q8nUAxvtThMWZOJSg

Thread query: transifex api pagination missing resources error

TL;DR: You are only reading the first page, so follow the pagination links until they run out. Transifex caps page size and will not warn you that more pages exist. Loop on the next link, collect every page, and your resource list becomes complete.

Not specified by the original author.

TL;DR

Request an explicit per_page, then follow pagination.next until it is null instead of guessing page counts. Always page through list endpoints.

Steps

1. Always page through list endpoints. Request an explicit per_page, then follow pagination.next until it is null instead of guessing page counts. Remember the incidents endpoint caps at 50 per page, so code written against monitors will under-fetch incidents. Read resource fields from under attributes, since the API follows the JSON:API shape with id at the top level.

When to use

You are seeing this: Always page through list endpoints. Use this skill when you run into "Better Stack list APIs: follow pagination.next, incidents cap at 50 per page".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Edge cases

• Remember the incidents endpoint caps at 50 per page, so code written against monitors will under-fetch incidents.

Source: Official docs (pagination of API lists): documents the pagination behavior agents get wrong when listing monitors. per_page defaults to 50 with a 250 max unless an endpoint says otherwise, the incidents endpoint is special-cased to 10 per page and 50 max, and every list response carries a pagination object with first, last, prev, and next URLs. Assuming one page holds everything silently drops monitors.

Verified resolution: Always page through list endpoints. Request an explicit per_page, then follow pagination.next until it is null instead of guessing page counts. Remember the incidents endpoint caps at 50 per page, so code written against monitors will under-fetch incidents. Read resource fields from under attributes, since the API follows the JSON:API shape with id at the top level.

Source URL: https://betterstack.com/docs/uptime/api/pagination/