agent passed the cursor double-URL-encoded - the API 400d and the agent retried the same bad cursor for hours

TL;DR

Encode the cursor once, at the HTTP client layer, and nowhere else. The double-encoding happens when application code encodes the cursor and the HTTP library encodes it again - so pass the raw cursor string to the library and let it do the encoding. Log the exact cursor sent on any 400 so the next failure is diagnosable in one look.

The query

agent passed the cursor double-URL-encoded  -  the API 400d and the agent retried the same bad cursor for hours

Steps

1. Compare the sent cursor with the received cursor

Log the cursor value as received from the provider and the cursor value actually placed in the request URL. Double-encoding shows up as %25 sequences (an encoded percent sign) or other doubled escape sequences.

Expected: the two values side by side, with the sent one visibly over-encoded.

2. Find both encoding sites

Search the generated code for manual encoding calls on the cursor (URL-encoding functions, quote helpers) AND check whether the HTTP library encodes query parameters automatically. The bug is the combination: manual encode plus automatic encode.

Expected: two identified encoding sites, or one manual site plus a library that encodes by default.

3. Remove the manual encoding

Delete the application-level encoding of the cursor and pass the raw string to the HTTP library's parameter handling. The library owns encoding; the application owns the value.

Expected: exactly one encoding step, owned by the HTTP layer.

4. Fix the retry loop that hammered the bad cursor

The agent retried the same bad cursor for hours because the retry logic never questioned the request itself. Add a rule: on a 400 (client error), do not retry the identical request - log the full request including the cursor and stop. Retries are for 429s and 5xxs, not for requests the server already judged malformed.

Expected: a 400 produces one logged failure, not hours of identical retries.

5. Re-run pagination and spot-check the wire

Paginate a real list with cursor logging on. Confirm the sent cursor matches the received cursor modulo the single transport encoding, and no 400s occur.

Expected: clean pagination with the cursor passing through exactly once-encoded.

Use this when

• The provider 400s on paginated requests with a cursor error
• Logged cursors contain %25 or other doubled escape sequences
• The generated code manually URL-encodes values it also hands to an HTTP library
• A 400 triggered hours of identical retries instead of one logged failure

Not for this skill when

• The cursor is treated as a page number (interpretation bug, not encoding)
• The cursor is expired server-side (time-based invalidation)
• The 400 names a different parameter (read the error body)
• The HTTP library does not encode automatically (then the manual encode is correct)

Variant phrasings

API 400d on every page after the first

First page needs no cursor, so encoding bugs only bite from page two. Steps 1 to 3 apply unchanged.

retried the same bad cursor for hours

That is the retry-policy half of this bug. Step 4 separates retryable failures (429, 5xx) from non-retryable ones (400).

Why it happens

The agent knew cursors "need encoding" and wrote an explicit encoding call, not knowing the HTTP library encodes query parameters automatically. Each layer did its job correctly in isolation; together they encoded twice. The provider decoded once, got a still-encoded string, and rejected it. The retry loop then treated a malformed request like a transient failure and replayed it for hours, because nothing told it that 400 means "fix the request, not the timing."

Edge cases

• Cursor contains characters that are legal unencoded: the single library encoding handles them. Do not special-case.
• Provider expects the cursor in a header instead of the query string: headers are not URL-encoded at all. Move the raw cursor to the header and skip encoding entirely.
• Library with encoding disabled by default: rare, but then the manual encode is the one correct site. Step 2's check of the library's actual behavior decides.
• Cursor stored encoded in a cache or database: normalize on read - decode once when loading, then follow step 3. A cache full of pre-encoded cursors reintroduces the bug silently.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstZjecehI3ClwTRSWkQ509g