to json not a pre-encoded string

Not specified by the original author.

When Zendesk API writes silently do nothing, check for double-encoded JSON. The requests library's json parameter expects a dict and encodes it itself; handing it a pre-built JSON string encodes it twice and the API ignores the mangled payload. Pass the dict, set Content-Type to application/json, and use email/token as the auth username for token auth.

Context: Stack Overflow #38923106 (accepted answer, 4 votes): the asker's macro updates via the Zendesk API silently did nothing. The accepted answer spotted double-encoding: they passed an already-formatted JSON string to requests' json parameter, which encodes its argument again. The fix is to pass a plain dict and let requests encode it once.

Matched source

Source: Published skill Original query: "Zendesk API via requests: pass an object to json=, not a pre-encoded string" Key terms: encoded, json, object, pass, requests, string, zendesk

Source: Stack Overflow #38923106 (accepted answer, 4 votes): the asker's macro updates via the Zendesk API silently did nothing. The accepted answer spotted double-encoding: they passed an already-formatted JSON string to requests' json parameter, which encodes its argument again. The fix is to pass a plain dict and let requests encode it once.

Verified resolution: When Zendesk API writes silently do nothing, check for double-encoded JSON. The requests library's json parameter expects a dict and encodes it itself; handing it a pre-built JSON string encodes it twice and the API ignores the mangled payload. Pass the dict, set Content-Type to application/json, and use email/token as the auth username for token auth.

Source URL: https://stackoverflow.com/questions/38923106/why-are-my-zendesk-macros-being-updated-but-no-change-actually-going-through

Not specified by the original author.

TL;DR

The body you sent is not valid JSON. The usual culprits are a trailing comma, single quotes instead of double quotes, or sending form-encoded data instead of JSON. Fix the payload so it is strict JSON with the q field present, and set Content-Type to application/json.

serper.dev 400 invalid JSON body

Use this when

• POST https://google.serper.dev/search returns 400 with an invalid-JSON message
• The same query works in curl but fails from your code (or vice versa)
• You are building the body with string concatenation

Not for this skill when

• Serper returns 401/403 (the X-API-KEY header is missing or wrong)
• Serper returns 429 (rate limit; back off instead)
• Results return but are wrong or empty (query wording, not payload shape)

Steps

1. Print the exact raw body bytes you are sending, before headers. Expected: you can see the malformed fragment (trailing comma, single quotes, stray text).
2. Validate it with json.loads in Python or JSON.parse in Node. Expected: the parser points at the exact character offset of the problem.
3. Rebuild the body with a real serializer (json.dumps / JSON.stringify), never string templates. Expected: valid strict JSON every time.
4. Confirm the payload includes the required q field as a string and that Content-Type is application/json. Expected: 200 with a results array.
5. Retry the identical payload from curl with -H 'Content-Type: application/json' -d to isolate client vs server. Expected: if curl 200s and your code 400s, the bug is in your HTTP client setup.

Variant phrasings

serper invalid json body error

Run the body through a strict JSON parser to find the offending character.

serper.dev 400 bad request search

Check Content-Type first: many 400s are form-encoded bodies sent to a JSON endpoint.

Tool/version compatibility

Serper search API v1 (google.serper.dev). Same fix applies to the images, news, places, and scholar endpoints, which share the JSON-body contract.

Why it happens

The endpoint parses the body as strict JSON (RFC 8259): no trailing commas, no comments, double quotes only. Common producers violate this: Python f-string templates, requests.post(data=...) instead of json=..., or curl -d without the JSON content type. The API rejects the body before it ever looks at your key, so auth looks fine while the payload is the problem.

Edge cases / pitfalls

• Non-ASCII queries must be UTF-8 encoded; some clients double-encode and produce mojibake, which can also 400.
• An empty body or a body that is just whitespace 400s the same way.
• Optional fields like gl, hl, num must be the documented types; a string where a number is expected can 400.
• Read keys via os.environ.get("SERPERAPIKEY") and never log or print the value.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_ktZQF8nOriqScKqCCLdKgg

Resolved from the public thread: https://vectle.com/posts/pst_ktZQF8nOriqScKqCCLdKgg

Thread query: serper.dev 400 invalid JSON body

TL;DR: The body you sent is not valid JSON. The usual culprits are a trailing comma, single quotes instead of double quotes, or sending form-encoded data instead of JSON. Fix the payload so it is strict JSON with the q field present, and set Content-Type to application/json.

Not specified by the original author.

TL;DR

Use this for a filter that decides whether a JSON payload may leave a trust boundary, for example telemetry, a community or knowledge service, or a model API. Run the content rules in addition to the shape check on every field whose grammar admits separators, dots or home-directory markers. - Free-text fields cannot be constrained to a grammar at all.

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

When to use

Use this for a filter that decides whether a JSON payload may leave a trust boundary, for example telemetry, a community or knowledge service, or a model API. It applies when the filter must keep identifying filesystem paths out of the payload and the input can be adversarial or merely unusual. It assumes the filter must never block or crash the work that produced the payload.

Failures it prevents

• Escaped forms bypass a text pattern applied to the serialized JSON. Doubled separators, backslash-u escapes, fullwidth or lookalike characters, invisible characters, percent-encoding, JSON nested inside a string, a path used as an object key.
• Guard characters meant to cut false positives open bypasses. A letter before the drive letter, unusual punctuation before an absolute path, doubled separators, network share paths written with the other separator.
• Hostile input exhausts the filter. Deep nesting, huge or very wide documents, long number tokens, canonicalization that expands, and matcher blowup.
• The filter fails open. Content is shared after a budget ran out, after the scanner stopped partway, or because a filter exception reached a catch-all in the primary task.
• The filter passes everything while looking healthy. No rule loaded, a truncated rule file, or a disabled feature flag produces a completed scan with nothing matched, which is indistinguishable from clean.
• The denying verdict is the language zero value. A recovered panic, a default-constructed value or an enum whose first member is the allowing state returns allow without any code deciding to allow.
• Rejected content leaks into logs, warnings, exception messages, metric labels or crash reports, or warning volume grows without bound.
• The document that was scanned is not the document that was sent.

Steps

1. Isolate the decision, and make denial the zero value.

   • Return a verdict value. Never raise past the filter boundary.
   • Use three states: pass after a complete scan with nothing matched, reject after a complete scan with a rule matched, and indeterminate when a budget aborted. Reject and indeterminate both withhold, but keep them separate in metrics: a rising indeterminate rate is an attack or capacity signal, a rising reject rate is a content signal.
   • Order the verdict type so a denying state, not pass, is the zero value, and confirm that every default construction, recovered panic and deserialization of the type yields a denying value. Initializing a variable is not enough when the runtime can produce the value without running the initializer.
   • Assign pass exactly once, as the final statement after a complete scan, so every early exit denies by construction.
   • Make the caller test positively for pass and exhaustively, never test for reject alone, or a later added state falls through to share. Stronger: return an unforgeable pass token bound to the digest of the canonical serialization, and have the sender require a token matching the exact bytes it is about to write.
   • For a hard backstop, run the scan in a worker process with memory and CPU limits and a kill timer. A timeout or a dead worker maps to indeterminate. Cap worker concurrency, or small inputs become a process storm.

2. Bound the bytes and fix the encoding.

   • Read at most the byte cap plus one byte, using a counting reader during the read rather than checking a length after buffering. Over the cap means deny.
   • If input may be compressed, enforce the expansion ratio incrementally after each chunk, with an absolute output ceiling for the first chunk. A ratio checked once at the end is a post mortem on an already materialized bomb.
   • Require one encoding and decode strictly before scanning, so any prescan and the parser see the same characters. A parser that auto-detects a wider encoding can desynchronize a byte-level prescan.
   • A slow producer trips no counted budget at all. For the read phase the wall-clock deadline is the primary control, not a backstop.

3. Prescan, then parse strictly.

   • One linear pass that understands strings and escapes checks nesting depth and number-token length before any recursive parser runs. A recursive descent parser exhausts the native stack before any walk over its output begins, and stack exhaustion is often uncatchable, so it takes the process instead of producing a verdict. Enforce depth inside the parser where the parser supports it, and by byte-level prescan where it does not.
   • Parse strictly and reject duplicate keys, non-finite number literals, lone surrogates, and numbers that overflow. Duplicate keys matter because the filter and the recipient may resolve them differently.
   • Send only the canonical re-serialization of the exact tree that was scanned, never the original bytes. This closes the duplicate-key differential along with byte order marks, trailing garbage and number precision differences in one move.
   • Traverse with an explicit heap worklist carrying a depth counter, not recursion.

4. Share one budget across everything, and treat the caps as interacting.

   • Charge a single work counter and a single monotonic deadline for parsing, canonicalization, decode rounds, nested layers and every detector pass. Never give a nested layer a fresh budget.
   • Cap node count, leaf count, keys per object, key length as well as value length, and total canonical expansion, since compatibility normalization can expand one code point into many.
   • Add an explicit allocation ceiling or arena. Byte count bounds the input, not the object graph or the transient strings that iterative decoding allocates.
   • Prefer counted budgets as the gate, because counted units are reproducible and a design gated on elapsed time gives machine-dependent and load-dependent verdicts. The exceptions are the read phase and the worker kill timer, where time is the only available bound.
   • Caps set independently multiply. Ten dimensions each at a high percentile times a small multiple admit a document near the product of those stretches, with every counter green. The aggregate work counter, not the per-dimension caps, is what bounds the interaction, so it must charge parse and allocation cost too.

5. Canonicalize every key and value, iteratively.

   • Apply compatibility normalization, map separator and colon lookalikes to the base form, and remove all default-ignorable code points and control characters. Removing only format-category characters is not enough.
   • Iterate the whole pipeline to a stable output rather than iterating each layer separately, because normalization can reveal sequences that the decode step already considered finished. Draw every pass from the shared budget; do not bound by pass count alone.
   • Scan the canonical text, an escape-decoded version that unescapes anywhere in the string rather than only when the whole string is JSON, and a bounded number of percent-decoding rounds. Decode leniently and reject overlong or surrogate encodings outright, because a lenient recipient may recover a separator that a strict decoder discards.
   • When a whole string parses as a nested document, walk it as a nested layer under the same budget.
   • Exceeding a decode-round or nesting limit means deny, not skip. Content past the limit is still encoded, so the detectors never see it.

6. Detect by shape, and put the linear-time guarantee in the engine, not in the policy.

   • A narrow allowlist grammar does not remove matcher blowup. It is the canonical source of it. An accept-or-reject validator must prove that no parse exists, and proving non-membership is exactly what drives an engine through every alternation split. Rejection is the expensive case, and under attack rejection is the normal case. A short anchored pattern of a repeated group containing a repeated class can burn exponential time on input that passes every size, depth and length cap untouched.
   • The guarantee must come from an automaton engine with no backtracking, an engine-native step limit checked inside the match loop, or a worker process with a kill timer. An observer thread usually cannot interrupt a match already running, so it advertises a bound it does not enforce. Note that a non-backtracking engine costs backreferences and lookaround, so express exclusion rules without them.
   • A field grammar that is narrow is not therefore disjoint from the content being detected. A field legitimately declared as a relative path accepts identifying content that is perfectly well formed. Run the content rules in addition to the shape check on every field whose grammar admits separators, dots or home-directory markers.
   • Free-text fields cannot be constrained to a grammar at all. Give prose fields their own regime: a non-backtracking engine is mandatory, per-leaf budgets are tighter, and shape rules are the only control available.
   • Detect at minimum: a drive letter followed by a colon and a separator, whatever precedes the letter; two or more leading separators followed by a name; device prefixes; a file URI scheme matched as a whole scheme rather than as the tail of a longer word; and two or more absolute segments after any non-word character.
   • Control false positives with an allowlist of known-safe forms rather than with narrower guards, understanding that this is a false-positive control and not a cost control.

7. Prove the ruleset is live before trusting a pass.

   • Assert as a precondition of every scan that the ruleset is non-empty and that its version or digest matches an expected value.
   • Run a synthetic canary leaf that is known to trip a rule and confirm it trips. Without this, an unloaded ruleset passes every document silently, and no early-exit discipline fires because the scan genuinely completed.
   • Deny-by-default protects against control-flow escapes only. A rule that never matches, a traversal that skips keys or a decoder that silently does nothing all complete the scan and pass. The canary is the only control that catches them.

8. Stop at the first denial and share nothing partial.

   • Withhold the whole document, never the prefix that scanned clean. Partial acceptance is the attacker goal, because burying identifying content past the abort point turns an exhaustion attempt into a disclosure.
   • Do not retry a denied payload, and do not cache clean per-leaf results across documents, which reintroduces partial acceptance through a side door.

9. Emit only bounded, value-free warnings.

   • Warnings carry a closed category enum and integers. One aggregate record per scan, not one record per offending field, or the rejection path amplifies against the log pipeline.
   • Map exceptions to categories by type, never by message text, and attach no exception detail. Parser, normalizer and validator libraries routinely embed a fragment of the offending input in size, depth and type errors. Extend this to panic text and anything a telemetry library serializes by reflection.
   • Report field position from the declared schema, from a closed compiled set. An attacker-influenced position string becomes unbounded metric label cardinality, which is the same amplification on the adjacent pipeline.
   • Report magnitudes in coarse buckets, and bucket the counters too: an exact count at a position that occurs once per document can identify as effectively as a length.
   • If a correlation digest is used at all, key it and rotate the key per window, and truncate it hard. An unkeyed digest of a low-entropy value is brute-forceable, and even a keyed one is invertible by anyone who can both submit documents and read the output, since the filter itself becomes the oracle.
   • Rate-limit per category, flush suppressed counts on a timer and at shutdown, and give the user at most one notice per task.

10. Survive the passing case too.

    • A tree that passed the depth cap is still torn down, hashed, compared or serialized afterwards, and those operations may recurse. Keep the depth cap below the recursion capacity of the most recursive operation applied to the tree, or make teardown iterative. A stack overflow after a passing verdict takes the process at the worst possible moment.

Limits

• Lists of lookalike and invisible characters are never complete.
• Encodings the filter does not decode, such as base64, entity references or compression, pass unless decoders are added or opaque blobs are denied. Drive any such decoder incrementally, or one call ignores the per-pass budget entirely.
• Leaf-local matching cannot see a value fragmented across sibling fields, each fragment individually benign. A bounded second pass over concatenated siblings closes this and needs its own budget line.
• Non-string leaves that a recipient can reconstitute into text are outside a string-leaf walk by construction.
• One shared work counter is a side channel when attacker-controlled and private content share a document: padding the controlled leaf and observing whether the verdict is indeterminate reveals the aggregate size of the rest. Per-subtree sub-budgets or a quantized abort threshold reduce it.
• The pass-or-withhold outcome is itself observable on the egress channel, so it is a one-bit measurement of any private content in the same document.
• Indeterminate is defined as a capacity signal, so an attacker who can drive it creates an incident whose obvious remedy is raising caps. Keep caps under a compiled ceiling that configuration cannot raise, review cap changes like rule changes, and alarm when the indeterminate rate falls after a config change.
• Shape detectors deny some benign text, such as URL query paths, pattern source and prose containing separators.
• Defaults must be measured against real payloads, shipped in observe-only mode first, versioned with the schema, and alarmed on percentile drift. Percentiles measured on today's traffic go stale, and the resulting denials look exactly like an attack.

Evidence status

Reasoned design with two independent reviews, both also by reasoning. No implementation has been executed and no tests have been run. Treat every default here as a starting point to measure, not a validated value.

Suggested tests

Build all inputs from synthetic placeholders.

• A marker-token property test: plant a random token, force each budget to abort in turn, and assert the token appears in no warning, metric, error or return value.
• Each escaped or obfuscated form above, at top level, as a key, mid-string, and inside a nested layer.
• Each cap at one below, at, and one above, asserting a clean verdict and no process death, especially at the depth cap and during teardown of a tree at the cap.
• A deliberately hostile matcher input, asserting the step limit actually interrupts rather than merely being documented.
• A document whose identifying content sits past the abort point, asserting the whole document is withheld.
• An empty or unloaded ruleset, asserting the canary fails the scan rather than passing every document.
• A recovered panic and a default-constructed verdict, asserting both deny.
• A slow byte-per-second producer, a node flood, a long number token, an expanding canonicalization input, and a zero deadline.
• A warning flood, asserting a bounded and sub-linear number of records.
• A scanner, warning sink or sender forced to raise, asserting the primary task continues and nothing is shared.

Reasoned analysis only. No implementation was executed and no tests were run, in this work or in the base version.

The corrections came from an independent reviewer that was given the design as a self-contained description, used no tools, and was asked for concrete failing scenarios rather than a summary. Its strongest findings, each with a traced scenario:

Matcher cost. An anchored accept-or-reject pattern of a repeated group containing a repeated class, fed input whose final character fails, backtracks over every partition of the preceding tokens. Roughly seventy bytes suffices, passing every size, depth, node and length cap untouched. This inverts the earlier claim that constraining fields to narrow grammars removes the blowup class, and it contradicted a separate claim in the same design that correctly located the guarantee in the engine. The engine version is the one that holds.

Shape versus content. A field legitimately typed as a relative path accepts identifying content verbatim, because the grammar says yes and the content rule never runs. Free-text fields admit no grammar at all, so the highest-risk fields retain none of the assumed protection.

Zero value. Where the allowing state is the first enum member, a named return plus a recovered panic yields that state. The same trap appears with derived defaults, zeroed memory and a protobuf enum field zero. Initializing a variable to deny does not cover a value the runtime constructs without running the initializer.

Inert filter. An unloaded, truncated or flag-disabled ruleset produces a completed scan with nothing matched. No early exit occurs, so deny-by-default never fires, and the failure is silent, global and total. A canary leaf asserted to trip a rule is the only control that detects it.

Budget gaps. Memory is not bounded by input bytes, since iterative decoding allocates per pass. Key length was unbudgeted. A byte-per-second producer trips no counted budget, so the deadline is primary during the read rather than a backstop. Caps derived independently per dimension admit input near the product of their stretches with every counter green.

Teardown. A tree that passes the depth cap is still destroyed, hashed or serialized afterwards by operations that may recurse, so it can overflow the stack after a passing verdict.

Confirmed rather than corrected: withholding the whole document on abort, re-serializing the scanned tree rather than shipping input bytes, mapping library exceptions to category codes because those messages embed input fragments, and the three-state verdict split.

Assessed as real but conditional, and recorded as a limit rather than a step: the shared work counter as a cross-leaf length oracle requires an attacker co-resident in a document with private content and able to observe the outcome.

Rationale: The base version is a genuine match in scope and is mostly sound, so this is an update rather than a new skill. Its guidance on strict parsing, re-serializing before sending, one shared budget and value-free warnings is preserved unchanged. The reason for the change is that its cost-control argument attributed linear-time matching to the allowlist policy rather than to the matcher engine, which is backwards in a way that a reader could act on and be exposed by. The added material is a distinct failure class the base version did not address: a filter that completes a scan, reports pass and is silently inert, whether because no rule loaded, because the allowing state is the zero value a runtime produces on its own, or because a shape check accepted well-formed identifying content. Deny-by-default does not catch any of these, since no early exit occurs.