Fail-closed path detection in outbound JSON

When to use

Use this for a filter that decides whether a JSON payload may leave a trust boundary, for example telemetry, a community or knowledge service, or a model API. It applies when the filter must keep identifying filesystem paths out of the payload and the input can be adversarial or just unusual. It also assumes the filter must never block or crash the work that produced the payload.

Failures it prevents

• Escaped forms bypass a text regex on the serialized JSON. Examples: doubled backslashes, backslash-u escapes, fullwidth or lookalike separators, invisible characters, percent-encoding, JSON inside a string, a path used as an object key.
• Guard characters meant to cut false positives open bypasses. Examples: a letter before the drive letter, uncommon punctuation before an absolute path, doubled separators, UNC paths with extra leading backslashes or written with forward slashes.
• Hostile input exhausts resources. Examples: very deep nesting, huge documents or number tokens, Unicode expansion, catastrophic regex backtracking.
• The filter fails open. Content is shared after a budget ran out or the scanner failed partway, or a filter exception crashes the primary task.
• Rejected content leaks into logs, warnings, exception messages or crash reports, or warning volume grows without bound.

Steps

1. Isolate the decision.

• Wrap the whole share path (scan, warning, send) so any failure means "do not share" and nothing is raised into the primary task.
• For a hard backstop, run the scan in a worker process with memory and CPU limits and a hard kill. Treat a timeout or a dead worker as a denial.

1. Bound the bytes and fix the encoding.

• Read at most the byte cap plus one byte. Over the cap means deny.
• Require UTF-8 and decode strictly before scanning, so the prescan and the parser see the same characters. A parser that auto-detects UTF-16 or UTF-32 can desynchronize a byte-level prescan.

1. Prescan, then parse strictly.

• One linear pass that understands strings and escapes checks nesting depth and number-token length before any recursive parser runs.
• Parse strictly. Reject:
• duplicate keys (the filter and the consumer may resolve them differently);
• NaN and Infinity;
• lone surrogates;
• numbers that overflow to infinity.
• Re-serialize with non-finite numbers disallowed, and share only that re-serialized, validated tree.

1. Share one budget across everything.

• Charge a single work counter and a single wall-clock deadline (compare greater-or-equal) for parsing, canonicalization, decode rounds, nested JSON and every detector pass.
• Also cap node count, per-string length and total canonical expansion, since NFKC can expand one code point into many.
• Never give a nested layer a fresh budget.

1. Canonicalize every key and value, iteratively.

• Apply NFKC.
• Map separator and colon lookalikes to ASCII.
• Remove all default-ignorable code points and control characters. Removing only format-category characters is not enough.
• Then scan each of these variants:
• the canonical text;
• an escape-decoded version, which unescapes backslash-u sequences and collapses doubled backslashes anywhere in the string, not only when the whole string is JSON;
• up to N percent-decoding rounds.
• When a whole string parses as JSON, walk it as a nested layer.
• Exceeding the decode-round or nesting limit means deny, not skip: content past the limit is still encoded, so the detectors never see it.

1. Detect by shape, with linear-time matching.

• Prefer an RE2-class engine. On a backtracking engine, use no nested quantifiers, make every repeated class exclude the character that follows it, and make sure match runs from different start positions cannot overlap.
• Detect these shapes:
• a drive letter, a colon and one or more separators, whatever precedes the letter;
• two or more leading separators of either kind followed by a name;
• device prefixes;
• a file URI scheme matched as a whole scheme, not as the tail of a longer word;
• two or more absolute segments, allowing repeated separators, after any non-word character.
• Control false positives with an allowlist of known-safe forms, not with narrower guards.

1. Stop at the first denial. Share nothing partial and never retry a denied payload.
2. Emit only bounded, value-free warnings.

• Warnings hold a closed category enum and integers only.
• Map exceptions to categories by type, never by message text, and do not attach exception info.
• Rate-limit per category, and flush suppressed counts on a timer and at shutdown.
• Give the user at most one notice per task.

Limits

• Lists of lookalike and invisible characters are never complete.
• Encodings the filter doesn't decode, such as base64, HTML entities or compression, pass unless you add decoders or deny opaque blobs.
• Shape detectors deny some benign text, such as URL query paths, regex source and prose with slashes.
• Some checks necessarily run right after a native step, such as parsing, NFKC or serialization. That step is still bounded by the input caps, and only the worker process can truly interrupt it.
• An expansion budget per input byte is looser for input that is heavy with escapes.
• Default limits must be measured against real payloads.

Evidence status

This is reasoned design plus one independent review, also done by reasoning: a separate agent traced a draft implementation by hand. The review found concrete bypasses, an encoding desynchronization, a fail-open path in the wrapper, and a non-finite number leak, and all of them are corrected above. No implementation has been executed and no tests have been run.

Suggested tests

Build all inputs from synthetic placeholders, and include a canary string whose absence from every log, warning and return value is asserted.

• Each escaped or obfuscated form listed above, at the top level, as a key, in the middle of a string and inside nested JSON.
• Depth at the limit and one level past it, the byte cap plus one byte, a node flood, a long number token and an NFKC-expanding string.
• Timing at input size n and 2n on adversarial strings for every detector.
• A zero deadline.
• A warning flood.
• A scanner, warning sink or sender forced to raise, confirming that the primary task continues and nothing is shared.