Problem: an egress privacy filter scans serialized JSON for identifying filesystem paths with a regex written for ordinary text. In the serialized form, a Windows-style path has doubled backslashes (and may use \uXXXX escapes for separators, drive colons or letters), so a pattern expecting a single backslash separator never matches, and the value gets through.
Known evidence (reasoning): the regex is looking at the wrong layer. JSON escaping is a transport encoding. The meaning is in the decoded string values. Keys also count as strings. Nested JSON-in-a-string, URL-encoding and alternative separators (forward slash, UNC prefixes, extended-length prefixes) add more layers to consider.
Proposed approach: parse strictly (reject duplicate keys, lone surrogates and invalid escapes), walk every key and value, apply bounded canonicalization (Unicode NFKC, mapping fullwidth or lookalike separators to ASCII, limited-depth percent-decoding and nested-JSON decoding), then run path detectors on the canonical form. Fail closed on anything that cannot be parsed or decoded within limits. Rejection telemetry should record only a JSON-pointer-like location with array indices, a rule id, the value length and a keyed HMAC for dedup. It should never record the value, a prefix of it or an unkeyed hash.
Unknowns: how far to canonicalize before false positives on ordinary prose containing slashes become costly; whether to treat keys differently from values; whether a keyed digest of rejected values is acceptable at all given low-entropy inputs; and how to test the filter without putting realistic paths in fixtures.
Changed question: the same egress filter now has to survive hostile inputs, meaning very deep nesting, very large documents, and strings chosen to cause catastrophic regex backtracking. Sharing must fail closed, and the primary work that produced the payload must never be blocked or crashed.
Design hypothesis. This is reasoning only and has not been verified by executed tests.
- Check each budget before the costly step it protects. Read at most the byte cap plus one byte, then stop. Run a linear, string-aware bracket-depth prescan before handing anything to a recursive parser. Cap per-string length, number-token length and total node count as well.
- Use one shared work counter (fuel) and one wall-clock deadline across every layer: parsing, Unicode canonicalization, percent-decoding rounds and nested JSON-in-string decoding. If each nested layer got a fresh budget, nesting would multiply the cost.
- Budget NFKC expansion explicitly, because some code points expand to many characters.
- Exceeding a decode or nesting limit must deny, not skip. A path nested one layer past the limit is still escaped in its raw form, so text rules never see it.
- Prefer a linear-time regex engine. If you use a backtracking engine, keep the patterns free of nested quantifiers, and make each repeated class exclude the delimiter that follows it, so every match attempt ends at the next separator or whitespace. Also run a timing test at input size n and 2n.
- Stop at the first denial, which gives both a bounded worst-case cost and a bounded warning count. Share nothing partial.
- The scanner returns a verdict and never raises. Map exceptions to a closed category enum by type, never from the exception message, since some parsers include input snippets there.
- Warnings are fixed-format strings made only of enum values and integers. Rate-limit them per category and periodically emit a count of suppressed warnings. Nothing derived from the input is logged.
- Use a separate worker process with memory and CPU rlimits and a hard kill as the backstop for native stack overflow or anything the cooperative budgets miss.
Open points: realistic default limits, and whether a process boundary is worth its latency when the in-process budgets are proven tight.
Correction after an independent review of the previous hypothesis. The review was done by reasoning only: a separate agent traced the draft implementation step by step, and nothing was executed.
Refuted or incomplete claims:
- The preceding-character guards on the path detectors created bypasses. Each of these passed:
- a drive path directly after a letter or digit;
- an absolute path after uncommon punctuation such as a colon, bracket, pipe or code-span marker;
- doubled slashes between segments;
- UNC paths with three or more leading backslashes;
- UNC paths written with forward slashes.
Fix: detect by shape with minimal guards, accept more false positives, and use an allowlist for known-safe forms instead of weaker guards.
- Escape sequences were decoded only when an entire string value was JSON. A JSON-escaped path in the middle of a string, or inside inner JSON that failed strict parsing, passed. Fix: also scan an escape-decoded variant of every string.
- Removing only format-category characters missed other default-ignorable code points, such as the combining grapheme joiner, variation selectors and Hangul fillers, and it missed control characters. Fix: strip the full default-ignorable set and controls, and extend the lookalike map for separators and colons.
- The byte-level prescan assumed an ASCII-compatible encoding, but the parser also accepted UTF-16 and UTF-32, so the prescan could lose track of string boundaries. Fix: require UTF-8, decode strictly first, and prescan the decoded text.
- The claim that every budget is checked before the step it protects was overstated. The node count, Unicode expansion, the nested-layer limit and output size were each checked just after a native step. That step is still bounded by the input caps, but the claim should be stated honestly, and a worker process with a hard kill remains the backstop.
- The wrapper did not catch exceptions from the warning sink or the share call, so a failure there could reach the primary task. Fix: wrap the whole share decision, not just the scan.
- An overflowing float literal parsed to infinity and would have been re-serialized as a non-standard token. Fix: reject non-finite numbers at both parse and serialize time.
- Smaller issues:
- the deadline comparison should be greater-or-equal;
- suppressed-warning counts need a timer or shutdown flush;
- depth was off by one across nested layers;
- matching the file scheme as a bare substring flagged longer words that end in the same letters.
The same review confirmed:
- one shared budget across all layers;
- deny rather than skip when a decode or nesting limit is exceeded;
- linear-time matching for the patterns, because match runs from different start positions do not overlap;
- stopping at the first denial;
- mapping exceptions to categories by type;
- log lines made only of enum values and integers.