Design question. An outbound privacy filter scans serialized JSON for identifying filesystem paths using a regex written for ordinary text. A Windows path inside a JSON string is serialized with doubled backslashes, and could also appear with unicode escapes for the backslash or colon, so the text regex misses it or matches inconsistently. Two sub-questions. First, where in the pipeline should path detection run so the input is the decoded string value rather than the escaped wire form, and what normalizations are needed before matching, such as unicode escapes, forward versus back slashes, UNC prefixes, and file URIs. Second, how should rejection be reported without writing the rejected value to logs, for example reporting only the JSON pointer to the offending field, a rule identifier, and the value length or a keyed hash, so operators can debug without the filter itself becoming a leak. Known evidence: regexes applied to the escaped wire form are brittle because the same value has many equivalent encodings. Unknowns: whether the community has a settled pattern for structured walk plus decode-then-match, and for privacy-safe rejection diagnostics.
Validating JSON string values for filesystem paths when escaping defeats text regexes
Resolution by reasoning only, no executed tests. The defect is a layer mismatch, not a weak pattern: JSON serialization has many equivalent encodings of one decoded string, such as a doubled backslash, a unicode escape for the backslash or colon, an escaped forward slash, or a document nested inside a string value. A regex that expects a literal separator consumes the first backslash of a pair and then fails on the second, so no backslash separated path matches on the wire, while the same doubling causes false positives elsewhere. Approach: parse the document, reject duplicate keys and enforce depth and size limits, walk every decoded string including keys, apply a bounded convergent normalization before matching, and re-serialize from the validated tree so the transmitted bytes are exactly what was checked. A boolean verdict that lets the caller reserialize its own copy reopens the bug. Normalization should be a fixed number of rounds, treating still-decodable output as a rejection, should fold separators to one form, strip file URI and network prefixes, and drop invisible characters. Prefer a per-field positive grammar for structured fields and reserve the path detector for free text. Rejection diagnostics: record a schema derived field identifier or JSON pointer, a rule identifier, a coarse length bucket, a character class summary, and a keyed hash under a rotating secret rather than a plain digest, since short strings are brute forceable. Never record substrings, match offsets with context, or the value in exception messages, because exceptions surface in crash reports. Fail closed on parse failure or budget exhaustion by dropping the whole document, never a partial pass. Tests should generate fixtures in process and assert booleans, so a failing assertion never prints the candidate. The three existing skills matched at task start already document this pattern, so no new skill is warranted.
Changed question, reasoning only, no executed tests. The threat model now includes hostile depth, hostile size, and matcher inputs that trigger pathological backtracking. Finding: independent per-dimension limits are not enough, because each can be under its cap while the product of depth times width times per-string matcher cost still explodes. Use one shared work ledger, decremented by every unit of work in every stage: bytes consumed by the tokenizer, nodes created, strings walked, normalization rounds, and matcher steps. Order the gates cheapest first: a byte cap on the serialized input before parsing, then an iterative tokenizer with an explicit depth counter so a deep document cannot overflow the host stack, then the tree walk, then normalization, then matching. Replace or wrap the backtracking matcher: prefer a linear-time engine or a hand-written structural scanner that counts separators and segments in one pass, cap the per-string length passed to any matcher, and treat a wall-clock deadline as a secondary guard only, since thread interruption is unreliable. Distinguish two senses of failing closed. Fail closed for sharing: any exhausted budget or parse error rejects the whole document, never a partial pass, because whoever forces the abort could otherwise place the sensitive value after the abort point. Fail open for the host: the filter must return a typed outcome, allowed with canonical bytes or rejected with a category, and must never throw, hang, or block the primary work. Diagnostics: a fixed enum of rejection categories such as size, depth, work, parse, duplicate key, and content match, with a field identifier, a coarse budget fraction bucket, and a keyed hash for content matches. Bound the warnings themselves: at most a small fixed number per category per window, then a single suppressed count, so a flood of hostile documents cannot become a log volume attack. Never include partial scan state in a budget-exhaustion warning, since unvalidated partial results are a second leak channel. The existing skills already cover the shared ledger and the two senses of failing closed, so this remains duplicate guidance and no new skill is warranted.