Design question about an outbound privacy filter that scans JSON for identifying filesystem paths. The known weakness is that a regex written for ordinary text is applied to the serialized JSON, where a Windows path has its backslashes doubled by JSON escaping, and unicode escapes or separator substitution can further change the bytes so the pattern never matches. The question is how to structure validation so it operates on decoded string values rather than serialized bytes, catches both Windows and POSIX style paths plus common encodings, and reports rejections with only metadata such as JSON pointer, rule name and a hash or length, never the matched substring. Unknowns are how far to normalize before matching, how to bound cost on deeply nested or large documents, and how to test the rejection path for leaks. This is a pure reasoning exercise with no repository involved.
Validating decoded JSON string values for filesystem paths without logging rejected content
Resolution, from reasoning only, no tests executed. The escaped Windows path evasion is a layer mismatch: the pattern was written for decoded text but is run over serialized bytes, where each backslash appears as a pair and may also appear as a unicode escape. Patching the pattern for the doubled form leaves percent encoding, nested serialized documents and base64 open. The fix is to parse the document, walk the tree without recursion, apply rules to each decoded string value, and then emit only the canonical serialization of the tree that was walked, so the validated value and the emitted value are the same bytes. Matching happens on a match-only normalized copy: fold lookalike separators, strip format controls first, run named decoders for a bounded number of rounds, and treat any stripped prefix such as a drive letter, network share prefix, file scheme or home shorthand as a match by itself. Score structural signals per whitespace token rather than per string so mid sentence embedding does not evade start anchors. Rejection records carry a schema derived field pointer, a rule identifier from a closed enumeration, a coarse length bucket and a keyed hash under a rotating secret never stored with the log; budget exhaustion records carry only the category and a coarse budget fraction, no field pointer. Parser and matcher error messages are mapped to local codes and their text discarded. Verdict initializes to deny and becomes allow only after a completed clean walk; any abort withholds the whole document. Testing the rejection path: plant a random marker in synthetic payloads, drive every abort and reject path, and assert the marker never appears in any log, metric label, typed result or caught exception, while pairing each identifying fixture with a benign control that must be allowed. The two skills read during this task already state all of this, including the corrections about token level scoring, invisible characters, prefix only values and value free budget warnings, so no new skill is proposed.
Follow up finding on resource exhaustion, reasoning only, nothing executed. Two additions beyond the existing skills. First, an operational supervisor model: the filter runs in a separate process with operating system memory and CPU limits plus a supervisor held wall clock deadline, because a thread stuck in pathological backtracking cannot be interrupted safely. The supervisor sets the verdict to deny before spawning, waits for a typed result, the deadline, or process exit, and assigns allow only from a typed allow message carrying canonical bytes. Deadline and crash each map to their own category with the verdict already deny. The supervisor captures only the exit status and never forwards worker standard error, since a panic message or stack trace can embed the subject string, and that leak sits outside the worker's own value free logging. The primary task continues because sharing is a side effect whose default is nothing sent. Second, a fixed slot counter rule for warning cardinality: every warning is an increment at a slot from a compile time enumeration, with the slot space equal to the product of a category enumeration and a bucket enumeration, roughly seven budget categories times three fraction buckets plus deadline, crash, parse, one slot per content rule, and one truncated slot. No document string, key text, field pointer or engine error text can become a label, so ten thousand documents with distinct keys create no new series, and one aggregate record per pass with capped per slot counts bounds emission volume. Caveat: counted budgets are deterministic, the deadline is not, so determinism tests must disable or virtualize the clock and the deadline rate should be read as host pressure rather than attack signal.