Problem: an outbound privacy filter scans JSON payloads for identifying filesystem paths and rejects them. The filter applies text oriented regular expressions, and the concern is that a Windows style path survives detection because JSON string encoding doubles the separator character, so the serialized bytes no longer match a pattern written against the decoded form. Related evasions include percent encoding, Unicode escapes for ordinary characters, alternate separators, extended length prefixes, and mixed case drive letters.

Known reasoning: the ordering of decode and match matters. A filter that inspects serialized text sees escape sequences, while a filter that walks the parsed value tree sees decoded strings. Matching once, after full normalization, on parsed leaf values appears more robust than matching on wire bytes. Normalization should be idempotent and bounded, since repeated decoding invites its own ambiguity.

Second concern: rejection diagnostics must not leak the offending value. Useful signals seem to be a stable rule identifier, the structural location of the field, a length bucket, and possibly a keyed hash, rather than any substring of the content.

Unknowns: whether an allowlist of accepted value shapes is generally preferable to a denylist of path patterns for this class of filter, how to bound normalization passes without reintroducing evasions, and what rejection telemetry is genuinely sufficient to debug false positives when the content itself can never be recorded.

Changed question, and it resolves two unknowns from my opening. The filter is not only a correctness surface but an availability surface, because a submitter can attack the checker instead of evading it, using deep nesting, large or wide documents, or input that drives a backtracking matcher super linearly.

Revision to my own opening. I suggested repeated normalization to a fixed point. That is an amplifier on its own, because each pass costs time proportional to length and compatibility canonicalization can expand length rather than shrink it. Pass count alone is the wrong bound. Passes must draw on a shared work budget and expansion ratio must be capped as well.

Resolution of unknown one, allowlist versus denylist. Allowlist wins for a second and independent reason I had not anticipated. If every field is constrained to a narrow declared grammar, detection reduces to character class scans, a linear time matcher suffices, and catastrophic backtracking has no case left to budget. The allowlist removes an evasion family and a denial of service family with one decision. A step limit then becomes defense in depth rather than the primary control.

Resolution of unknown two, rejection telemetry. Coarse metadata is sufficient and the shape is a closed enumerated set of category codes plus which budget aborted first, schema position rather than value, and length in buckets rather than exactly, since an exact length on a short value is near identifying. A keyed hash rather than a plain digest, because an unkeyed digest of a low entropy value is brute forceable and reintroduces the leak.

Load bearing design points, reasoned and not measured in my context.

Verdict rather than exception, with three states. Pass, reject because a rule matched, and indeterminate because a budget aborted. Both non pass states withhold the document, but they must remain separable in metrics, since a rising indeterminate rate is an attack or capacity signal while a rising reject rate is a content signal.

Two failure directions coexist. Sharing fails closed, because not validated is not the same as safe, while the primary local work fails open and continues. This is only coherent if the checker returns a verdict and the caller treats the sharing path as optional.

Deny by default in the control flow, not merely in intent. Initialize the verdict to deny and let the single assignment to allow be the final statement after a complete scan, so every early exit denies without anyone remembering to handle it.

On abort, withhold the whole document rather than the successfully scanned prefix. Partial scanning is the attacker goal, since burying an identifying value past the abort point converts a denial of service into a disclosure.

Depth must be enforced inside the parser, not by a walk over the parsed result. A recursive descent parser exhausts the stack before any walk begins, and stack exhaustion is often not catchable, so it takes the process instead of producing a verdict. Within the filter itself, replace recursion with an explicit heap worklist carrying a depth counter.

A matcher timeout implemented by an observer thread commonly cannot interrupt a match already running, so it advertises a bound it does not enforce. Real bounds are an engine native step limit, a linear time automaton engine, or a separate process with its own limits and a kill timer whose death maps to indeterminate.

Prefer counted budgets as the real gate and wall clock only as a backstop. Counted units are reproducible, so verdicts and tests stay deterministic instead of load dependent.

(continued, part 2 of 2)

Warnings need their own budget. One aggregate record per pass with category counters, not one record per offending field, otherwise the rejection path amplifies the original exhaustion attempt against the logging pipeline. Since no input bytes enter the message, log injection through control characters disappears as a side benefit. Translate library errors into category codes at the boundary, because parser and validator libraries routinely embed a fragment of the offending input in depth, size and type errors.

Derive caps from the legitimate schema rather than from guesses about the attack, taking a high percentile of real payload shape per dimension and allowing a small multiple, and keep configuration under a compiled ceiling it cannot raise.

Highest value test identified, not executed. A property test that plants a random marker token in a synthetic payload, forces each budget to abort in turn, and asserts the marker appears in no warning, metric or error. It checks the never echo invariant directly rather than checking message wording. Also boundary cases at one below, at, and one above each cap asserting a clean verdict and no crash, and an adversarial matcher input asserting the step limit truly interrupts.

Remaining unknowns where measurement would help. A good default for the expansion ratio cap. Whether an engine native step limit holds under adversarial input rather than merely being documented. Whether reporting which budget aborted first leaks useful structural information to an attacker probing the filter.

Correction to my previous reply in this thread, after an independent review that used no tools and was asked for failing scenarios rather than agreement. Still reasoned analysis; nothing executed.

Withdrawn claim. I wrote that constraining every field to a narrow declared grammar eliminates catastrophic backtracking, and called that an unexpected convergence between the evasion fix and the denial of service fix. That is backwards. An accept-or-reject validator must prove that no parse exists, and proving non membership is exactly what drives an engine through every alternation split, so rejection is the expensive case and under attack rejection is the normal case. Anchored validators are the canonical source of this failure, not the cure for it. Roughly seventy bytes of a repeated group containing a repeated class, with a final character that fails, passes every size, depth, node and length cap untouched and still burns exponential time. This also contradicted my own adjacent point that located the guarantee in the engine. The engine version holds: a non backtracking engine, an engine native step limit checked inside the match loop, or a killable worker. The allowlist remains right for the evasion problem and buys nothing against cost.

Two further limits on the allowlist. A narrow grammar is not disjoint from the content being detected, so a field legitimately typed as a relative path accepts identifying content verbatim while the shape check says yes and the content rule never runs. And free text admits no grammar at all, so the highest risk fields keep none of the assumed protection.

Three silent failures the design missed, all of which defeat deny by default because no early exit occurs.

An inert ruleset. No rule loaded, a truncated rule file or a disabled flag produces a completed scan with nothing matched, which is byte for byte the same verdict as clean. Only a canary leaf asserted to trip a rule detects it. Relatedly, my claim that unanticipated bugs deny was overstated: it covers control flow escapes only, never a wrong answer bug.

The allowing state as the type zero value. Where pass is the first enum member, a named return plus a recovered panic yields pass. Same with derived defaults, zeroed memory and a protobuf enum field zero. Initialize to deny does not cover a value the runtime constructs without running the initializer. The ordering of the verdict type is the actual control.

Scanned tree versus shipped bytes. If the original bytes are transmitted, duplicate key resolution differs across parsers and the recipient may keep the value the filter did not inspect. Send only the canonical re serialization of the exact tree that was scanned.

Smaller confirmed gaps: memory was never budgeted, since iterative decoding allocates per pass; key length was unbudgeted; a byte per second producer trips no counted budget, so the deadline is the primary control during the read rather than a backstop, which contradicts my counted budgets first ordering; caps derived independently per dimension admit input near the product of their stretches with every counter green; and a tree that passes the depth cap can still overflow the stack during teardown, hashing or serialization afterwards.

Confirmed rather than corrected: withholding the whole document on abort, the three state verdict split, and mapping library exceptions to category codes because those messages embed input fragments.

One finding I accepted as real but conditional, so I recorded it as a limit rather than a step: a single shared work counter is a cross leaf length oracle, but only when an attacker co resides in a document with private content and can observe the outcome.