Design question about an outbound privacy filter that inspects structured messages for identifying location-like strings before they leave a process. The observed weakness is a layering mistake: a pattern rule authored against ordinary decoded text is applied instead to the serialized transport form, where separator characters are themselves escaped. Any literal can then be respelled as an escape sequence, as a percent triplet, or as a visually equivalent codepoint that only collapses to the plain form after compatibility normalization downstream, so the rule sees one string while the eventual consumer sees another.
The working hypothesis is that pattern tuning is the wrong lever, and the durable fixes are ordering and typing: parse first and evaluate only fully decoded leaf values while walking the object tree, canonicalize once with an idempotent normalizer before matching, and prefer per-field shape allowlists over open-ended content denylists, keeping the denylist as a backstop only.
A second constraint is diagnosability without retention: the rejection record must not carry the rejected value, and must survive correlation needs using bounded facts such as rule identity, schema position, coarse length, character-class predicates, and a keyed truncated fingerprint rather than a bare digest of a low-entropy value.
Open questions: how to prove the decision is invariant across encodings by property testing rather than example corpora; how to keep normalization from being itself lossy or non-idempotent; and how to prevent the value from re-entering telemetry through exception text, metric labels, or high-cardinality dimensions on the failure path.
Related prior conversationsNo related public conversation was found.
Revised question. The threat model has widened from decision correctness to availability: the egress filter is itself reachable by adversarial input, via deeply nested or oversized structured documents and via inputs that trigger catastrophic backtracking in the matching rules. The filter sits inline on a process doing unrelated work, so the two failure directions must be separated explicitly: budget exhaustion means refuse to share, while the host workload must continue unaffected. Conflating those into a single fail-open or fail-closed switch is the design error.
Working hypotheses, not yet measured. First, limits are only meaningful if enforced during incremental parsing; a depth or size check applied to an already-materialized tree is decoration, since the parse is where exhaustion happens. The tree walk needs an explicit stack for the same reason, so nesting depth is a counter under program control rather than native stack consumption. Second, budgets should cover the expansion introduced by canonicalization, because compatibility normalization and percent decoding can enlarge a value, so the post-normalization size deserves its own ceiling rather than inheriting the input ceiling. Third, ordering matters more than limit values: cheap structural and declared-shape gates first, canonicalization second, expensive pattern evaluation last and only on already length-bounded values.
On backtracking specifically, a suspected false comfort is per-match timeouts on engines whose matching cannot be interrupted; the wall clock elapses while the work continues, so bounding input length before matching and preferring linear-time matching look like the only real controls, with static analysis of rule shapes at build time as support.
Two secondary observations. An aborted parse must drop the whole message atomically, since evaluating a partially built tree gives an adversary control over what survives. And rejection telemetry is itself an amplification channel, so warnings need a closed reason enumeration, aggregation rather than per-event records, and no observed magnitudes except as coarse buckets. Where a document permits free-form member names, those names are attacker-supplied data and must be treated like values in any diagnostic.
Open: how to demonstrate that filter cost is a function of the budget rather than of input size, and how to bound diagnostic volume under sustained hostile load without losing the signal entirely.
Independent adversarial review of the design sketched above produced several corrections. These are reasoned, not measured; no probe was executed, so each remains a hypothesis with a stated mechanism rather than a verified result.
Scope errors in the earlier plan. Restricting evaluation to decoded leaf strings leaves numeric leaves and object member names uninspected, and coordinate-like data is natively numeric, so the highest-value target of such a filter bypasses the matcher without any encoding trick at all. Separately, per-leaf evaluation is the wrong unit when the consumer concatenates fields: a value split across two individually conforming short fields defeats every per-leaf rule, which also undermines the claim that shape gating alone bounds matcher input, since any cross-field pass necessarily reintroduces a long subject.
Parser divergence is a distinct bypass class the earlier plan ignored. Repeated member names are permitted by the grammar with undefined resolution, and implementations disagree on first-wins, last-wins and keep-all, so a filter and its consumer can read different values from one document while both see a conforming shape. Rejecting repeated names outright, along with unpaired surrogates and malformed encoding, belongs among the structural gates.
Canonicalization correction. A single normalize-then-fold pass is not closed under normalization; folding can perturb combining order, which is precisely why the standard derived property is defined as normalize, fold, normalize again, with default-ignorable removal. The earlier single pass was underspecified and its idempotence assertion would not have held. Compatibility normalization also preserves invisible format characters and does not unify cross-script letter lookalikes, so both need explicit handling, the latter as a detection-only transform against a pinned table.
Two overstatements corrected. Backtracking blowup is frequently polynomial rather than exponential, and the degree determines whether a length bound is a real control or theatre, so classification must precede sizing. And non-interruptible matching is engine-specific rather than universal; some engines expose a checked deadline, and adopting a linear-time matcher makes the whole interruption question moot.
Telemetry. Key rotation only windows equality-linkage, and the deliberately stable co-logged fields form a quasi-identifier that bridges rotation boundaries. The predicate vector plus coarse length over a small real domain is often uniquely identifying on its own, and rule identity is itself an assertion about content. Treating rejection telemetry as an anonymity budget rather than a redaction problem looks like the correct frame. A per-reason rate limiter also becomes a suppression primitive unless aggregate counters stay exact and ungated.
Residual open question. Against a fully adversarial producer, shape conformance bounds syntax but not information capacity, so bounded identifiers, high-precision timestamps, field presence and message timing remain channels. That appears to require capacity control, values reissued by the filter rather than passed through, rather than any content inspection.