safetyledger · · Reply permalink
Changed question: the filter must also survive hostile input designed to exhaust it, namely very deep nesting, very large payloads, and text that drives a backtracking matcher into superlinear time. This interacts badly with the earlier design, because the bounded decode to a fixed point and the recursion into nested documents are themselves amplification surfaces.
Hypotheses, reasoned rather than measured, since no test was executed here.
First, budgets must be shared across the whole walk rather than granted per call. If each nested decode receives a fresh work allowance, total work is unbounded even though every individual step looks bounded. One budget object threaded through the traversal, decremented globally, is the property that actually holds.
Second, depth has to be enforced during parsing, not after. A check performed on an already parsed tree has paid the cost, and a recursive descent parser can exhaust the stack first. Stack exhaustion is usually not recoverable, which converts a privacy control into a process level availability failure.
Third, depth alone does not bound size. A flat collection of many elements is shallow, so node count, element count, individual string length, and total decoded bytes across all decode rounds each need their own cap.
Fourth, a deterministic step counter is preferable to a wall clock deadline as the primary backstop, because it makes aborts reproducible across loaded and idle machines. A deadline is still useful as a secondary guard for costs the counter does not model.
Fifth, on the matcher itself, the durable fix is an engine with linear time guarantees rather than an attempt to write safe patterns for a backtracking engine. The shape predicates under discussion do not need backreferences or lookaround, so the capability loss is small. Where a backtracking engine is unavoidable, linting the pattern set for nested quantifiers at build time is more reliable than runtime mitigation, and the declared per field grammar check should run first because it is linear and rejects most hostile input before any matcher is reached.
Sixth, on failing closed without harming the primary work, the useful shape is a three valued outcome of allow, deny, and undetermined, where budget exhaustion yields undetermined and undetermined is treated as deny for sharing. The important detail is that this be structural rather than disciplined: make deny the default value of the decision type, so that an early return or a missed assignment cannot produce allow. Exceptions from the filter are caught at the boundary and mapped to deny, never rethrown into the caller, since propagating turns a privacy control into an availability bug while swallowing into allow turns it into a leak.
Seventh, warning emission is itself a denial of service surface. An attacker who can provoke one warning per message can use the logging path as the amplifier, so category warnings need deduplication and a per category rate limit with a suppressed count summary. Warnings should carry a closed set category code, the declared schema position, and the configured limit exactly, with any observed measurement reported only as a coarse bucket, since exact observed depth or length is a measurement of attacker controlled data.
Proposed test, not yet executed, that seems cheap and high value: place a distinctive synthetic marker inside the hostile payload and assert that the marker never appears anywhere in emitted logs, metrics labels, or returned error text. That turns the no echo requirement into a property rather than a review checklist item. Adversarial cases should assert both termination and the specific category code, under a hard ceiling on the test itself so a regression fails rather than hangs.