Isolating an egress privacy filter in a worker so sharing fails closed and the caller keeps running
Use when a filter that screens outgoing structured payloads must survive hostile depth, size or matcher cost without stalling or killing the application. Covers why in-process cancellation is insufficient, a subprocess supervision model with four outcomes, a transmit capability that owns the validated bytes, a circuit breaker, a bounded denied-payload cache, and warning records that stay bounded and content free.
Isolating an egress privacy filter in a worker so sharing fails closed and the caller keeps running
When this applies
A component screens outgoing structured documents, typically JSON telemetry, for identifying strings before an optional sharing step. The primary application work never depends on the sharing result. Untrusted input can reach the filter, so it may face extreme nesting, very large values, or text that drives a pattern matcher into pathological cost. Reach for this after deciding to validate decoded values, when the remaining question is how the filter can be denied service without denying service to its host.
The failure it prevents
Two opposite mistakes, both invisible on a green dashboard.
- The filter becomes the outage. A stalled matcher or a blown memory cap inside the application process freezes or kills the primary task. A privacy control has turned into an availability bug.
- The failure becomes a leak. A deadline or crash leaves a code path that can still transmit the original document, or a partial scan result, because nothing structurally prevents it.
Steps
1. Run the filter in a subprocess, not an in-process cancellable task. An in-process task cannot be interrupted inside a native matcher call that has no yield point, and a memory cap is process wide, so tripping it kills the host. Only a separate process gives a real wall clock deadline and a real memory cap. Pass the payload by pipe or shared memory, never by argument list, environment or temporary file, since those are readable by other processes and can persist.
2. Use an automaton based matcher, not a syntax restricted backtracking one. Banning backreferences and lookaround does not make a backtracking engine linear. Nested or ambiguous quantifiers alone produce exponential cost, and a work ledger that charges per detector call cannot see the blowup. Either use an engine with guaranteed linear time or hand written scanners. The deadline is a backstop, not the plan.
3. Model exactly four outcomes in the supervisor. Allow with canonical bytes, deny with a category, deadline expired, and worker gone. The supervisor returns to the caller in every case. Serialization failure after a successful parse, for example a lone surrogate or a non finite number, is an explicit deny, not an exception.
4. Make the transmit capability own the bytes. The sender must accept a single unforgeable wrapper that the supervisor constructs only from an allow outcome and that carries the canonical bytes inside it. If the handle and the bytes are separate arguments, a handle from one document can send the bytes of another, and the capability is only a convention.
5. Bound resource caps jointly. A byte cap on the encoded input does not bound worker memory, because tree node overhead in managed runtimes can be many times the input size. Add a node count budget and derive the byte cap and the worker memory cap together, otherwise a legitimate payload trips the memory cap and feeds the breaker.
6. Add a circuit breaker on deadline and crash counts. Count those two outcomes per time window. Above a threshold, disable sharing entirely for a cooldown while the primary task continues. Keep a bounded, recently seen cache of encoded input digests keyed by filter version so an identical retry is denied without rescanning. Expect the cache to help only for byte identical retries; the breaker is the real defense.
7. Keep warnings bounded and content free. Categories are a closed enumeration, including byte exceeded, depth exceeded, work exhausted, deadline, worker crash, parse error, and one per detector. Emit at most one warning per category per document with an integer hit count. After a per window cap, only increment a suppressed counter per category. Records carry the category, hit count, integer depth, a positional index path and ledger balances. Never include key names, since keys can be attacker controlled, and never include a value substring. If records can leave the machine, drop exact length and offset fields, because they reveal the length of the identifying value.
Limits
Support here is reasoned analysis and one independent adversarial review. No code was executed and no timings were measured. A subprocess per document has real cost; a pooled worker with per request deadline and periodic recycling is the usual compromise, and it must be recycled on any deadline or crash. A path split across sibling values or fields is not caught by per value detection; the mitigation is a per field schema allowlist that denies unknown fields. A wall clock deadline makes the verdict nondeterministic on a loaded host, so determinism tests need a virtual clock.