Egress path filter review corrections: keys, decoding gate, telemetry, false denials, one scratch pipeline
Corrections two independent reviews found in an outgoing JSON path filter that already validated decoded values, budgeted work and isolated its matcher. Covers key scanning, the receiver defined decoding gate, one scratch pipeline before decode and score, non Latin drive letters, false denials from number round trip and decode fixed points and separator density, dropping the telemetry hash, and worker error streams.
Egress path filter review corrections
Trigger
Use this after an outgoing structured document filter already parses strictly, matches identifying path shapes on decoded string values, meters work from one budget ledger, runs the matcher in an isolated worker, and emits value free rejection records. This skill lists what two rounds of adversarial review still found wrong in such a design. Apply it as a checklist before calling the filter done.
The failure it prevents
A design that is correct at the layer level can still leak through small gaps: a path in an object key, a nested encoding the filter declines to unwrap, a zero width character between a drive letter and its colon in a free text field, a drive letter from a non Latin script, a user controlled map key echoed in a telemetry pointer, or a library diagnostic quoting its input on the worker error stream. It can also falsely deny legitimate data: ordinary decimals and large integers, short alphanumeric tokens that happen to base64 decode, URLs and dates that trip separator density, and question and answer formatting that looks like a bare drive letter. Both failure classes pass a layer level review.
Steps
- Scan key text with the same grammar used for values, and deny on an unknown key rather than only reporting it. A key is transmitted verbatim by canonical re serialization, so a reported but retained key is a leak. For allowlisted map fields with dynamic keys, the telemetry pointer must render the dynamic segment as a fixed placeholder, never the key.
- State the threat model of the nested decoding gate. Unwrapping only values the recipient parser would decode means the filter, not the far side reader, decides what counts as encoded. Hex, base32, alternate alphabets, or base64 with injected separators all pass as opaque and are trivially reversed by a human or a model on the far side. Document that the filter stops path shapes from a cooperative producer and is not exfiltration containment against an adversarial one. The allowlist of field shapes is the real control: deny opaque high entropy runs in prose fields, and in fields explicitly allowed to carry encoded content decode unconditionally rather than when plausible.
- Run one scratch pipeline before every decode attempt and before scoring, in every field, not only constrained ones: strip format and bidirectional controls, strip combining marks, apply compatibility normalization, map separator and colon confusables, case fold. Compatibility normalization does not remove zero width characters, so a pipeline that strips them only before decoding leaves a free text field where a zero width space between a drive letter and its colon defeats the drive rule while rendering identically. Reject rather than strip these characters in constrained fields. Reject NUL and lone surrogate escapes at parse time, since a NUL terminates some matchers while the serializer emits the rest.
- Do not pre scan bracket depth on raw bytes. A string value full of literal opening brackets would count as deep nesting and cause a false deny, and a string aware pre scan must track escapes correctly or inverts the error. The strict parser with an explicit stack already enforces depth.
- Drop the telemetry hash. Identifying path values are structured and low entropy, so anyone holding the keying secret recovers a username by enumeration. A per window salt only delays that dictionary attack to whoever holds the salt. Field pointer, rule code and coarse length bucket are enough to reproduce a false positive locally.
- Close the pointer label space. Array indices are unbounded and recursive schemas give unbounded pointer depth. Bucket indices into a few ranges and truncate pointer depth to a fixed level. Use one external budget code for any budget trip and keep the specific budget name in local debug output only.
- Fill the grammar gaps. Treat any single letter in any script followed by a colon as drive shape, since compatibility normalization does not fold Cyrillic or Greek capitals to Latin. Keep a confusables table for separators and colons. Add mounted drive, volume, SMB and file scheme prefixes, extended UNC syntax, named home shortcuts, and profile environment variables. Tokenize on ASCII whitespace only, because normalization maps non breaking spaces to ordinary spaces and would split a path into innocent fragments.
- Avoid the known false denials. Carry numbers as validated raw text with digit and exponent caps; rejecting numbers that fail a serializer round trip denies decimals with a trailing zero, exponent notation, negative zero and integers beyond fifty three bits, and numbers cannot carry a path shape. Define a successful decode as output that is a string, valid text, and different from the input; a fixed point such as a bare integer or the word null terminates the ladder and does not deny. Require a minimum length before attempting base64. Set the round cap near six, since nested percent encoding legitimately reaches three; the byte and node ledger is the real bound. Exempt tokens with a scheme and host from separator density but still check their path component for drive and home shapes, and require a backslash or a known prefix word before forward slash density fires. Decide per field shape whether a bare letter and colon fires, since question and answer formatting produces that token in prose.
- Normalize on a scratch copy and emit the original strings. Separator folding and case folding exist only for matching, and the confusables table must map only toward separators and colons.
- Harden the worker channel. Treat a clean exit with empty or truncated output as deny. Frame the result with a length prefix and terminator, cap the length prefix before allocating, require exactly one frame then end of file, check exit status only after end of file, and drain the pipe concurrently. Use one worker per document or a request nonce in the frame so a late frame from a timed out document cannot be read as the next verdict. Send the worker error stream to a bounded buffer that is never persisted, and include that stream in the planted marker test. State plainly that the matcher step budget is a second ledger pre charged from the first.
- Cap decoded output during decoding, not after. Charge nested depth additively: depth at the enclosing string plus depth inside it. Size the explicit stack for the product of decode rounds and maximum depth, which is the worst case of that sum, but do not charge the product against the depth cap or every nested document denies. Charge normalization output to the string length caps and apply the stream safe combining mark limit.
- Make fail closed structural. The allow verdict and the canonical bytes are one value with a single constructor reachable only from the clean walk branch, and the sender accepts only that type. The budget spend function aborts rather than returning a boolean a caller can ignore. Require a top level object so a bare top level string cannot reach the clean walk allow path. Translate library errors to local codes at the throw site, not at the boundary, since messages quote input. Add an outer boundary for non exception aborts such as stack overflow inside a normalization or regex library, or run the whole filter in the worker and let the parent serialize its own tree.
- In encoding differential tests assert on the sorted set of rule identifiers, not one identifier, since two rules can fire on the same value, and pair each identifying case with a benign control that must be allowed.
Limits
Paths split across sibling array elements or adjacent fields remain undetected by any per value scan; only a schema that forbids free form string arrays and caps per field length closes that gap. The absence of a share is a one bit oracle that the turn contained a path shape. Per window counters still reveal activity rhythm. The design still assumes the worker sandbox denies network and filesystem access and that canonical bytes leave only through the parent process single send path.
Basis
Reasoned analysis and two independent adversarial reviews. No tests were executed, no code was run, and no implementation was inspected. Every threshold mentioned is a placeholder the adopting team must derive and test.