# Egress JSON redaction: validate decoded values, budget the work, never echo rejections

Use when a filter must stop identifying strings from leaving in outgoing JSON, and must survive hostile depth, size and matcher cost. Covers the layer at which to validate, why an allowlist does not remove backtracking risk, invisible character and mid sentence bypasses, a counted budget ledger, the two distinct senses of failing closed, and rejection telemetry that cannot leak the value it rejected.

Exact reference: {"kind":"skill_version","skill_id":"skl_uU6sh4MeLBJCsV31NKsFAA","version_id":"skv_sKnGI1PNExsl_-VZ1Y2-NA"}

Applicability: [{"constraint":"Any language or runtime; assumes a parsed tree and a canonical serializer are available","technology":"Structured document egress filtering","version_scheme":"unknown"},{"constraint":"Assumes compatibility normalization and a confusable skeleton are separate stages from decoding","technology":"Unicode normalization and confusable detection","version_scheme":"unknown"}]

## When this applies

A component inspects structured documents before an optional outbound path transmits them, and must prevent identifying strings, typically filesystem paths, from leaving. Reach for this whenever a redaction rule is written as a pattern, and whenever such a filter is exposed to input an attacker can shape.

## The failure it prevents

A pattern authored against ordinary readable text, then applied to the serialized document, silently never matches, because the separator a human reads as one character is escaped into two on the wire. The rule appears to work in review and protects nothing. The same class covers unicode escapes, percent encoding, base64, and a document nested inside a string value. Patching the pattern to expect the escaped form closes one case and leaves the class open.

## Steps

**1. Validate the decoded value, and return the bytes you validated.** Parse, walk the tree, apply every rule to each decoded string, then serialize canonically. The result of an allow decision must carry those canonical bytes, and the outbound path must be structurally unable to transmit anything else. A boolean verdict quietly reopens the bug: if the caller reserializes its own object, the emitted bytes came from a different encoder than the validated value, which is exactly the differential the design exists to close. Not mutating caller data means not mutating in place, not declining to produce the artifact.

**2. Allowlist field shapes, but do not infer anything about matcher cost from that.** Declare a permitted shape per field and reject unknown fields by default, because denylisting identifying shapes is unbounded. Positivity has no relation to backtracking complexity: a narrow allowlist shape is exactly where nested quantifiers appear, such as a repeated group of word characters followed by a separator. Require a linear time engine, or statically verify each pattern is backtracking free, and keep a matcher step budget as real defense in depth.

**3. Reject invisible characters rather than normalizing them away.** Compatibility normalization does not remove zero width spaces, soft hyphens, joiners or bidirectional controls, and a confusable skeleton excludes them under a separate rule that a decode then normalize then confusable pipeline never invokes. One zero width space inside a prefix defeats a prefix anchored signal while rendering identically to a reader. Reject format and bidirectional controls in constrained fields; stripping then matching creates a fresh differential against the recipient decoder.

**4. Score identifying shape per token, not per string.** Signals anchored at string start or end, plus a whole string separator density, are bypassed by embedding the value mid sentence in any free prose field, and every realistic payload has at least one. Split the decoded value on whitespace, compute every signal and the density per token, and reject the document if any token scores. Never rewrite a scoring value; rewriting is how partial identifiers survive.

**5. Route every cost through one budget ledger with a single spend entry point.** Cap raw bytes while streaming, container depth, node count, keys per object, per string decoded length and an aggregate string total, decode rounds, and normalized length. Cap exponent magnitude separately from mantissa digits, since arbitrary precision cost scales with the exponent value and not its digit count, so a short literal can request a billion digits; better still, carry numbers as validated raw text when no rule inspects their value. Reject duplicate keys rather than resolving last wins, since any parse differential is an evasion generator. Enforce during the work, not after it. Make parse, decode, walk and serialize non recursive, since a worklist walk over a tree a recursive parser already died building buys nothing.

**6. Fail closed in two distinct senses.** Initialize the verdict to deny and assign allow only as the final statement after a completed clean walk, so every early exit is already a denial. Separately, make the filter total toward its caller: a boundary catch all maps any escape to deny rather than propagating. The asymmetry that results is correct, since an abort suppresses the optional outbound path while the primary work continues. On any abort withhold the whole document, never the portion already scanned, because burying a value past the abort point is what converts a denial of service into a disclosure.

**7. Make rejection telemetry incapable of carrying the value.** Log a field pointer derived from the schema, a rule identifier from a closed compile time enumeration, and a keyed hash under a rotating secret held outside the log store, never a plain digest, since low entropy strings are brute forceable from a candidate list. Do not log exact length; but note that an exact character class histogram sums back to the exact length and is more discriminating than length alone, so bucket the histogram on the same coarse scale or reduce it to presence bits. Index counters by the closed enumeration so no attacker controlled string becomes a label, because a cardinality explosion leaks availability even when no value is echoed. Translate engine errors into local codes, since parser and matcher limit errors often embed subject fragments or offsets. Emit one aggregate record per pass and roll up across passes.

**8. Test the invariants, not the wording.** Plant a random marker in a synthetic payload, force every abort path, and assert the marker appears in no record, metric or label. For the encoding differential test, assert on the pair of verdict and rule identifier and pair each identifying case with a benign control that must be allowed, because asserting only that verdicts match across encodings is satisfied by a filter that denies everything. Prove early abort by asserting on consumed units rather than elapsed time.

## Limits

Counted budgets are deterministic, but a wall clock backstop makes the verdict deterministic only modulo that deadline, which may convert allow into deny on a loaded host; run determinism tests with a virtual or disabled clock. A reachability criterion requiring that each budget be trippable first must exempt designated backstops, or it pressures someone into loosening a cap. A runtime cap on distinct telemetry categories is dead code once the enumeration is closed at compile time. Derive caps from a high percentile of legitimate payload shape under a compiled ceiling configuration cannot raise. A byte cap does bound node count and depth linearly, so cap nodes for the hostile per node heap constant rather than for reachability. A pre normalization length cap does bound the normalized result by a published expansion constant, but far too loosely to be the operative budget.

## Basis

Reasoned analysis and adversarial review only. Nothing here was measured or executed; every threshold is a placeholder for a value the adopting team must derive and test.


## Supporting basis and limitations

Support is reasoned analysis and one independent adversarial review. No tests were executed, no code was run, no timings were measured, and no implementation was inspected. Every numeric threshold is a placeholder.

Derivations that the review did not fault: the layer mismatch account of the original defect, since escaping means the scanned representation differs from the one the pattern author had in mind; the requirement that the validated value be the emitted value, since two decoders create a differential; the need for both a byte and a node cap, though the correct justification is the per node heap constant rather than a failure of bytes to bound nodes; withholding the whole document on abort, since partial scanning upgrades a denial of service into a disclosure; and keying telemetry counters by a closed enumeration, since attacker controlled labels leak availability without echoing a value.

Claims that the review falsified and that are corrected in the body: that an allowlist of narrow field shapes removes catastrophic backtracking, which is false because positivity does not constrain matcher complexity and nested quantifiers appear naturally in such shapes; that a byte cap fails to bound node count, which is wrong in one direction since roughly two bytes per nesting level means bytes bound nodes linearly; that a pre normalization cap does not bound post normalization cost, which is overstated because maximum expansion factors are published constants and the real defect is looseness; that a numeric literal length cap bounds arbitrary precision cost, which it does not because cost tracks exponent value rather than digit count; and an unqualified determinism claim that a wall clock deadline contradicts.

Live bypasses identified against an earlier version of this design, and closed in the body: an identifying value embedded mid sentence in a free prose field, which evades every start anchored, end anchored and whole string density signal; and a default ignorable character inserted inside a prefix, which survives compatibility normalization and confusable skeletons while rendering identically to a human reader.

Remaining unknowns, stated rather than resolved: a good default for the normalization expansion ratio cap, whether naming the budget that aborted first discloses useful structure to someone probing the filter, and whether isolating a matcher in a separate process is worth its cost when the engine cannot be replaced.

## Change and rationale

New skill capturing a corrected design for redaction filters on outgoing structured documents. Covers validating the decoded value and returning the validated bytes, allowlisting field shapes without assuming anything about matcher cost, rejecting default ignorable characters, scoring identifying shape per token, a single counted budget ledger, two distinct senses of failing closed, and rejection telemetry that cannot carry the rejected value. Includes the limits that an adversarial pass falsified, notably that a byte cap does bound node count and that a pre normalization cap bounds expansion by a published constant.

The originating defect is a layer error that looks like a working control in review, so it recurs wherever redaction is written as a pattern over a serialized document. The steps here are the ones an adversarial pass did not falsify, and several are counterintuitive enough to be worth recording: a boolean verdict reopens the encoding differential, a positive allowlist does not remove backtracking risk, and an exact character class histogram defeats the length bucketing it accompanies. Each of those was asserted confidently in the originating discussion and then corrected, which is the signal that they are easy to get wrong rather than obvious. The limits section exists so an adopter does not inherit the specific claims that were falsified.