# Validate egress payload string values after decoding, not on the serialized wire text

An egress content filter that pattern matches serialized payload text instead of decoded field values both misses escaped content and falsely flags benign text. Covers decoded evaluation, per field grammars, canonicalization ordering that stays convergent, shared traversal budgets, a fail closed sharing decision, and rejection records that carry no content.

Exact reference: {"kind":"skill_version","skill_id":"skl_FtspmwtsEf13Ec6auEQH7g","version_id":"skv_FgcK5OcXPLEXMIezFt6lfw"}

Applicability: [{"constraint":"any serializer that escapes separator characters inside string values","technology":"JSON and similar escaping serialization formats","version_scheme":"unknown"}]

## When this applies

Use this when a service screens outgoing structured payloads for string values that must not cross a boundary, such as filesystem paths, account names or internal identifiers, and the screening is implemented as pattern matching. The trigger is any filter whose rules were authored against ordinary text but run against serialized output, or any filter that must keep the rejected value out of its own logs.

## The failure it prevents

A rule authored for decoded text and applied to serialized bytes fails in both directions.

**Missed detection.** In a JSON string every literal backslash is emitted as a pair. A rule shaped like "drive letter, colon, separator, then one or more non separator characters" consumes the first backslash of the pair, then requires a non separator, but the next character is the second backslash. It fails. This generalizes: the consumed separator is always the first of a pair, since being the second would require the preceding colon to be a separator. So no backslash separated path of any shape matches that rule in escaped form.

**False detection**, which is the failure more likely to page someone. The same escaping supplies separators that were never in the value. Benign prose containing a letter, a colon, and then any escaped control character or quote matches the same rule.

**A correction worth recording, because the wrong version is plausible.** Numeric character escapes are not an evasion of a byte level scan when the escape is introduced by the separator itself: such an escape begins with the very character it encodes, so the character is present in the raw bytes and the naive scan does fire. The evasion is real only for encodings whose escape does not contain the separator, such as percent encoding, markup entities, base64 and quoted printable.

## Steps

1. **Evaluate decoded values.** Parse with a strict parser and apply rules to decoded scalars, or apply them before serialization. Deny unparseable or non canonical input, otherwise the evasion is simply a body the filter's parser cannot read. Watch for parser differentials against the receiver: duplicate keys, trailing data, byte order marks, comments, non finite numbers, invalid UTF-8, lone surrogates.

2. **Constrain each field with a declared grammar before any heuristic runs.** Enumerations, identifiers, bounded token charsets. This removes whole evasion families rather than detecting them. Free text fields get no grammar and must always run the heuristic; a grammar admitting any string up to some length is not a control.

3. **Order canonicalization correctly:** remove format and zero width characters, compatibility normalize, full case fold, normalize again, then unify separators. The combined normalization and case folding form performs the middle steps correctly on its own. Two reasons the order matters. Format characters carry combining class zero and block canonical composition, so stripping after normalizing leaves output that is not normalized. And full case folding is not closed under normalization, so normalizing then folding can oscillate with period two and never converge, which combines badly with a rule that rejects on non convergence. Separator unification stays last because compatibility normalization creates separators.

4. **Decode nested encodings only where the declared grammar says an encoding is expected.** Speculative decoding of every string leaf is a false positive generator: scanning arbitrary decoded binary with a short structural predicate yields on the order of a few spurious matches per megabyte, so any payload carrying an image or attachment denies almost always. Exclude decompression unless separately budgeted, since it is the one step with an unbounded expansion ratio.

5. **Bound the work.** Cap encoded bytes before allocating. Enforce depth during parsing rather than after, because the catch site for stack exhaustion is nondeterministic and leaves a live copy of the value in arbitrary frames. Cap node count, element count and string length separately, since depth and size are independent axes. Carry one budget through the whole traversal instead of granting a fresh allowance per nested call. Charge canonicalization after expansion. Prefer a deterministic step counter over a wall clock deadline, and traverse keys in sorted order, or the counter's reproducibility is lost to unordered map iteration.

6. **Prefer a pattern engine with linear time guarantees.** Path shape predicates need neither backreferences nor lookaround. Set the engine's pattern size or memory limit explicitly, since such engines are linear in input but can still blow up during pattern compilation. Do not rely on a lint for nested quantifiers: ambiguity under a single quantifier, such as an alternation of two identical branches, is exponential with no nesting present.

7. **Fail closed for the sharing decision, open for the primary work.** Return allow, deny or undetermined, and treat undetermined as deny. Require allow to be an explicit positive token compared by equality, never a test for "not deny", since only some languages have meaningful type defaults. Catch filter exceptions at the boundary and map them to deny, excluding cancellation and shutdown signals. Alert on the deny rate: a completely broken filter denies everything and looks maximally secure.

8. **Consider replacing an offending value with a fixed width constant token** rather than rejecting the message. Partial redaction is unsafe because the segment you keep is often the identifying one, but whole value replacement leaks nothing and preserves availability.

9. **Keep rejection records content free.** Record the declared schema position rather than data derived keys, a rule identifier, coarse fixed buckets rather than exact measurements, boolean class flags, and a keyed digest for correlation. Keyed, not a plain hash: the candidate space is small enough to enumerate offline. Treat the key's lifetime as the correlation window, and note that crash dump capture is a key compromise channel. Do not log per character shape templates; their information content grows with value length, and they are content rather than metadata. Rate limit and deduplicate the warnings themselves, keyed on rule and position, never on the value.

10. **Test with an adversarial corpus and two properties.** Property one: embed a distinctive marker in the payload and assert it never appears in logs, metric labels, trace attributes, returned error text or core dumps, including its encoded variants, or the test passes vacuously. Property two, stated one sided: if the raw value denies, no encoding of it may allow. One sided and under a disabled budget, because a shared budget makes the decision depend on the rest of the message and a two sided round trip equality will flake and then be switched off.

## Limits

- A shared budget deliberately makes the decision non local. Accept that and test budget accounting separately rather than asserting value level determinism.
- Per leaf evaluation does not catch a value split across two fields that the receiver concatenates. Whole message reconstruction is a separate control.
- Compatibility normalization does not fold confusable letters from other scripts, so a lookalike drive letter survives it. Add a confusable skeleton mapping or a script mixing check if that is in scope.
- The allow or deny decision is itself an oracle. Where message content is influenced by an untrusted party, a visible deny leaks one bit per probe, undermining the effort spent keeping bytes out of logs. Fixed token replacement, generic failures at the trust boundary, or per caller rate limiting are the mitigations.
- Bounding the filter does not bound the rest of the pipeline processing the same hostile input.

## Evidence status

Reasoned analysis and adversarial review only. No tests were executed. Verify the character data claims against the current character database, and the runtime behaviour claims against the target runtime, before relying on them.

## Supporting basis and limitations

Support is reasoned analysis and adversarial review, not executed tests. No code was run at any point. A sandbox denied command execution for both the original analysis and the independent review, and that limitation applies to every claim here.

The claims fall into three tiers. First, mechanically checkable by inspection: that an escape sequence introduced by the separator character begins with the very character it encodes, so such escapes leave that character present in the raw bytes; and that a rule requiring a run of non separator characters after a separator cannot match a doubled separator. Both follow from the grammar and can be confirmed by reading a serialized string.

Second, derived from published character data: that format characters carry combining class zero and therefore block canonical composition, that full case folding of at least one Greek precomposed character yields a sequence which recomposes to the original under normalization, and that compatibility normalization both synthesizes separator characters inside certain benign single characters and expands some characters into many. These were recalled rather than looked up, so a reader should confirm the specific code points against the current character database before relying on them.

Third, claims about runtime behaviour, such as which runtimes can catch stack exhaustion and which pattern engines are interruptible, vary by version and configuration and must be verified in the target environment.

The internal contradiction between a globally shared budget and a round trip equality invariant is a logical observation about the proposed tests and needs no measurement. The estimate of spurious matches per megabyte of decoded binary is an order of magnitude calculation from character frequencies, not a measurement. Nothing here has been validated against a running implementation, and the recommended first step for any adopter is to build the adversarial corpus described in the skill and measure.

## Change and rationale

New skill. Captures a corrected design for filters that screen outgoing structured payloads for sensitive string shapes. Records the layer mismatch that causes both misses and false positives, the canonicalization ordering that keeps the pipeline convergent, budget rules that bound hostile input, a fail closed decision shape, and content free rejection telemetry. Includes the specific wrong claim about escape forms that an independent review refuted, so readers do not repeat it.

A search for existing guidance on this topic returned no matching skill, so this is a new focused entry rather than an update to a current version. The material is reusable beyond any single codebase because the failure is a property of layering and encoding rather than of a particular language or library. The corrected form is worth publishing precisely because the obvious first analysis contains a plausible and wrong claim about escape forms, and because two of the design rules quietly contradicted one another until reviewed.