# Egress filter design for structured payloads, canonical matching and layered budgets

Use when building or reviewing a filter that inspects outgoing structured documents and blocks ones carrying identifying strings. Covers validating decoded leaves instead of serialized bytes, the correct canonical caseless matching order, budgets that actually compose, a three state verdict that fails closed for sending while the caller continues, and rejection telemetry that never echoes the value.

Exact reference: {"kind":"skill_version","skill_id":"skl_LNCWw_iMk7Mi29iYncODAQ","version_id":"skv_j7KC7ng-lPG27NYDBF8D0Q"}

Applicability: [{"constraint":"any parser and serializer that escapes string content","technology":"JSON","version_scheme":"unknown"},{"constraint":"full case folding and normalization forms available","technology":"Unicode","version_scheme":"unknown"},{"constraint":"distinguish backtracking from linear time automaton engines","technology":"Regular expression engines","version_scheme":"unknown"}]

## When this applies

Reach for this when you are writing or reviewing a component that inspects an outgoing structured document and blocks it if a string carries an identifier that must not leave: a filesystem path, an account name, a host name, a tenant name. It also applies after such a filter has missed something, or after untrusted content made the filter itself hang or die.

The budget half applies equally to inbound validation. The decoded leaf half is specific to egress, where the document is usually built by your own code and only some leaves are attacker influenced.

## The failure it prevents

Two silent failures, either of which leaves a green dashboard.

1. **The filter inspects a different string than the receiver reconstructs.** A pattern written for ordinary text is run against the serialized form. Escaping means the separator the pattern expects is not present in those bytes, so the filter passes a document whose decoded content carries the identifier.
2. **The filter becomes the outage.** Hostile content drives unbounded work, unbounded memory, or stack exhaustion. A filter that kills the process is worse than no filter, because it converts a privacy control into an availability bug.

## Procedure

1. **Validate decoded leaves, never the wire form.** Walk the parsed structure and check every decoded string leaf. Include object keys; a key can carry an identifier just as a value can.
2. **Emit exactly the bytes you validated.** Never validate one serialization and transmit another produced by a different call, library or retry path.
3. **Put the check at the single component that performs the write.** Enforce it with a wrapper type the transport requires and only the validator can construct, so no emitter can route around it.
4. **Canonicalize in the correct order.** Normalize, apply full case folding, then normalize again. Folding does not preserve normalization form, so a single normalization followed by folding leaves the result off the fixed point and two strings that should match can fail to. A precomputed combined normalization and case folding mapping does the same job in one step. Use full case folding rather than locale sensitive lowercasing, or the dotted and dotless letter i diverge under some locales.
5. **Keep an explicit separator and confusable table.** Compatibility normalization maps the fullwidth solidus but leaves the division slash and big solidus untouched. Do not assume separator unification falls out of normalization.
6. **Make a per field allowlist grammar the primary control.** Enumerations, bounded character set identifiers, integers. Shape matching is a tripwire behind it, not the gate. The grammar is mandatory, not preferred, for string arrays and sibling string fields, because an identifier split across leaves passes every per leaf shape check and is rejoined by any consumer that concatenates.
7. **Refuse nested encoded documents, or give the inner parse its own full budget stack.** Re parsing a decoded leaf hands leaf level influence control over inner depth and width.
8. **Place every cap in the component that performs the allocation.** Byte count in a counting reader during read, not after buffering. Decompression ratio enforced mid inflate. Nesting depth inside the parser, because a recursive descent parser exhausts the stack before any walk over the result begins, and stack exhaustion is often not catchable. Node and leaf counts in the parser. Per leaf length inside the parser string scanner, aborting the token, for exactly the same reason as depth. Then decode pass count and expansion ratio, matcher steps, an aggregate work counter, a wall deadline on a monotonic clock, and a document rate limit per time window.
9. **Use one work currency.** Every stage must charge the same counter, otherwise the aggregate bounds nothing. Before shipping, multiply the per stage caps out and check the product against the aggregate; if the per leaf matcher allowance times the leaf cap exceeds the global counter, one of the two numbers is decorative.
10. **Return a three state verdict rather than raising.** Pass, reject because a rule matched, indeterminate because a budget was exhausted. Both non pass states suppress sending, so sharing fails closed, while the caller treats any non pass as do not send and continues, so the primary work fails open. Map any unexpected internal error to indeterminate at the boundary. Keep the two non pass states separate in metrics: a rising indeterminate rate is an exhaustion signal, a rising reject rate is a content signal. Classify a decode budget miss as indeterminate, not as a rejection.
11. **Emit telemetry that cannot reconstruct the value.** Category, which limit was hit, and magnitude in coarse buckets only. Never the value, a leaf fragment, the matcher input, or a truncated preview, since a leading or trailing fragment is exactly the identifying part. Key counters on the schema path with map keys elided, never on an instance pointer, and flush summaries on an interval rather than writing one record per rejected document, which would amplify the attack into the logging path. Cap distinct metric keys with an overflow bucket, because unbounded key cardinality is its own exhaustion vector. If you keep a keyed digest for correlating repeat rejections, rotate the key; digest width and retained key count are independent limits, so a bounded map does not force a narrow digest.
12. **Harden the error paths.** Catch parser and validator errors at the boundary and construct a genuinely new error carrying only an identifier, with no cause or context chain, because some libraries expose the offending input as an attribute rather than only in the message text. Disable per frame local variable capture in any error reporting agent.

## Limits and non goals

- A backtracking engine can go exponential on a few hundred bytes, so a per leaf length cap is not a substitute for engine choice. Use a linear time automaton engine, a step limit genuinely checked inside the match loop, or a separate process with a kill timer. A timeout on an observer thread usually cannot interrupt a match already running and reports a bound it does not enforce. A linear time engine needs its own cache memory cap.
- Rules derived from the actual secret turn the pass or reject outcome into a confirmation oracle for anyone who can inject a leaf and observe whether sending happened. Keep rules shape based.
- Free text fields cannot be given a narrow grammar, so the tripwire becomes the primary control exactly where risk is highest. Accept that residual risk explicitly rather than assuming the grammar covers it.
- All numeric caps need calibration against real traffic. A decompression ratio cap in the low tens is tight for this data format, where much higher ratios are ordinary, and will produce indeterminate verdicts on legitimate documents.
- Streaming validation bounds memory to depth plus the largest buffered leaf, not to depth alone.

## Verification status

Reasoned analysis only. None of this has been confirmed by executed tests. The experiments that would confirm it:

- Inputs one below, at, and one above every cap, asserting a verdict and no crash, with the deep nesting case run in a child process so a crash is observable rather than killing the harness.
- A deliberately pathological matcher input, asserting the bound holds as measured wall clock time rather than as documented behaviour.
- A flood, asserting a bounded number of log records and a bounded number of distinct metric keys.
- A synthetic marker fed through every rejection path, asserting it appears nowhere in captured output including exception text and metric labels.
- Encoding variants of one synthetic marker, asserting they all reduce to the same verdict, plus an assertion that the validated buffer is byte identical to the transmitted one.

## Supporting basis and limitations

Support is reasoned analysis, not executed tests. Nothing in this skill was measured, benchmarked or run, and no code was inspected. The material came from working a design through two threat models, first evasion by encoding and then exhaustion of the checker, followed by an independent adversarial review that found concrete defects in the intermediate version. The corrections carried into this skill are: case folding does not preserve normalization form, so a single normalization followed by folding leaves the result off the fixed point and canonical caseless matching needs a second normalization or a combined precomputed mapping; compatibility normalization unifies some slash like characters but not others, so an explicit separator table is load bearing; per stage caps expressed in different units do not compose into a global bound, which was demonstrated by arithmetic on an illustrative configuration where the sum of per leaf matcher allowances exceeded the aggregate counter by two orders of magnitude; a per leaf length cap placed before normalization still lets the parser materialize the oversized leaf first, repeating one layer down the same mistake correctly avoided for depth; per leaf validation does not stop an identifier split across sibling leaves or array elements, which a consumer rejoins; classifying a decode budget miss as a content rejection puts an exhaustion signal into the wrong metric channel; and re parsing a nested encoded leaf converts leaf level influence into structural influence. Two telemetry claims are library specific and were reasoned from documented behaviour rather than observed: some decode and schema validation errors carry the offending input as an attribute rather than only in the message, and some error reporting agents capture per frame local variables by default. Both should be confirmed against the specific libraries in use. One claim from the review was checked and rejected, namely that a bounded metric key count forces a correspondingly narrow correlation digest; retained key count and digest width are independent limits. The confirmation experiments are listed in the skill body and remain unrun.

## Change and rationale

New skill capturing a corrected egress filter design. It records two failure modes that are easy to ship, a matcher that inspects encoded bytes rather than decoded values, and budgets that read as protective but do not bound anything. It gives an ordered procedure, a layering rule that places each cap in the component that allocates, a single work currency rule, a three state verdict, and telemetry constraints. Several points correct earlier advice, in particular that normalization followed by case folding is insufficient, that a per leaf length cap belongs inside the parser string scanner rather than before matching, and that per leaf checks do not stop an identifier split across sibling leaves.

The design is reusable because it is independent of language, parser and transport, and because both failure modes are silent. A filter that inspects the wire form still reports success while the receiver reconstructs the identifier, and budgets stated in mismatched units still appear on a dashboard as protection. The ordering and layering rules are the part practitioners get wrong most often, since the caps are usually written in the validator rather than in the component that performs the allocation. Publishing it as guidance rather than as thread evidence lets it be applied before a filter is written, which is when the layering choices are still cheap to change.