Prove Independent Unicode and UTF-8 Limits with a Crossed Test Matrix
A focused test-design procedure for proving that semantic Unicode length and final UTF-8 payload size are enforced by separate validators, with fixtures covering all four pass and fail combinations and unambiguous failure attribution.
Prove Independent Unicode and UTF-8 Limits with a Crossed Test Matrix
Use this procedure when one network field or payload must satisfy both a semantic Unicode length limit and a UTF-8 byte ceiling. It is especially useful when tests currently cover only ASCII or only one rejection path.
The objective is not merely to test each limit once. It is to prove that the implementation computes two different measurements, applies both decisions, and attributes rejection to the correct constraint.
Freeze the two contracts
Before choosing fixtures, write down:
- The semantic unit: grapheme clusters, Unicode code points, or language-specific code units.
- The exact value to which the semantic limit applies.
- The byte boundary: final field value, serialized body, framed message, or another explicitly named representation.
- The encoding and serializer settings at that boundary.
- Required normalization and whether it occurs before either measurement.
- Whether each maximum is inclusive.
- Whether validation reports every violated constraint or stops at the first one.
Do not build the matrix until these details are fixed. A fixture can move between cells if normalization, escaping, framing, or the semantic counting unit changes.
Construct all four outcomes
Choose fixtures that produce this matrix:
| Semantic limit | UTF-8 byte limit | Required result |
|---|---|---|
| Pass | Pass | Accept |
| Fail | Pass | Reject for semantic length |
| Pass | Fail | Reject for encoded size |
| Fail | Fail | Reject according to the documented multi-error or precedence policy |
The two disagreement rows are the essential part. If they cannot be constructed for the chosen ceilings, use test-only ceilings that preserve the production counting rules, or test the validators separately as well as through the combined entry point. Do not weaken production semantics merely to make a fixture convenient.
Measure the semantic value with the declared Unicode unit. Produce the byte value by running the scoped representation through the production normalization, serialization, and UTF-8 encoding path, then count the resulting bytes. Do not derive both expected values from the same production helper being tested.
Reasoned example
Assume an inclusive maximum of two Unicode code points and an inclusive maximum of four UTF-8 bytes. The byte ceiling applies to the final field value, with ordinary UTF-8 encoding and no normalization or escaping after validation.
| Fixture | Reasoned code points | Reasoned UTF-8 bytes | Expected outcome |
|---|---|---|---|
| é | 1 | 2 | Both pass |
| AAA | 3 | 3 | Semantic limit fails; byte limit passes |
| 😀A | 2 | 5 | Semantic limit passes; byte limit fails |
| 😀AA | 3 | 6 | Both fail |
These are reasoned encoding examples, not executed tests. They do not establish counts for a serialized body, an ASCII-only serializer, a normalized value, a framed message, or a different Unicode counting unit.
For a grapheme-cluster limit, replace the semantic expectations with grapheme segmentation results and retain an independently measured byte oracle. For a code-unit contract, count exactly the code units named by that contract even when they differ from code points.
Exercise exact boundaries
For each validator, add a fixture exactly at its inclusive ceiling and one just beyond it while holding the other dimension on the intended side of its limit. This prevents an off-by-one result from being hidden by the other validator.
When possible, include:
- ASCII text, which can make the two counts accidentally look interchangeable.
- Two-byte, three-byte, and four-byte UTF-8 scalar values.
- Precomposed and decomposed forms when the contract does not normalize them to the same representation.
- Multi-code-point grapheme clusters when the semantic unit is user-perceived text.
- Characters whose serializer escaping changes the measured body size when the byte limit applies after serialization.
- Fixed envelope overhead when the ceiling applies to the whole body.
Assert the decision and its attribution
For every fixture, assert more than a generic rejection:
- The semantic measurement and its named unit.
- The encoded byte measurement and its named boundary.
- The final accept or reject decision.
- The specific violated constraint or constraints.
- The documented precedence when both fail and the API returns only one error.
A combined validator may short-circuit for efficiency, but its tests must still prove the skipped path independently. Otherwise the both-fail row can pass while one validator is absent.
Avoid returning or logging the entire rejected payload merely to explain the size error. Counts, limits, units, boundary names, and safe field identifiers are usually sufficient.
Detect common false confidence
This matrix is designed to catch these defects:
- A string-length helper is reused as a UTF-8 byte count.
- Byte length is measured before required normalization or serialization.
- A field-level measurement is substituted for a whole-body limit.
- ASCII-only fixtures make character count and byte count appear equivalent.
- The first validator always rejects, leaving the second path untested.
- The final decision is correct but the reported constraint is wrong.
- Expected counts are computed with the same faulty helper as the production result.
Verification record
When the tests are actually run, record the runtime, serializer mode, normalization policy, measured representation, and observed counts. Label those observations as executed results.
Until that execution occurs, describe the fixture tables only as reasoned examples. Do not present inferred counts as runtime evidence.