Fit a Grapheme-Safe Prefix into a Serialized UTF-8 Budget

Use this procedure when a network contract permits shortening user-visible text and limits the UTF-8 byte size of a serialized field or body. If the contract requires rejection rather than shortening, reject the oversized value instead.

Establish the measured representation

Identify the exact representation covered by the byte ceiling: the final field value, serialized body, or framed message. Apply only contract-required normalization and canonicalization before truncation. Use the production serializer with the same escaping, envelope fields, and encoding settings used for transmission.

Choose the truncation unit independently from the byte unit. For user-visible text, prefer extended grapheme clusters so a retained prefix does not end inside a combining sequence, emoji modifier sequence, or zero-width-joiner sequence. A protocol that defines another semantic unit takes precedence.

Select the longest fitting prefix

1. Segment the normalized source into complete truncation units.
2. Define a candidate by taking the first k units, inserting that prefix into the real payload, serializing the complete measured representation, encoding it as UTF-8, and counting the resulting bytes.
3. Measure the candidate with zero retained units. If even that representation exceeds the ceiling, report that fixed envelope overhead cannot fit; truncating the text cannot solve the request.
4. Find the greatest k whose measured byte count is at most the inclusive ceiling.
5. Return the prefix only after checking the postconditions below.

A linear scan over prefixes is the safe default. Binary search is valid only after establishing that measured size is nondecreasing as k grows. Ordinary deterministic serialization of an uncompressed string field is usually prefix-monotone because adding a unit cannot remove already serialized bytes. Compression, content-dependent deduplication, canonicalization that rewrites neighboring text, or other whole-message transforms can violate that assumption. For such pipelines, use a proven method for that transform or scan every candidate; do not infer monotonicity from a few samples.

Do not estimate whole-body capacity by subtracting a constant envelope size when content escaping can vary. A quote, backslash, control character, or ASCII-only serializer mode can make different source units contribute different serialized byte counts. Measuring each candidate through the production serializer keeps this variation inside the calculation.

Exact-fit empty boundary

Treat equality separately from overflow. If the zero-unit candidate exactly equals an inclusive ceiling, the empty prefix is valid; fixed envelope overhead is impossible only when that candidate is larger than the ceiling. Continue to measure nonempty candidates according to the pipeline's established search rule. For a prefix-monotone representation, any candidate larger than the exactly full empty representation cannot fit.

For a concrete reasoned example, assume compact JSON containing one text property, ordinary direct UTF-8 output, no spaces, no later transformation, and an inclusive eleven-byte whole-body ceiling. The empty representation {"text":""} occupies eleven UTF-8 bytes and therefore fits exactly. Retaining the ASCII letter A produces {"text":"A"}, which occupies twelve bytes and exceeds the ceiling. The correct maximal prefix is empty, not an envelope-overhead failure.

This example does not establish counts for a different property name, whitespace policy, escaping mode, framing layer, exclusive ceiling, or downstream transformation. Measure those representations directly.

Required postconditions

For a chosen prefix with k units:

• Its measured representation is within the byte ceiling.
• If k is smaller than the source length, the candidate with k plus one units exceeds the ceiling. This proves maximality under a prefix-only policy.
• The returned text ends on a complete truncation-unit boundary and remains valid Unicode.
• Any separate semantic limit is also satisfied using its declared counting unit.
• No later transformation changes the representation before the contract's measurement boundary. If another layer reserializes or reframes the payload, measure the bytes at that later boundary instead.

Keep rejection and shortening observably distinct. Diagnostics should state the semantic units retained, the final UTF-8 bytes used, the ceiling, and whether fixed envelope overhead prevented any value from fitting.

Reasoned example

Assume compact JSON of the form an object with one text property, UTF-8 output, ordinary JSON escaping, no later transformation, and an inclusive sixteen-byte whole-body ceiling. The empty body representation occupies eleven bytes. A source quote is escaped with a backslash, so retaining that one grapheme produces thirteen bytes. Adding a four-byte emoji produces seventeen bytes, which exceeds the ceiling. Under a prefix-only policy, the quote alone is therefore the maximal fitting prefix.

These counts are a reasoned illustration, not an executed test. Real implementations must verify their runtime's exact serializer output and the byte buffer presented at the relevant transport boundary.

Focused verification

Exercise the empty prefix, the selected prefix, and the next longer prefix. Include candidates containing ASCII, two-byte and four-byte scalars, combining sequences, zero-width-joiner emoji, quotes, backslashes, and control characters. If binary search is used, separately verify or prove the pipeline's nondecreasing-size property. Distinguish these executed checks from reasoned examples in reports.