# Fit Unicode Text to a UTF-8 Payload Budget Without Splitting Graphemes A focused procedure for shortening Unicode text to the longest permitted semantic prefix that fits an exact network byte budget, while counting omission markers, production serialization, and Unicode limits independently. Exact reference: {"kind":"skill_version","skill_id":"skl_rBLeZeC8vDqAFXyub2V4dg","version_id":"skv_c1rqQotlu7l7JiBY5pWd2Q"} Applicability: [] # Fit Unicode Text to a UTF-8 Payload Budget Without Splitting Graphemes Use this procedure when a network contract permits shortening a text value that exceeds a byte ceiling and the product must preserve the longest meaningful prefix. Continue to validate any Unicode character limit separately; fitting the byte budget does not prove that the semantic limit passes. ## Freeze the shortening contract Record these decisions before truncating: - The exact representation whose bytes are limited: the final field value, serialized body, framed message, or another named boundary. - All required normalization, escaping, envelope, and encoding steps before that boundary. - The semantic unit that may be removed. For user-visible text, use extended grapheme clusters unless the contract names a different unit. - Whether an omission marker is required and its exact text. - Whether the byte maximum is inclusive. - Whether an empty retained prefix, a marker by itself, or complete rejection is allowed. An omission marker consumes the same measured budget as the retained text. Do not append it after validation. ## Build one exact measurement function Define one candidate builder that accepts a retained semantic prefix and performs the production transformations in their contractually required order. It must insert the real omission marker when shortening occurred, place the result in the real envelope, serialize with production options, encode the scoped representation, and return its byte count. Keep semantic counting separate. The byte measurement function answers only whether that exact candidate fits the named encoded boundary. ## Select a safe prefix 1. Apply only normalization or canonicalization required before segmentation. 2. Segment the value into the contractually allowed removal units. Never use code-unit indexes as if they were grapheme boundaries. 3. Measure the unshortened production candidate. Return it unchanged when it fits both the byte rule and the separate semantic rule. 4. Measure the shortest allowed shortened candidate, including the omission marker. If that candidate exceeds the byte ceiling, fail instead of silently dropping the marker or changing the contract. 5. Evaluate candidates only at complete semantic-unit boundaries. 6. Starting with the longest permitted prefix, build and measure the exact production candidate. Return the first candidate that fits. 7. If no candidate fits, follow the documented empty-value or rejection policy. 8. Run the independent semantic-limit validator on the selected result and report the two measurements separately. Searching from longest to shortest is correct even when final encoded size is not monotonic, although it can require many serializations. A binary search or a stop-at-first-failure forward scan is valid only after proving that adding a semantic unit cannot reduce the measured size for the exact transformation pipeline. Do not assume monotonicity when compression, content-dependent envelopes, conditional fields, deduplication, or another context-sensitive transform is inside the measured boundary. ## Do not truncate encoded bytes Never slice a UTF-8 buffer at the remaining byte count and decode the prefix. The slice can end inside a multibyte scalar. Even a scalar-safe cut can split a grapheme cluster, such as a base letter from a combining mark or part of a joiner sequence. Truncate the semantic value at an allowed boundary, rebuild the production representation, and measure again. ## Reasoned examples These are reasoned encoding examples, not executed tests. Assume an inclusive five-byte ceiling on the final field value, ordinary UTF-8, no normalization or escaping, and grapheme-cluster truncation. - The value made of ASCII A, grinning face, and ASCII B occupies six bytes: one plus four plus one. - Without an omission marker, the longest fitting grapheme prefix is ASCII A followed by grinning face, which occupies exactly five bytes. - With a single ellipsis character as the required marker, the marker occupies three bytes. ASCII A followed by the marker occupies four bytes, while ASCII A followed by grinning face and the marker occupies eight bytes. The longest fitting shortened result is therefore ASCII A followed by the marker. A decomposed e followed by a combining acute accent commonly forms one grapheme cluster and occupies three UTF-8 bytes. Treating its two code points as independent truncation positions can change the visible text even when both byte slices happen to decode. These counts do not establish the size of a JSON body, an ASCII-only escape mode, a normalized value, a compressed message, or a framed transport. Measure those exact representations when the contract includes them. ## Verify the procedure For the concrete implementation, test and record: - A value already within the byte and semantic limits. - A value whose longest prefix lands exactly on the inclusive byte ceiling. - A case where the next complete semantic unit exceeds the ceiling. - Combining marks, emoji modifier sequences, joiner sequences, and regional-indicator pairs. - Precomposed and decomposed text under the documented normalization policy. - An omission marker that changes which prefix fits. - Quotes, backslashes, control characters, or other values whose serializer representation differs from the source string. - A marker-only candidate and a case where even that candidate is too large. - Any pipeline feature that could make encoded size non-monotonic. For a selected result, assert that the full production representation fits. Under a monotonic pipeline, also assert that adding the next complete semantic unit fails. Without a monotonicity proof, establish maximality by checking every longer permitted prefix or by another exhaustive equivalent. Record actual runtime observations only after the tests execute. Until then, describe expected sizes and outcomes as reasoned examples. ## Failure reporting Report the original semantic measurement, selected semantic measurement, final encoded byte count, byte ceiling, counting unit, measured boundary, and whether a marker was inserted. Avoid logging the rejected text merely to explain its size. ## Supporting basis and limitations A bounded knowledge search and full reads of the three closest skills found broad byte-versus-character validation, a four-outcome independence matrix, and streaming chunk validation. None specified maximal-prefix selection with a marker and a monotonicity gate. The procedure and numeric examples below are reasoned from ordinary Unicode segmentation, UTF-8 widths, and exact candidate serialization. No executable tests, packet captures, production measurements, external sources, conversation evidence, or review approval were used. ## Change and rationale Create standalone guidance for selecting a maximal whole-grapheme prefix under a UTF-8 network-payload ceiling, including exact candidate measurement, omission-marker budgeting, monotonicity safeguards, and maximality checks. Existing guidance establishes separate Unicode and UTF-8 measurements, exact production serialization, complete-unit shortening, crossed validation tests, and streaming decoder state. It does not provide a concrete procedure for choosing the longest safe shortened value, accounting for a suffix marker, or handling pipelines whose final encoded size has not been proven monotonic. This skill addresses that narrower operational gap without replacing the general validation guidance.