Build Length-Prefixed UTF-8 Frames from the Encoded Payload

Use this procedure when an outbound protocol frame carries Unicode text or a textual serialization and its length prefix declares a number of payload octets. Apply a separate semantic text rule when the contract also limits grapheme clusters, Unicode code points, or another named Unicode unit.

The framing invariant is simple: the prefix must be derived from the exact payload byte buffer that will be sent. A character count, code-unit count, estimate, or earlier serialization cannot safely stand in for that buffer's byte length.

Freeze the three independent contracts

Record these details before building a frame:

• The semantic counting unit and the value to which it applies.
• Any required normalization and whether it occurs before semantic validation.
• The exact payload representation covered by the byte ceiling.
• Whether serialization, escaping, compression, checksums, or terminators are inside that payload representation.
• Whether the byte ceiling is inclusive.
• What the length prefix counts: payload bytes, the whole frame, or another precisely named region.
• The prefix format, byte order, canonical encoding rule, and largest representable value.

Keep the semantic maximum, payload byte maximum, and prefix capacity separate. A value may satisfy one or two of them while failing another.

Construct one frame

1. Start with the semantic value and apply only normalization required by the protocol contract.
2. Count the declared Unicode units and reject with a semantic-limit error if that maximum is exceeded.
3. Produce the complete payload representation with the production serializer and settings.
4. Encode that representation as UTF-8 exactly once and retain the resulting byte buffer.
5. Read the payload length from the retained buffer.
6. Reject with a payload-size error if the length exceeds the protocol's byte ceiling.
7. Reject with a framing-capacity error if the length cannot be represented by the prefix format. Do not wrap, truncate, clamp, or cast through a narrower integer.
8. Encode the prefix from that byte length using the required byte order or canonical variable-length form.
9. Assemble the frame and transmit the retained payload buffer. Do not reserialize or re-encode after deriving the prefix.

If the contract says the prefix counts the whole frame, compute that region exactly as specified. Do not silently substitute the common payload-only convention.

Preserve the measured bytes

The safest interface returns both the prefix and an immutable or otherwise retained payload buffer. This prevents later differences caused by changed serializer options, mutation, nondeterministic metadata, normalization, or Unicode escaping.

If a transport API requires separate writes for prefix and payload, separate writes do not change the declared payload length. Partial writes must be retried from the remaining bytes of the same buffers, not from newly generated text.

Keep failures distinct

Report at least these outcomes separately:

• Semantic Unicode limit exceeded.
• Payload UTF-8 byte ceiling exceeded.
• Length is not representable by the prefix format.
• Serialization or UTF-8 encoding failed.
• Transport write failed after a valid frame was constructed.

Diagnostics should name the semantic unit, byte boundary, measured payload length, and prefix format. Avoid logging the rejected text merely to explain its size.

Reasoned frame example

This is a reasoned calculation, not an executed test.

Assume:

• The payload is raw UTF-8 text with no normalization or escaping.
• The semantic maximum is two Unicode code points.
• The payload byte ceiling is inclusive at five bytes.
• The prefix is an unsigned two-byte big-endian integer that declares payload bytes only.

The value consisting of a grinning-face emoji followed by ASCII A has two Unicode code points, so it satisfies the semantic maximum. Its UTF-8 payload occupies five bytes: four for the emoji and one for A. The prefix is therefore the two bytes 00 05. The complete frame occupies seven bytes, but the prefix correctly declares five because it counts only the payload.

Under a four-byte payload ceiling, the same value must be rejected for encoded size even though its semantic count still passes. Deriving the prefix from the two-code-point count would incorrectly declare 00 02 and desynchronize a receiver.

Verification checklist

Execute implementation tests before claiming runtime evidence:

• An ASCII payload whose character and byte counts happen to match.
• A multibyte payload that passes the semantic limit but reaches the byte ceiling exactly.
• The same semantic shape one byte over the payload ceiling.
• The largest length representable by the prefix and the first unrepresentable length.
• Empty payload behavior.
• Fixed-width byte order or canonical variable-length prefix encoding, as applicable.
• Serializer settings that change escaping and therefore payload length.
• An assertion that the prefixed length equals the length of the exact buffer handed to the transport.
• A test that partial writes resume from retained bytes without regenerating the payload.
• A receiver-side integration check that consumes exactly the declared payload bytes and leaves the next frame aligned.

Until those tests are run, describe frame sizes and acceptance decisions only as reasoned expectations.