Shape partial aggregate responses so a naive client cannot read an unavailable source as empty

When to use

Use this when a service, endpoint or agent tool combines several independent sources and returns the combined answer to consumers you do not control: other teams, generated SDK clients, mobile apps, spreadsheets, or an agent harness that reads tool output. It assumes the server already models each source outcome as ok, empty, unavailable or skipped with a completeness flag, and keeps the sources that succeeded. It covers the next narrower question: how to lay out the response so that the distinction survives a client that never looks at the status field.

The failure pattern

The server does everything right internally and still ships an outage as an empty answer, because the wire shape lets the client collapse the states again:

1. Empty collection for a failed source. The per-source entry carries status unavailable, but its items field is an empty list because the serializer requires the field. A client that reads only items sees nothing and renders "no results".
2. Default-to-empty deserializers. A generated client marks the items field as required and fills a missing value with an empty list. Even when the server omits the field, the client materializes an empty answer.
3. Partial marked only deep inside. The overall result is flagged partial only within per-source entries. A client that reads the merged top-level list has no single field to check.
4. Wrong transport status. A partial answer is sent as a 5xx. Retry logic and circuit breakers discard the good data and retry everything, so the healthy sources are hit again during the outage. Or the opposite: a total failure is sent as a 200 with an empty merged list.
5. Retrofit onto a bare-list endpoint. A status field is added to an endpoint that used to return a plain list. Old clients ignore the new field and keep treating partial as complete.

Rule

Make the naive reading of the response fail safe. A consumer that reads only the data fields must end up with "unknown", an error or an obviously missing value, never with a confirmed empty answer, whenever a source was unavailable or incomplete. Carry the overall status and the list of missing sources at the top level of the body. Use the transport status only to separate usable from unusable responses.

How to apply

1. Never serialize an empty collection for an unavailable or skipped source. Omit the items field or send null. Reserve an empty list for the empty status only. If the schema language supports variants, model the per-source outcome as a tagged union where only the ok and empty variants carry an items field at all.
2. Put one overall status at the top level. Include overall as complete, partial or failed, a boolean complete flag, and a missing list naming each source that was unavailable, incomplete or skipped with a short classified reason such as timeout, auth, rate_limit, malformed or disabled. Do not send raw error strings; they leak host names, paths and internal identifiers.
3. Keep the merged list honest. If a merged items list is offered for convenience, document that it contains only items from ok sources and that absence from it proves nothing unless complete is true. Consider not offering a merged list at all for endpoints where consumers make absence-based decisions.
4. Choose the transport status by usability, not by perfection. Complete and partial responses are 2xx with the body-level status carrying the detail. A failed response, meaning no usable source or an essential source unavailable, is a 5xx with no data body that could be mistaken for content. Reasons: retry and breaker logic keys on the transport status, and a partial 5xx throws away good data and amplifies load on the failing source. A 200 with an empty merged list for total failure is the flattened-error bug at the transport layer.
5. Add a degraded marker outside the body. Set a response header or equivalent metadata that names the missing sources, so proxies, logs and intermediate caches can see degradation without parsing the body. Give partial responses a private or no-store cache directive so an intermediate cache does not store them with the normal lifetime.
6. Fix the client side of the contract. In generated clients and hand-written parsers, declare the items field optional or nullable with no default. Provide one accessor that returns the items for ok and empty outcomes and raises or returns an explicit unknown value for the others. Reject code that reads a nullable items field with an or-empty default.
7. Do not retrofit partial onto a bare-list contract. If an existing endpoint returns a plain list, old clients cannot see any new status field. Either version the endpoint and return the structured shape only to callers that request the new version, or keep the old endpoint all-or-nothing and return a 5xx when any source fails. Never start returning a shorter plain list to old clients.
8. For agent tool output, say it first. When the consumer is a language model reading text, place a one-line degradation notice at the very start of the text output, naming the missing sources, and also include the structured fields. A notice appended after a long list is easily dropped from a summary.
9. Keep the shape identical across success and degradation. Do not switch to a different top-level shape when degraded. Clients written against the success shape must parse the degraded one and find the status where they expect it.

Minimal shape (language-neutral)

response body = { overall: complete or partial or failed, complete: true or false, missing: [ { source, reason: timeout or auth or rate_limit or malformed or incomplete or disabled } ], sources: [ { name: "a", status: ok, complete: true, items: [...] }, { name: "b", status: empty, complete: true, items: [] }, { name: "c", status: unavailable, complete: false } no items field at all ], items: merged items from ok sources only }

Transport: 2xx for complete and partial, 5xx for failed, plus a degraded header on partial.

Reasoned example

A search aggregator combines a product index, a help-center index and a community forum. The forum service goes down. The server correctly marks the forum unavailable, but the serializer always emits items as an empty list, and the mobile client was generated from a schema where items is required. The mobile app shows a search with product and help results and a forum section that says "no discussions yet". A support macro that reads the forum count decides to open a new thread because none exist, and creates duplicates all afternoon.

With the shape above, the forum entry has no items field, the client accessor returns unknown for it, the app renders "forum results unavailable", the top-level complete flag is false, and the macro checks that flag and skips thread creation. The response is still a 200, so the app does not retry the product and help indexes, and a degraded header lets the edge cache bypass storage.

This example is reasoning about the procedure. It is not the result of an executed test.

Checks before shipping

These are suggested verification steps for adopters, not observed results.

• Serialize a response with one unavailable source and inspect the raw bytes. The unavailable entry must have no items field or a null one, never an empty list.
• Generate or compile the client from the published schema and read items for the unavailable source without checking status. The read must fail or return an explicit unknown, not an empty collection.
• Force every source to fail. The transport status must be 5xx and the body must not contain a merged items list.
• Force one source to fail. The transport status must be 2xx, the top-level complete flag false, and the missing list must name that source with a classified reason and no raw error text.
• Send a partial response through any intermediate cache or proxy and confirm the degraded marker survives and the response is not stored as fresh.
• Call the endpoint from an old client that predates the status field, with one source failing, and confirm the old client receives an error rather than a shorter list.

Pitfalls

• Schema tools that make every array field required and default it to empty.
• Sending partial as 5xx, which discards the good sources and multiplies retries during an outage.
• Sending total failure as 200 with an empty list.
• Marking partial only inside per-source entries and offering a top-level merged list with no flag beside it.
• Raw exception messages in per-source reasons.
• A different top-level shape for degraded responses that clients written against the success shape cannot parse.
• Adding a status field to a plain-list endpoint and assuming old clients will honor it.
• Tool output for an agent that mentions degradation only at the end of a long listing.