Keep partial results and tell empty data apart from an unavailable dependency

When to use

Use this when one operation collects data from two or more independent sources (services, databases, search indexes, files, tools) and puts it together for a caller, a UI or an agent. Typical cases are dashboards, federated search, aggregation endpoints, batch jobs and agent tool calls that fan out.

The failure pattern

Two wrong designs are common, and each one hides the other problem:

1. **All or nothing.** One failed source throws away everything, including results from the sources that worked. The caller gets an error even though most of the answer was available.

2. **Error flattened to empty.** A failed source is caught and swapped for an empty list, zero or null. The caller cannot tell "there is nothing" from "we could not look". An outage then looks like a real answer, and nobody notices it.

The second design is the more dangerous one, because it seems to work. Later logic such as "no records, so delete the cache", "no conflicts found, so proceed" or "no prior results, so create a new one" then acts on data that was never actually seen.

Rule

Treat every source's outcome as one of at least three separate states, and carry that state to the caller:

- **ok with data**: the source answered with one or more items.

- **ok and empty**: the source answered and confirmed there are no items. This is a real, trustworthy answer.

- **unavailable**: the source did not give a trustworthy answer (timeout, connection refused, auth failure, rate limit, malformed reply, cancelled). Its content is unknown, not empty.

If it helps, add a fourth state, **skipped**, for sources that were left out on purpose (disabled by configuration, not relevant, or out of budget). This keeps a choice not to look separate from a failure to look.

How to apply

1. **Model the outcome per source, not just the payload.** Return a structure that holds a status for each source next to its items, for example a list of entries with a source name, a status of ok, empty, unavailable or skipped, the items, and an optional short reason. Do not use a bare empty collection to mean failure.

2. **Isolate failures.** Run independent sources so that one source's error or timeout cannot cancel or discard the others. Give each source its own timeout, and collect settled outcomes instead of short-circuiting on the first rejection.

3. **Keep the successes.** Put together whatever came back as ok or empty, and mark the overall result as partial whenever any required source is unavailable.

4. **Choose the overall status on purpose.**

- Every source ok or empty: complete.

- Some sources unavailable, at least one usable: partial. Return the data and the list of missing sources.

- Every source unavailable: failed. Do not present this as an empty result.

- If a source is essential for correctness (for example, an authorization or conflict check), a failure there makes the whole operation fail or pause, not degrade.

5. **Keep the distinction at every boundary.** Serialization, caching, logging and UI layers often collapse the states again. Check that an "unavailable" status is not written to a cache as an empty value, is not rendered as "no results found", and is not counted as zero in metrics.

6. **Guard destructive or conclusive actions.** Before acting on an absence (deleting, deduplicating, declaring something missing, creating a duplicate), require that the relevant source reported ok and empty, not unavailable.

7. **Tell the consumer plainly.** In user-facing or agent-facing output, say which sources were missing and that the result may be incomplete. For example: "Showing results from 3 of 4 sources; the fourth timed out." Do not say "no results" when a source failed.

8. **Retry only the failed parts.** Because each source's status is tracked, a retry can go to just the unavailable sources and merge with the results already kept.

Minimal shape (language-neutral)

result = {

overall: complete | partial | failed,

sources: [

{ name: "a", status: ok, items: [...] },

{ name: "b", status: empty, items: [] },

{ name: "c", status: unavailable, reason: "timeout after 2s" },

],

items: merged items from sources with status ok

}

Checks before shipping

- Force one source to time out. The other sources' data should still appear, the result should be marked partial, and the failed source should be named.

- Make one source return a real empty answer. It should be reported as empty, not as unavailable, and should not trigger alerts.

- Force every source to fail. The overall result should be failed, not an empty success.

- Check that caches and downstream consumers keep the three states apart.

- Check that no destructive branch runs when the needed source is unavailable.

Pitfalls

- Catch-all handlers that return a default value such as an empty list or zero.

- Optional-chaining or null-coalescing defaults that quietly turn a missing reply into an empty one.

- Treating an HTTP 404 the same way for every endpoint: for a collection lookup it may mean empty, but for a service route it may mean misconfiguration. Decide per source what counts as a trustworthy empty answer.

- Log-only reporting. If the partial status is only logged and not returned, callers still act on incomplete data.