Classify each dependency reply into ok, empty, unavailable or skipped with a per-source mapping table

When to use

Use this at the point where one source's raw reply is turned into a status for the caller: the HTTP or RPC client wrapper, the database or search adapter, the tool-result handler in an agent, the file or queue reader. It applies whenever the caller downstream models each source as ok, empty, unavailable or skipped and keeps partial results.

This skill assumes that model already exists. It covers the earlier, narrower question that the model leaves open: given one concrete reply from one concrete source, which state is it, and how do you decide that consistently instead of case by case?

The failure pattern

The state model is only as good as the classification feeding it. The same misclassification keeps reappearing in different clothes:

1. Transport status read as meaning. A not-found status is mapped to empty everywhere. On a collection lookup that may be right. On a route behind a gateway during a deploy, or on a resource the caller is not allowed to see, it is an outage or a permission problem wearing an empty answer's clothes.
2. Success status with an error body. The source answers with a success status and an error envelope in the body. A handler that checks only the status treats the envelope as data, finds no items, and reports empty.
3. Missing field defaulted to empty. The body parses, but the field holding the items is absent because of schema drift or a partial serialization. A default of an empty collection fills the gap, and the absence of the field becomes the absence of data.
4. Free-text tool output. An agent tool returns a sentence. A handler looks for the word results, finds none, and concludes there are none. The sentence was an error message.
5. Catch-all fallback to empty. Any unrecognized signal lands in a final branch that returns an empty collection because that keeps the caller's type happy.
6. Full page read as unfinished, or unfinished page read as full. A page that holds exactly its size cap arrives with no continuation token. One handler marks it incomplete and the aggregate never becomes complete for that total. Another handler marks it complete on a source that only sometimes sends the token, and the remaining pages are silently dropped.

Each of the first five produces an empty state without the source ever confirming that there is nothing. The sixth produces a wrong completeness flag. The downstream model then trusts it.

Rule

Empty is a positive claim and needs positive evidence from the source's own contract. Unavailable is the default for everything that is not positively ok or positively empty. Completeness is likewise a positive claim: a reply is complete only when the source's contract says so, not when a marker merely happens to be absent. Classification is written down per source as an explicit table of signal to state to reason code, kept next to that source's adapter and reviewed whenever the source's contract changes.

Four consequences follow:

• A reply may be classified empty only when the source answered on the expected route, in the expected shape, and the shape itself says there are no items.
• A reply may be classified ok only when the body validates against the expected shape and contains at least one item.
• A reply may be marked complete only when the contract guarantees that an incomplete reply would have carried a marker, and no marker is present.
• Any signal that is not in the table is unavailable, with a reason code of unclassified, and is logged so the table can be extended. It is never empty.

How to apply

1. Write the table before the handler. For each source, list every signal you expect: transport failures, each status code or code family, body validation outcomes, pagination markers, and, for tools, each documented result shape. Give each row a state and a short reason code. Rows for empty must cite the evidence that makes the answer positive. Rows that set completeness true must cite the contract clause that makes the absence of a marker meaningful.
2. Layer the checks in order and stop at the first failure. Transport first (resolution, connection, TLS, timeout, cancellation, circuit open). Then protocol status. Then content type. Then body parse. Then schema validation. Then the item count. A failure at any layer is unavailable with that layer's reason code. Only a reply that passes every layer reaches the ok or empty decision.
3. Treat not-found by route class, not globally. Decide separately for each route whether not-found can mean empty. A lookup of one resource by identifier, on a route the adapter has otherwise confirmed working, may map not-found to empty. A collection or search route should return a success status with an empty collection when there is nothing, so not-found there is unavailable. If the not-found body does not match the source's own error format, for example an HTML page or a gateway signature, it is unavailable regardless of route.
4. Keep permission problems out of empty. Unauthorized and forbidden are unavailable with an auth reason code. Some sources hide the existence of a resource behind not-found when the caller lacks permission. For those sources, not-found may map to empty only when permission has been established some other way; otherwise map it to unavailable with reason possibly-forbidden.
5. Validate the body before counting. Require the item field to be present and of the expected type. A missing or wrongly typed field is unavailable with reason malformed, not empty. A success status carrying the source's error envelope is unavailable with the envelope's error code as the reason.
6. Detect partial replies at classification time, and decide the capped-page case by the contract's marker guarantee. Partial content status, a present continuation token, or a truncated stream classify as ok or empty with completeness false. A page filled to its size cap with no marker is the boundary case, and the count alone cannot settle it. Record for each source whether its contract guarantees a marker on every non-final page. Where the guarantee exists, a full page with no marker is ok and complete. Where the contract is silent, or documents the marker as best effort, a full page with no marker is ok with completeness false and reason partial, and the adapter requests one more page; an empty next page with no marker then confirms completeness. Do not wait for the aggregator to guess this later.
7. Treat rate limits, overload and circuit-open as unavailable. They mean the source refused to look. Give them their own reason codes so retry policy can tell them from hard failures.
8. Classify tool results by structure, not by wording. An agent tool may return empty only if its contract returns a structured zero marker: a count field of zero, an empty array, or an explicit no-match flag. Free text that mentions no results is unavailable with reason unstructured unless the tool's contract guarantees that exact string. Tool errors surfaced as text are unavailable with reason tool-error.
9. Carry the reason code through. The state and reason code travel with the source outcome to the aggregator, the cache and the consumer. A state without a reason cannot be audited or retried sensibly.
10. Log and count classification misses. Every hit on the default unavailable row emits a metric and a log line with the signal that was seen. Review these regularly; each one is either a new table row or a source contract change.

Minimal shape (language-neutral)

classification table for one source = [ { signal: "dns or connect failure", state: unavailable, reason: "transport" }, { signal: "timeout or cancelled", state: unavailable, reason: "timeout" }, { signal: "circuit open", state: skipped, reason: "circuit-open" }, { signal: "status 401 or 403", state: unavailable, reason: "auth" }, { signal: "status 429 or 503", state: unavailable, reason: "overloaded" }, { signal: "status 5xx other", state: unavailable, reason: "server-error" }, { signal: "status 404, item route, body matches source not-found format", state: empty, reason: "not-found-item" }, { signal: "status 404, collection route or foreign body", state: unavailable, reason: "route-missing" }, { signal: "status 200, error envelope", state: unavailable, reason: "envelope-error" }, { signal: "status 200, body fails schema", state: unavailable, reason: "malformed" }, { signal: "status 200, items array length 0", state: empty, reason: "confirmed-empty" }, { signal: "status 200, items below cap, no marker", state: ok, complete: true, reason: "ok" }, { signal: "status 206 or continuation token present", state: ok or empty, complete: false, reason: "partial" }, { signal: "status 200, items at cap, no marker, contract guarantees marker on non-final pages", state: ok, complete: true, reason: "ok-last-page" }, { signal: "status 200, items at cap, no marker, contract silent or best effort", state: ok, complete: false, reason: "partial-unmarked" }, { signal: "anything else", state: unavailable, reason: "unclassified" } ]

marker guarantee for one source = one of guaranteed (every non-final page carries a marker; absence means last page) best-effort (marker may be omitted; absence proves nothing) unknown (treat as best-effort until the contract is confirmed)

Whether circuit-open is skipped or unavailable is a per-source choice. Skipped says the adapter chose not to call; unavailable says it tried. Either way it is not empty.

Reasoned examples

Gateway deploy and a global not-found rule

A search feature fans out to a product index behind an API gateway. During a rolling deploy the index route is briefly unregistered, and the gateway answers not-found with an HTML body for a few seconds.

With a global rule that not-found means empty: every search during that window reports zero products from the index. The aggregator marks the source empty and complete. A page shows no products. A job that removes search boosts for products with no index hits reads the empty answer and strips the boosts.

With a per-source table: the row for not-found on a collection route maps to unavailable, and the body check fails the source's error format anyway. The index outcome is unavailable with reason route-missing, the aggregate is partial, the page says the product index did not respond, and the boost job sees unavailable and skips the run.

Exactly fifty items and no continuation token

A listing adapter reads a collection with a page size of fifty. The source returns fifty items and no continuation token. The same bytes must classify differently depending on the source's marker guarantee.

Source one documents that every page except the last carries a token. Its marker guarantee is recorded as guaranteed. The reply is ok, reason ok-last-page, completeness true. The aggregator may treat the collection as fully read. An absence-based cleanup that waits for a complete read proceeds normally.

Source two documents pagination but says nothing about whether the token is always present, and its adapter has once observed a page with no token followed by more data. Its marker guarantee is recorded as best-effort. The same reply is ok, reason partial-unmarked, completeness false. The adapter requests page two. If page two is an empty collection with no token, the adapter now has positive evidence of the end and marks the read complete. If page two carries items, the adapter continues and the earlier decision stands as correct.

Had source one been handled by the base rule alone, every collection whose total is an exact multiple of fifty would have stayed incomplete forever, and the cleanup job would silently never run for those collections. Had source two been handled by a rule that reads no token as last page, the remainder of the collection would have been dropped without any signal. The marker guarantee is the fact that separates the two, and it belongs in the table, not in the handler.

These examples are reasoning about the procedure. They are not the result of executed tests.

Checks before shipping

These are suggested verification steps for adopters, not observed results.

• For each row of the table, inject that signal and assert the exact state and reason code. The default row must be reachable by a signal not otherwise listed.
• Return a success status with the source's error envelope. The state must be unavailable, not empty.
• Return a success status with the items field missing. The state must be unavailable with reason malformed.
• Return not-found with an HTML body on an item route. The state must be unavailable, not empty.
• Return a page filled to its size cap with no end marker on a source whose marker guarantee is guaranteed. The state must be ok with completeness true and reason ok-last-page.
• Return the same page on a source whose marker guarantee is best-effort or unknown. The state must be ok with completeness false and reason partial-unmarked, and the adapter must issue one more page request.
• Return a page below its size cap with no marker. The state must be ok with completeness true regardless of the marker guarantee.
• For an agent tool, return the sentence that no results were found without the structured zero marker. The state must be unavailable with reason unstructured.
• Confirm the reason code survives serialization to the aggregator and to the consumer.

Pitfalls

• A default branch that returns an empty collection to satisfy a return type.
• Checking the status code and never parsing or validating the body.
• Optional field access with an empty default around the items field.
• One shared not-found rule across every route of a source.
• Mapping forbidden to empty because the caller has nothing it is allowed to see.
• Treating a rate limit as a hard failure and a hard failure as a rate limit; both are unavailable, but their reason codes drive different retry behaviour.
• Matching tool output on wording that the tool never promised.
• Deciding completeness from the item count alone. A full page proves nothing either way; only the contract's marker guarantee does.
• Recording a marker guarantee as guaranteed because it has always held so far. Observation is not a contract; if the source has not written it down, record it as unknown.
• Extending the handler with a new special case without adding the row to the table, so the table stops describing the code.