Cache fan-out results per source and serve stale fallback with an explicit age marker

When to use

Use this when an operation combines results from two or more independent sources and there is a cache anywhere in front of it: a response cache, an in-process memo, a CDN or edge cache, a materialized view, or an agent's tool-result memo. Typical cases are dashboards, product or profile pages built from several services, federated search, and agent workflows that fan out to tools and remember the answers.

This skill assumes the caller already models each source's outcome as ok, empty, unavailable or skipped, with a separate completeness flag, and keeps partial results instead of failing the whole operation. It covers the next, narrower question: what may be written to the cache, what comes out of the cache when a source is down, and how the consumer can tell a fallback from a fresh answer.

The failure pattern

Caching turns a short outage into a longer one in four common ways:

1. Poisoned aggregate. The assembled answer is cached under one key. A partial answer produced during an outage is stored under the same key as a complete one, so the outage now lasts one extra cache lifetime. If the partial flag is dropped during serialization, nobody can even see that it is partial.
2. Refuse-to-cache thrash. To avoid poisoning, the layer caches nothing whenever any source failed. During an outage every request re-hits every healthy source and the failing one, so load rises exactly when the system is weakest.
3. Negative cache of an outage. A caught error becomes an empty list, and the empty list is written as a negative result. The outage now looks like a confirmed absence for the whole cache lifetime, and any later absence-based logic trusts it.
4. Silent stale fallback. A stale copy is served in place of the unavailable source with no age and no marker. The consumer cannot tell fresh from fallback, and metrics report the source as healthy.

Rule

Cache per source, not per aggregate. Give each cached source entry the same status vocabulary as a live outcome plus an observed-at time. Only ok and empty outcomes may be written. Unavailable is never written and never overwrites or deletes a prior entry. When a live call is unavailable, a prior ok or empty entry may be served as a fallback only if its age is within a per-source stale ceiling, and it must be labelled stale with its age. Stale entries support display and additive work. They never support a decision that something is absent.

How to apply

1. Key entries per source and per request scope. The key includes the source name, the normalized request parameters that affect that source, and a schema version. Assemble the aggregate after the per-source lookups, never before.
2. Store the outcome, not just the payload. Each entry holds the status (ok or empty), the completeness flag, the items, the observed-at time, any freshness evidence the source gave (cursor watermark, replication position, index build marker), and its lifetime. Write empty outcomes explicitly, so that a cache miss and a cached empty answer are different things.
3. Never write unavailable. On timeout, connection error, auth failure, rate limit or malformed reply, write nothing and delete nothing. The prior entry, if any, is the only fallback candidate and must survive the failure.
4. Use two horizons per source. A fresh lifetime says how long an entry may be served without a live call. A stale ceiling says the maximum age at which an entry may still be served when the live call fails. The stale ceiling is longer than the fresh lifetime and is chosen from how long that source's data can be out of date before showing it does harm: seconds for a balance, minutes for a price, hours for a review list. Beyond the stale ceiling the source is reported unavailable with no items, even though an old entry exists.
5. Serve the fallback with labels. When the live call fails and a prior entry is within the stale ceiling, return that entry with its stored status (ok or empty), a stale flag set to true, its age, a served-from value of stale cache, and the live failure reason. Do not upgrade an empty entry to ok or downgrade an ok entry to empty.
6. Mark the overall result as degraded. If any source was served from stale cache, the overall result is degraded, not complete, and the response names those sources and their ages. The consumer can then say "reviews as of twelve minutes ago" instead of presenting them as current.
7. Refresh in the background with single flight. After serving stale, attempt one refresh per key at a time, guarded by a lock or in-flight map, so a popular key does not send many concurrent retries at a source that is already struggling. A successful refresh replaces the entry. A failed refresh leaves it untouched.
8. Carry the labels across every boundary. Serialization must preserve status, stale flag, age and observed-at time. On an HTTP edge, a degraded aggregate should be marked so intermediate caches do not store it with the normal freshness lifetime, for example by using a private or no-store cache directive and a warning header. In metrics, count stale serves separately from ok and unavailable. A stale serve is never a healthy source call.
9. Guard absence decisions. Any consumer that deletes, deduplicates, deprovisions or creates-if-missing must require status ok or empty, completeness confirmed, and the stale flag false. A fresh-cached empty entry is acceptable for such a decision only if the decision tolerates data as old as the fresh lifetime. If it does not, bypass the cache for that decision and go live.

Minimal shape (language-neutral)

source outcome = { name, status: ok or empty or unavailable or skipped, complete: true or false, stale: true or false, servedfrom: live or freshcache or stalecache, ageseconds, observed_at, items, reason }

cache entry (only ever ok or empty) = { status, complete, items, observedat, freshuntil, staleuntil, freshnessevidence }

Reasoned example

A product page combines inventory, pricing and reviews. The reviews service is down for twenty minutes.

With a whole-aggregate cache of five minutes and a catch handler that returns an empty list: the first request during the outage caches a page saying there are no reviews yet. Every visitor for the next five minutes sees a product with no reviews, and the next fill of the cache repeats the mistake until the service recovers. A nightly job that removes review summaries for products with zero reviews reads that empty answer and wipes the summary.

With per-source caching and a one-hour stale ceiling for reviews: the reviews entry observed three minutes before the outage is served with the stale flag set and an age that grows through the outage. The page shows the reviews labelled as possibly out of date. The overall status is degraded, so the nightly job sees the stale flag and skips the product. A single-flight background refresh replaces the entry the moment the service answers again.

This example is reasoning about the procedure. It is not the result of an executed test.

Checks before shipping

These are suggested verification steps for adopters, not observed results.

• Force one source to fail with a warm cache. The response should contain the prior items, the stale flag, the age and a degraded overall status, and the cache entry should be unchanged afterwards.
• Force one source to fail with a cold cache. The source should be reported unavailable with no items, not empty, and no entry should be written.
• Make a source return a genuinely empty answer. An empty entry should be written. A later failure should serve it as stale empty, not as unavailable and not as fresh empty.
• Age a stale entry past its ceiling and fail the live call. The source should be reported unavailable even though an entry exists.
• Hit a degraded key concurrently. Exactly one refresh call should reach the failing source per refresh attempt.
• Inspect the serialized response and any intermediate cache. The stale flag and age should survive, and the degraded aggregate should not be stored as a fresh complete one.

Pitfalls

• Caching the assembled aggregate under a key that does not encode which sources were present.
• Writing an empty entry from a catch handler.
• Deleting the prior entry on error, which throws away the only fallback.
• One lifetime doing both jobs: a short one makes fallback useless, a long one serves stale data as fresh.
• Serving stale without an age, or logging the age but not returning it to the consumer.
• Letting an HTTP or CDN cache store the degraded response with the normal freshness lifetime.
• Treating a fresh-cached empty answer as good enough for a delete decision without checking that the decision tolerates the cache age.