agent cited a deprecated help article: freshness check

TL;DR

Agents cite whatever retrieval returns, and retrieval doesn't know an article was deprecated last quarter. The customer follows dead instructions and blames you. The fix is a freshness check: mark deprecated articles so they're excluded from retrieval, verify the cited article is current before sending, and keep a redirect from old URLs to their replacements.

The query

agent cited a deprecated help article: freshness check

Use this when

• Agents link outdated help articles to customers
• Customers follow dead or wrong instructions
• You're designing agent knowledge retrieval

Not for

• Writing help articles
• Help center SEO
• Training human agents on article use

Steps

1. Mark deprecated articles as excluded

In your knowledge base, flag deprecated articles so the retriever skips them. A deprecation flag beats deleting: the article stays for history but never reaches a customer. Audit the flags quarterly.

Expected output: deprecated articles excluded from retrieval results.

2. Check freshness at citation time

Before the agent sends a citation, verify the article is current: not deprecated, not superseded by a newer version. If it fails, retrieve again excluding it. This is the freshness check proper.

Expected output: no citation sent without a current-status verification.

3. Redirect old URLs to replacements

When an article is deprecated, point its URL at the replacement. Customers with old links, and agents with cached citations, land on current instructions instead of dead ones.

Expected output: redirects from every deprecated article to its replacement.

4. Review citation logs for staleness

Log every article the agent cites and review the list monthly. Repeat citations of the same old article mean the deprecation flag is missing or the replacement isn't ranking. The log is your early warning.

Expected output: a monthly citation review catching stale references.

Variant phrasings

ai agent citing old documentation

Steps 1 and 2: exclude deprecated, verify at send time.

chatbot links to deprecated help center article

Step 3's redirects fix the cached-link cases.

Why it happens

Knowledge bases accumulate: new articles get written, old ones linger, and nobody owns deprecation. Retrieval ranks by relevance, and an old article can be very relevant to the query while being completely wrong for the customer. Freshness is metadata the retriever needs but the author never adds, so the check has to live in the pipeline.

Edge cases

• Versioned docs need per-version freshness. 'Current' depends on which product version the customer has.
• Deprecation flags must propagate to every retrieval index, including cached embeddings. Re-index after flagging.
• Some old articles stay valid (concepts). Deprecate by accuracy, not by age.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_sNhD9vBJjWrRNnuz7nUTMA