## TL;DR
A language-model-driven agent will happily complete a pattern: KB numbers look like KB plus seven digits, and an agent reasoning from memory can generate convincing fakes. The control is architectural, never trust an identifier the agent produced: every KB in a plan must be validated against the authoritative feed (the WSUS catalog data, the Update Catalog, or the vendor API) at plan time, and any identifier not found is rejected and logged as a validation event. Agent memory may suggest candidates; only the feed may confirm them.

```text
agent hallucinated patch kb numbers not in vendor feed: data error
```

## Use this when
- Plans or reports list KB IDs that return no results in the Update Catalog.
- The phantom KBs follow the right format and plausible date ranges, making them hard to spot by eye.
- The identifiers appear in narrative summaries the agent wrote, then get copied into plans.

## Not for this skill when
- The KB exists but is for a different product or architecture. That is a selection error, caught by the same feed validation with product checks.
- The KB is real but not yet in the local WSUS because sync has not run. Validation should distinguish not-in-feed from not-yet-synced by checking the upstream catalog too.

## Steps
1. Reproduce by collecting the agent recent plans and checking every KB ID against the catalog programmatically. Verify: the scale of the problem (count of invalid IDs) is measured, not anecdotal
2. Add a validation gate in the planner: every update identifier must resolve in the authoritative feed with matching product and architecture before it can enter a wave plan. Verify: the gate rejects a test plan containing a fabricated KB
3. Constrain the agent workflow so candidate updates originate from feed queries, with the agent selecting and sequencing rather than naming from memory. Verify: plan-building prompts and tools only accept feed-sourced identifiers
4. Log validation rejections as their own event type with the rejected ID and its source in the agent reasoning. Verify: rejections are countable in the run report, so hallucination rate becomes a metric
5. Clean historical records: mark past reports containing invalid KBs as corrected, with the real update state re-derived from host evidence. Verify: fleet compliance figures are restated from validated data only

## Variant phrasings
### agent hallucinated kb numbers not in catalog validate feed
The main phrasing.
### llm invented identifiers plan validation gate patch
The architecture phrasing.
### fake kb plausible format how to detect reject
The detection phrasing.

Compatibility: Compatibility: Any LLM-driven planning agent with a catalog or feed to validate against (WSUS, Update Catalog, vendor APIs).

## Why it happens
Language models generate text that fits patterns, and an identifier is just a pattern with a checkable referent. The failure is not that the model errs; it is that the pipeline treats generated identifiers as data. Mature agent designs separate generation from verification: the model proposes, the feed disposes. That separation also catches honest mistakes (typos, transposed digits, a real KB for the wrong product) with the same machinery, so the gate pays for itself beyond the hallucination case.

## Edge cases / pitfalls
- Never let the agent grade its own identifiers by asking it whether they are real; validation must leave the model and hit the feed.
- A hallucination metric that suddenly spikes usually means the feed query path broke and the agent is filling gaps; investigate the feed first.
- Keep rejected IDs in the log; they are the training signal for tightening the agent tools and prompts.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_nNy6glii4m6cKagqbm_E_A