## TL;DR
A summary that calls an error-filled backup healthy is generated from vibes, not data, usually because the summariser receives raw logs and a cheerful template. Feed the summariser only the parsed outcome records, force it to cite the decisive status lines for every claim, and add a post-check that rejects any summary whose health claim conflicts with the parsed outcomes.

```text
agent's summary hallucinated a healthy backup despite the error in the logs
```

## Use this when
- Stakeholders receive healthy summaries for nights the console shows failed.
- Summaries omit failed objects that the parser did extract correctly.
- Summary tone stays positive regardless of input severity.

## Not for this skill when
- The parser never found the errors in rotated or multiline logs. Fix extraction first.
- The summary is accurate but distribution sends an old cached version. Delivery problem.

## Steps
1. Trace one bad summary back through its inputs to see whether parsed outcomes or the summariser caused the lie. Verify: the faulty stage is identified
2. Restrict summariser input to parsed outcome records with counts and per-object statuses. Verify: raw logs no longer reach the summariser directly
3. Require each health claim in the summary to reference the outcome records behind it. Verify: claims map to records in a sample output
4. Add a consistency gate comparing summary outcome words against parsed counts and blocking mismatches. Verify: a seeded mismatch is blocked in testing
5. Regenerate summaries for recent affected nights and issue corrections where stakeholders were misled. Verify: corrected summaries are distributed

## Variant phrasings
### agent summary contradicts logs healthy false
The contradiction phrasing.
### llm summary hallucination backup status gate
The gate variant.
### summary generation from parsed outcomes only
The input wording.

Compatibility: LLM-assisted reporting agents summarising backup outcomes for tickets, chat and email digests from structured parser output.

## Why it happens
Summarisation models smooth toward the statistically normal story, and most backup nights are healthy, so an unconstrained summariser literally expects success. Errors present as a few lines among thousands of normal ones get smoothed away. Constraining inputs to parsed outcomes and gating claims against counts removes the smoothing surface.

## Edge cases / pitfalls
- Never let the summariser see the previous night summary. It anchors on yesterday health.
- Counts in summaries should be copied programmatically, not generated. Numbers from models are decorative.
- A blocked summary should alert, not silently retry until it passes. The block is the signal.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_4RQDM4oK_4CKnbzingcyYA