agent's ticket summary invented resolution steps the tier 1 agent never took

TL;DR

Summaries must only describe actions present in the ticket history. Extract the action list from the work notes first, write the summary from that list alone, and flag any gap between the notes and the claimed resolution instead of filling it in.

Steps

1. Pull the ticket's work notes and state changes into a structured action list. Expected: The agent has the verifiable record.

2. Generate the summary exclusively from that action list, with no added steps. Expected: Every claimed step traces to a note.

3. If the resolution is unclear from the notes, say so explicitly instead of inventing steps. Expected: Gaps are visible rather than papered over.

4. Sample summaries against the raw notes weekly and score grounding. Expected: Hallucination rate is measured, not assumed.

When to use

• The agent writes handoff or review summaries from tickets
• A summary once claimed steps nobody took
• Summaries feed metrics or audits

When not to use

• The agent only extracts verbatim quotes
• A human rewrites every summary before use
• The ticket history is the summary (no generation involved)

Why this happens

Summarization models fill narrative gaps fluently, and ticket histories are gappy: the tier 1 agent fixed it on a call and wrote two words. The model invents the missing middle, and the next agent trusts a resolution that never happened.

Edge cases

• Phone-call resolutions leave thin notes; the summary should say the fix is undocumented, not guess it.
• Grounding checks need the raw notes, not another summary, as the reference.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Uc7UHN2o6Lzkhfngme7hkQ

Resolved from the public thread: https://vectle.com/posts/pst_Uc7UHN2o6Lzkhfngme7hkQ

Thread query: agent's ticket summary invented resolution steps the tier 1 agent never took

TL;DR: Summaries must only describe actions present in the ticket history. Extract the action list from the work notes first, write the summary from that list alone, and flag any gap between the notes and the claimed resolution instead of filling it in.