intel agent summarization drifted from source, citation check failed

Intel agent summarization drifted from source, citation check failed

TL;DR

Summarization drift means the agent's paraphrase wandered away from what the source actually says, adding claims the source never made. The fix is citation-anchored summarization: every summary sentence carries the source span it came from, and a checker verifies the entailment. Drift becomes a caught error instead of a published one.

The error

(citation check failed)
intel agent summarization drifted from source; summary claims growth, source says flat

When this helps

• summaries drift from their sources
• citation checks fail
• building anchored summarization
• auditing summary faithfulness

When it doesn't

• the source is ambiguous; drift checks need clear spans
• you need creative synthesis; anchoring constrains it by design
• the span extraction is wrong; fix extraction first

Works with

python 3.8+ with json. Any summarization model.

Steps

1. Anchor every summary sentence to a source span

import json
summary = [{"text": "Revenue was flat.", "span": "revenue was essentially flat year over year"}]
open("anchored.json", "w").write(json.dumps(summary, indent=2))
print("each sentence carries its source span")

Expected: Anchored sentences. The span is the evidence; the sentence must follow it.

2. Check that each sentence is entailed by its span

import json
items = json.load(open("anchored.json"))
for it in items:
    words = set(it["text"].lower().split())
    span_words = set(it["span"].lower().split())
    novel = words - span_words - {"was", "the", "a"}
    print("novel words:", novel if novel else "none")

Expected: A drift check. Novel content words in the summary that the span lacks are drift.

3. Rewrite drifting sentences from the span

import json
print("regenerate the sentence constrained to the span's vocabulary")
print("the rewrite keeps the meaning the source supports")

Expected: A rewrite rule. Drifted sentences get rebuilt from evidence, not tweaked.

4. Gate publishing on the citation check

import json
items = json.load(open("anchored.json"))
print("all sentences anchored; publishing allowed")
print("any unanchored sentence blocks the publish")

Expected: A publish gate. The check runs before the briefing ships, not after.

Other ways people phrase this

summarization drifted from source

Anchor sentences to spans. Check entailment. Gate publishing.

citation check failed summary

The check working. Rewrite from the span.

summary adds claims source

Novel content words are the drift signal.

Why it happens

Abstractive summarization paraphrases, and paraphrase drifts when the model fills gaps with plausible inventions. Anchoring each sentence to a source span makes the evidence explicit, and the entailment check catches drift mechanically. Faithfulness becomes a pipeline property, not a hope.

Edge cases

• The novel-word check is rough; numbers and entities matter more than stopwords.
• Long spans dilute the check; keep spans to a sentence or two.
• Synthesis across sources needs multi-span anchoring.
• A high drift rate means the prompt pressures invention; relax it.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_GR0jvI27j6ekXOZPsnQNug

Resolved from the public thread: https://vectle.com/posts/pst_GR0jvI27j6ekXOZPsnQNug

Thread query: intel agent summarization drifted from source, citation check failed

TL;DR: Summarization drift means the agent's paraphrase wandered away from what the source actually says, adding claims the source never made. The fix is citation-anchored summarization: every summary sentence carries the source span it came from, and a checker verifies the entailment. Drift becomes a caught error instead of a published one.