agent split one limitation clause across two extracted clauses — a page break mid-clause became a clause boundary

Not specified by the original author.

TL;DR

A page break fell mid-sentence in the limitation clause, and the agent treated it as a clause boundary, splitting one clause into two. Detect sentence continuation across page breaks (lowercase continuation, no terminal punctuation) and merge the fragments before clause segmentation.

The symptom

agent split one limitation clause across two extracted clauses  -  a page break mid-clause became a clause boundary

Steps

1. Spot the mid-sentence break

The first fragment ends without terminal punctuation and the second starts lowercase (or with a continuation word). That is one sentence, not two clauses. Expected: The split point identified as a page break, not a boundary.

2. Merge the fragments

Join the fragments across the page break, restoring the sentence. Expected: One complete clause text.

3. Re-run clause segmentation

Segment clauses on the merged text so numbering and boundaries reflect the real structure. Expected: Correct clause list with no phantom split.

4. Add the continuation rule

Before accepting a clause boundary at a page break, check for sentence continuation; if the sentence continues, merge first. Expected: Page-break splits blocked systematically.

When to use

• One clause appears as two fragments
• A fragment ends mid-sentence at a page break
• Clause counts are inflated by one

When not to use

• The two fragments are genuinely separate clauses (check punctuation and numbering)
• The break is at a section boundary (legitimate split)
• The document has no page breaks (different cause)

Why it happens

Clause segmenters often split on page or block boundaries as a heuristic. When a page break lands mid-sentence, the heuristic fires on layout instead of language, and one clause becomes two. Everything downstream (citations, diffs, counts) inherits the phantom split.

Edge cases

• Page break mid-word (hyphenation): rejoin the word when merging.
• Footnotes at the break: keep them separate; merge only the body sentence.
• Tables split across pages: stitch rows, not sentences.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Tocae4Ph16AcXDgP1PnzPw

Resolved from the public thread: https://vectle.com/posts/pst_Tocae4Ph16AcXDgP1PnzPw

Thread query: agent split one limitation clause across two extracted clauses - a page break mid-clause became a clause boundary

TL;DR: A page break fell mid-sentence in the limitation clause, and the agent treated it as a clause boundary, splitting one clause into two. Detect sentence continuation across page breaks (lowercase continuation, no terminal punctuation) and merge the fragments before clause segmentation.

Not specified by the original author.

TL;DR

A page break fell mid-sentence in the limitation clause, and the agent treated it as a clause boundary, splitting one clause into two. Detect sentence continuation across page breaks (lowercase continuation, no terminal punctuation) and merge the fragments before clause segmentation.

The symptom

agent split one limitation clause across two extracted clauses  -  a page break mid-clause became a clause boundary

Steps

1. Spot the mid-sentence break

The first fragment ends without terminal punctuation and the second starts lowercase (or with a continuation word). That is one sentence, not two clauses. Expected: The split point identified as a page break, not a boundary.

2. Merge the fragments

Join the fragments across the page break, restoring the sentence. Expected: One complete clause text.

3. Re-run clause segmentation

Segment clauses on the merged text so numbering and boundaries reflect the real structure. Expected: Correct clause list with no phantom split.

4. Add the continuation rule

Before accepting a clause boundary at a page break, check for sentence continuation; if the sentence continues, merge first. Expected: Page-break splits blocked systematically.

When to use

• One clause appears as two fragments
• A fragment ends mid-sentence at a page break
• Clause counts are inflated by one

When not to use

• The two fragments are genuinely separate clauses (check punctuation and numbering)
• The break is at a section boundary (legitimate split)
• The document has no page breaks (different cause)

Why it happens

Clause segmenters often split on page or block boundaries as a heuristic. When a page break lands mid-sentence, the heuristic fires on layout instead of language, and one clause becomes two. Everything downstream (citations, diffs, counts) inherits the phantom split.

Edge cases

• Page break mid-word (hyphenation): rejoin the word when merging.
• Footnotes at the break: keep them separate; merge only the body sentence.
• Tables split across pages: stitch rows, not sentences.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_HBEMJZmJ6Wsb-VW3OxAE7g

Resolved from the public thread: https://vectle.com/posts/pst_HBEMJZmJ6Wsb-VW3OxAE7g

Thread query: agent split one limitation clause across two extracted clauses - a page break mid-clause became a clause boundary

TL;DR: A page break fell mid-sentence in the limitation clause, and the agent treated it as a clause boundary, splitting one clause into two. Detect sentence continuation across page breaks (lowercase continuation, no terminal punctuation) and merge the fragments before clause segmentation.

Not specified by the original author.

TL;DR: Switch the chunker from fixed character windows to sentence-aware or clause-aware splits with generous overlap, so no chunk starts or ends inside a sentence. Then re-index and spot-check retrieval on negation-heavy clauses like "shall not exceed". The fixed-size splitter had no idea where sentences end, so it sliced "shall not exceed" in half and the retriever handed the agent a fragment that reads like permission.

the agent's RAG chunking cut a sentence mid-clause  -  "shall not exceed" split into "shall" and "exceed" and the retriever returned only the "shall" half

1. Inspect where the current chunks break. Dump five chunks around a known clause boundary and check whether any chunk starts or ends mid-sentence. Expected: you find at least one chunk that begins or ends inside a sentence, confirming the splitter is the problem.
2. Replace the fixed-size splitter with a sentence-aware splitter (split on sentence boundaries first, then pack whole sentences into chunks up to the size limit). Set overlap to at least one full sentence or roughly 20 percent of the chunk size, whichever is larger. Expected: every chunk in the new index starts and ends on a sentence boundary.
3. For contracts specifically, prefer clause-aware splitting: split on section and subsection headings first, and never split inside a numbered clause. Keep definitions and the clause that references them in overlapping chunks when they sit close together. Expected: chunks line up with the document's own structure, so "shall not exceed" and its number always travel together.
4. Re-index the contract corpus and re-run the failing query. Check the retrieved chunks around the target clause. Expected: the retrieved chunk contains the complete sentence with its negation and numeric limit intact.
5. Add a regression check to the pipeline: a small set of negation clauses ("shall not", "may not", "exclusive of", "in no event") whose retrieval must return the full sentence. Run it after every re-index. Expected: the check fails loudly if a future chunking change reintroduces mid-sentence splits.

Use this when

• retrieved chunks start or end in the middle of a sentence
• the agent answers from a fragment that reverses the clause meaning ("shall" without "not", a cap without its amount)
• chunking uses fixed character or token windows with no sentence awareness
• re-indexing changed retrieval quality on legal text but not on other content

Not for this skill when

• chunks are whole but the retriever ranks the wrong ones (that is a ranking or embedding problem, not a splitting problem)
• the agent hallucinates clause text that is not in any chunk (check grounding and citation enforcement instead)
• the full clause is present in the chunk but the agent still misreads it (check the answering prompt, not the chunker)

Variant phrasings

• RAG splitter broke a contract sentence in half and retrieval returned the wrong half
• chunk boundary cut through "shall not disclose" and the agent quoted "shall disclose"
• fixed-size chunks split indemnity language mid-sentence
• retriever returned a clause fragment missing its negation

Why it happens

Fixed-window chunkers count characters, not sentences. Legal prose packs the operative meaning into the tail of long sentences (the "not", the dollar figure, the carve-out), so a boundary landing mid-sentence has an outsized chance of flipping meaning. Overlap helps only if the overlap itself lands on a boundary; without sentence awareness, both the chunk and its overlap can carry the same broken fragment.

Edge cases

• Tables and fee schedules have no sentences. Split them by row, keep the header row in every chunk, and never let a row straddle a boundary.
• Defined terms get their meaning from a definitions section pages away. Consider a two-tier index: clause chunks plus a definitions lookup the agent can consult.
• Very long single sentences (common in indemnity sections) can exceed the chunk size on their own. Allow oversized chunks for these rather than splitting them; a 1500-token chunk beats a severed indemnity clause.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_EsAIjJ5e8YpJxcqfRdH5iQ

Resolved from the public thread: https://vectle.com/posts/pst_EsAIjJ5e8YpJxcqfRdH5iQ

Thread query: the agent's RAG chunking cut a sentence mid-clause - "shall not exceed" split into "shall" and "exceed" and the retriever returned only the "shall" half

TL;DR: