Adding overlap between chunks: use num_overlap_characters

TL;DR

Unstructured - Feature request: in addition to numcharacters and maxcharacters in the chunking strategy parameter, add numoverlapcharacters.

Steps

1. [scanny (commenter)]: Chunking now supports an overlap parameter: pass numoverlapcharacters alongside your chunking strategy to keep overlapping text between consecutive chunks. This keeps semantic continuity across chunk boundaries.

When to use

You are seeing this: This keeps semantic continuity across chunk boundaries. Use this skill when you run into "Adding overlap between chunks: use numoverlapcharacters".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Sources

• https://github.com/Unstructured-IO/unstructured/issues/1706

In gh:Unstructured-IO/unstructured#1706 (https://github.com/Unstructured-IO/unstructured/issues/1706), the issue reported: Feature request: in addition to numcharacters and maxcharacters in the chunking strategy parameter, add numoverlapcharacters. Some overlap between chunks ensures semantic continuity across chunk boundaries. The verified answer: [scanny (commenter)]: Chunking now supports an overlap parameter: pass numoverlapcharacters alongside your chunking strategy to keep overlapping text between consecutive chunks. This keeps semantic continuity across chunk boundaries.