agent's redline missed a one-word change inside a 200-row pricing table - the chunker never sent the table to the...
Fixes chunkers dropping tables so cell-level changes never reach review. Use when a diff misses edits inside large tables. Covers extracting tables separately from prose, diffing cell by cell, and adding a table-coverage guard. Not for prose edits.
TL;DR
The chunker split the document into prose chunks and never sent the 200-row pricing table to the model, so a one-word change inside it was missed. Extract tables separately from the text flow, diff them cell by cell, and add a guard: every table row must be accounted for in the comparison.
The symptom
agent's redline missed a one-word change inside a 200-row pricing table - the chunker never sent the table to the modelSteps
1. Extract tables separately
Use a table parser (pdfplumber, Camelot, docx tables) to pull every table into row-lists, independent of the prose chunker. Expected: All tables captured as structured data, including the 200-row pricing table.
2. Diff tables cell by cell
Compare the normalized row-lists between versions. The one-word change in the pricing table now surfaces. Expected: The missed edit appears in the diff.
3. Merge table diffs into the redline
Present table changes alongside prose changes, labeled by table and row. Expected: One complete change list covering prose and tables.
4. Add the table-coverage guard
Before finishing, assert: every table found in either version appears in the diff input. Log table counts per version. Expected: No table can silently fall out of scope again.
When to use
- Changes inside tables are missed
- Documents with large pricing or schedule tables
- The pipeline chunks by paragraphs or tokens
When not to use
- The table genuinely did not change (verify with the cell diff, not the chunker)
- Edits are in prose (the chunker handles those)
- Tables are images (OCR them first, then extract)
Why it happens
Prose chunkers split on paragraphs, headings, or token limits. A 200-row table is either one giant chunk that gets truncated or a blob the chunker skips. Either way the model never sees the cells, and cell-level edits pass through review invisible.
Edge cases
- Nested tables: flatten or recurse; do not silently drop inner tables.
- Tables spanning page breaks: stitch before diffing.
- Number formatting ($1,000 vs $1000): decide once whether it counts as a change.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_QVY5TY4rjbA7oCOGqyMRaA