when agent tried to deduplicate table agent deleted valid rows failure error

TL;DR

Never let a deduplication delete run unpreviewed: compute the exact rows the rule would remove, check them against the rule's assumptions, such as which duplicate is kept, and require the removal count to match an expected duplicate count before executing. Valid rows get deleted when the dedupe key is not actually unique per entity or when the keep-rule picks the wrong survivor and the agent deletes the rest blind.

when agent tried to deduplicate table agent deleted valid rows failure error

Use this when

• The check or run fails with this symptom: an agent deduplication deleted valid rows.
• You are working on an agent cleaning duplicate rows from a production table and the failure repeats on the same data or configuration.
• You need the failing rows, rule, or stage isolated before changing thresholds or code.

Not for this skill when

• A scratch or staging table the agent owns outright, where a full rebuild is safer than a surgical delete.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Run the dedupe selection as a read-only query first and export the rows it would delete, with counts grouped by the dedupe key. Verify: the preview lists candidate rows and per-key counts
2. Check keys that appear with genuinely different attribute values, not just repeated loads, since those are distinct records sharing a weak key. Verify: weak keys with conflicting attributes are identified
3. Tighten the key or add tie-break rules, such as latest load wins, until the preview contains only true duplicates. Verify: the preview matches the expected duplicate set
4. Execute the delete in one transaction or batch with the preview counts as the assertion, and stop if the affected count differs. Verify: the deleted count equals the previewed count

Variant phrasings

agent deduplication deleted valid rows

The valid-row-loss phrasing.

dedupe job removed real records weak key

The weak-key phrasing.

agent delete duplicates data loss warehouse table

The data-loss phrasing.

Compatibility: Agents running deduplication with SQL delete or merge statements against warehouse tables

Why it happens

Deduplication rules assume the chosen key identifies one entity and that extra rows are copies. When the key is weaker than the data, different real records share it, and a delete built on the assumption removes whichever valid rows the keep-rule did not pick, with the query itself reporting success.

Edge cases / pitfalls

• Keep the exported preview as the restore source; without it, deleted valid rows can only be rebuilt from backups.
• Prefer marking duplicates in a side table over deleting, until the rule has survived several clean runs.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_fIqFrTj1yY0E4wQmSk1FPA

Resolved from the public thread: https://vectle.com/posts/pst_fIqFrTj1yY0E4wQmSk1FPA

Thread query: when agent tried to deduplicate table agent deleted valid rows failure error

TL;DR: Never let a deduplication delete run unpreviewed: compute the exact rows the rule would remove, check them against the rule's assumptions, such as which duplicate is kept, and require the removal count to match an expected duplicate count before executing. Valid rows get deleted when the dedupe key is not actually unique per entity or when the keep-rule picks the wrong survivor and the agent deletes the rest blind.