agent cannot parse entitlement CSV with duplicate rows: parsing failed with error and the workflow broke

TL;DR

Fix the file handling before you rerun the agent, because the same entitlement CSV with duplicate rows will break the same parser every time. Start by isolating the exact records that fail, then deduplicate on the user, application and entitlement key, keeping conflicting duplicates aside for owner review instead of dropping them silently. Parse the cleaned input with a tolerant reader that quarantines bad records and continues, and reconcile the parsed totals against the source so a run that finishes is also a run that is complete.

agent cannot parse entitlement CSV with duplicate rows: parsing failed with error and the workflow broke

Use this when

• The agent run fails while reading the entitlement CSV with duplicate rows and never reaches the review work.
• Only some records fail, so a partial parse looks plausible but is incomplete.
• The same file is regenerated each cycle and breaks the run each cycle.

Not for this skill when

• The file parses and the problem is wrong values inside it. That is a data quality or validation task, not a parser task.
• The file never arrives because the export failed upstream. Fix the export first.

Steps

1. Preserve the original file untouched and run the parser in a diagnostic mode or on a copy, capturing the exact record or line where parsing fails and the parser message. Verify: the first failing record is identified in the original file.
2. Inspect the file structure around that record and check the header row, delimiter consistency, quoted fields containing commas and the encoding, since duplicate looking rows often follow a merged export from two sources. Verify: the structural cause of the failure is named in plain language.
3. Apply the targeted repair for this file type: deduplicate on the user, application and entitlement key, keeping conflicting duplicates aside for owner review instead of dropping them silently. Verify: a cleaned working copy exists and the original is still preserved.
4. Switch the agent parsing step to a tolerant reader that validates each record against the expected schema, writes failures to a quarantine list with reasons, and continues with the valid records. Verify: a test parse completes and produces both parsed records and a quarantine list.
5. Reconcile the counts: parsed plus quarantined must equal the source record total, and sample ten parsed records back against the source file by eye. Verify: counts reconcile and sampled records match the source.
6. Decide each quarantined record with a human reviewer, since some will be real access that still needs review through a manual path, and record those decisions in the review evidence. Verify: every quarantined record has a review outcome or a named owner.
7. Add the file checks that caught this failure to the start of the workflow, as a pre-flight validation step that reports problems before the main run begins. Verify: the workflow now validates this file type up front on its next run.

Why it happens

Review files come from exports, scans and hand maintained spreadsheets, and they carry every irregularity those origins produce. A strict parser built on clean test files treats the first irregular record as fatal, and an agent driving it retries the whole run against the same bytes. The repair pattern is consistent across file types: diagnose the real structure, repair or route around the bad records explicitly, and prove completeness with counts. Quarantine plus reconciliation is what separates a robust parse from a lucky one.

Variants and edge cases

agent cannot read entitlement CSV with duplicate rows in review workflow

The workflow phrasing. Run the parser diagnostically on a copy so the agent run log is not your only evidence.

how to handle bad records in entitlement CSV with duplicate rows without stopping the review

The resilience phrasing. Quarantine and continue, with a human deciding the quarantined records, is the safe default.

parser worked in testing but fails on production review file

The drift phrasing. Production files are larger and older; compare a production sample structurally against the test file.

Edge cases to check before closing the review cycle:

• Never assume a timed out call did nothing. Always check the Okta campaign item list before retrying a change.
• Two retries at the same settings prove nothing. Change one variable, size, timing or timeout value, per attempt and record it.
• Keep the run identifier on every output so a resumed run can prove it belongs to the same review cycle.

Compatibility: Agent workflow tooling with checkpointed tool calls, the named platform APIs at current versions, and a human reviewer able to verify results in the source system.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_ZBEY0Xxv6vsw82p918Ws6w