agent failed to compare evidently drift reports across two dates after report parse error

TL;DR

Align the two reports before comparing them: the agent failed to compare Evidently drift reports across dates after a parse error, so extract both reports through the same parser version, normalise metric names, and only then diff the drift results. Cross-date comparison fails when the two reports were produced or parsed differently. A report format change between the dates is the usual hidden difference.

agent failed to compare evidently drift reports across two dates after report parse error

Use this when

• The check or run fails with this symptom: an agent fails to compare Evidently drift reports across two dates after a report parse error.
• You are working on an agent workflow tracking Evidently drift over time for a model or a table and the failure repeats on the same data or configuration.
• You need the failing rows, rule, or stage isolated before changing thresholds or code.

Not for this skill when

• Reports from the same day and version that still fail to parse, which points at a damaged report file.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Record the Evidently version that produced each report. Verify: the versions match, or the difference is named
2. Parse both reports with one extractor and export both to the same flat metric table. Verify: both dates produce rows in the identical shape
3. Normalise metric and column names across the two exports. Verify: the same drift metric has one name on both dates
4. Diff the flat tables by column and metric. Verify: drift changes between the dates are listed
5. Save the flat export with each report. Verify: the next comparison starts from parsed data, not from the reports

Variant phrasings

agent failed compare evidently drift reports two dates parse

The shorter phrasing focused on the named failure.

evidently drift report comparison parse error

The symptom phrasing a searcher tries next.

compare drift reports across dates fix

The variant naming the failing stage or tool directly.

Compatibility: Evidently report versions and JSON exports, agent comparison workflows

Why it happens

Evidently report internals evolve between versions, so a parser fitted to the newer report fails or misreads the older one. Comparison code that assumes one format across a version boundary breaks even though both reports are individually valid.

Edge cases / pitfalls

• Store the parsed metrics at report time; future format changes then never reach historical comparisons.
• A missing column in one date is a finding about the feature set, not a parse failure; keep absent and unparsable distinct in the output.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstqUHIdT80Sf9RQc34GUWNA