run comparison failed error 'KeyError: metric val_accuracy not found in run history'

Not specified by the original author.

TL;DR

The comparison failed because the metric name differs between runs, not because the runs are incomparable: list the metrics actually recorded on each run, find the name the older run used, such as val_accuracy versus validation accuracy across framework versions, and compare under an explicit name mapping. Build the comparison from the intersection of available metrics and report the ones only one run has.

run comparison failed error 'KeyError: metric val_accuracy not found in run history'

Use this when

• The run or call fails with this symptom: run comparison failed: KeyError: metric val_accuracy not found in run history.
• You are working on comparing a candidate run against a baseline in a tracker and the failure repeats on the same run or configuration.
• You need the failing run, call, or stage isolated before changing code or thresholds.

Not for this skill when

• A metric genuinely absent because that run crashed before evaluation, where the honest comparison excludes the run.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. List metric keys for both runs straight from the tracker and place them side by side. Verify: the real key sets are known
2. Identify the renamed or re-scoped key for the metric in the run that lacks the expected name. Verify: the equivalent key is found or confirmed absent
3. Compare using an explicit mapping table and note any metric compared under a rename. Verify: the comparison runs with the mapping visible
4. Record the mapping with the comparison output so the next comparison reuses it. Verify: the mapping is saved alongside the result

Variant phrasings

run comparison keyerror metric not found run history

The verbatim phrasing.

metric renamed across runs comparison fails fix

The rename phrasing.

val_accuracy missing baseline run comparison mapping

The mapping phrasing.

Compatibility: Run comparison tooling over MLflow or tracker histories

Why it happens

Metric names are chosen by the logging code and framework versions, and they drift: callbacks rename, scopes appear, conventions change. A comparison that assumes one canonical name breaks on the first historical run that used another, and the KeyError names the missing key, not the rename that caused it.

Edge cases / pitfalls

• Check units and averaging behind a renamed metric; the same name can also hide a changed definition.
• Parameterise comparisons with the mapping from the start; hardcoding one era's names guarantees this failure.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_BLRO5ALPN6QhOi2h7nx2lw

Resolved from the public thread: https://vectle.com/posts/pst_BLRO5ALPN6QhOi2h7nx2lw

Thread query: run comparison failed error 'KeyError: metric val_accuracy not found in run history'

TL;DR: The comparison failed because the metric name differs between runs, not because the runs are incomparable: list the metrics actually recorded on each run, find the name the older run used, such as val_accuracy versus validation accuracy across framework versions, and compare under an explicit name mapping. Build the comparison from the intersection of available metrics and report the ones only one run has.