KEY ERROR 1 while using classification preset in Evidently

TL;DR

Upgrade Evidently to 0.4.1 or newer with pip install -U evidently. A bare KeyError: 1 from inside the classification preset is caused by mlflow's typing annotations poisoning typing.List's cache, which breaks Evidently's typed code paths. The fix landed in 0.4.1; downgrading does not help.

The error

KeyError: 1

Raised from inside Evidently's classification preset (e.g. running a classification report on a Spark/Databricks notebook). The traceback points into Evidently internals, not your code, and it appears suddenly with no data changes on your side.

Fix it

1. Check your installed Evidently version:

   pip show evidently | grep Version

   Success check: the version is below 0.4.1, confirming you are affected.

2. Upgrade Evidently:

   pip install -U evidently

   Success check: the installed version is 0.4.1 or newer.

3. Re-run the classification report. Success check: the report completes with no KeyError.

4. Do not downgrade as a workaround: the reporter confirmed downgrading to 0.3.3 did not help. Success check: not applicable; skip downgrading entirely.

When this applies

• Evidently classification preset raises a bare KeyError: 1 from library internals.
• The same report worked recently with no data changes.
• mlflow is installed in the same environment (the confirmed trigger).

When it does not apply

• The KeyError names one of your columns (e.g. KeyError: 'prediction'); that is a data/schema mismatch, not this bug.
• You are not using the classification preset, or Evidently is already at 0.4.1+.
• mlflow is not installed and the traceback points at your own code.

Compatibility

• Evidently: affected below 0.4.1, fixed in 0.4.1 and newer (via PR #712).
• Trigger requires mlflow installed in the same Python environment.
• Spark/Databricks notebooks and plain Python both affected.

Variant phrasings

• KeyError 1 evidently classification preset
• evidently classification report KeyError
• evidently KeyError 1 mlflow

Why it happens

mlflow uses typing annotations that get cached incorrectly by typing.List's internal cache. Evidently's classification preset relies on those typed code paths, so the poisoned cache makes a lookup fail with a bare KeyError: 1. The maintainers fixed the interaction in evidently 0.4.1. Source: https://github.com/evidentlyai/evidently/issues/701

Edge cases

• If upgrading is blocked by dependency pins, isolate Evidently and mlflow in separate environments as a temporary measure.
• The error can appear intermittently depending on import order; a fresh kernel or container that imports mlflow first is more likely to trigger it.

Asked in the thread (https://vectle.com/threads/thr_y0KPEJmDaONyiaL1WENHw): Ran an Evidently classification report on a Spark/Databricks notebook and got a bare KeyError: 1 from inside the classification preset. Worked fine the week before, no data changes on my side, and the traceback pointed into Evidently's internals rather than my code. Downgrading to evidently 0.3.3 didnt help. The maintainers traced it to an interaction with other libraries in the environment: mlflow uses typing annotations, and a wrong annotation got cached by typing.List's @lrucache, which broke Evidently's typed code paths. Verified resolution: Fixed in evidently 0.4.1 (via PR #712). Upgrade with pip install -U evidently to 0.4.1 or newer. The root cause was mlflow's typing annotations poisoning typing.List's @lru_cache, which then broke Evidently's internals with a cryptic KeyError: 1. The reporter confirmed the upgrade fixed it and asked for the issue to be closed.

Source: https://github.com/evidentlyai/evidently/issues/701