Evidently: one empty column crashes the whole drift report

Evidently: one empty column crashes the whole drift report

TL;DR

Before running an Evidently drift report on a DataFrame with sparse or new columns, filter out the empty ones or pass an explicit column list to the preset. Otherwise a single all-null column kills the whole run, including the quality checks that would have told you the column was empty. Context: From GitHub issue evidentlyai/evidently#1480, answered by a maintainer with the docs and a workaround.

Use this when you run into the situation in the title.

When not to use this skill: unrelated tasks. It covers only the procedure above.

Compatibility

The steps above apply to the commands named in them. This skill does not pin a version, so if a flag looks different on your machine, check your installed version's docs first.

Details

If any column in the reference or current DataFrame is empty, the whole Report crashes with a ValueError about an empty column being provided for drift calculation, instead of warning and continuing with the other columns. The Data Quality preset, which could happily report the column as empty, goes down with it. The maintainer confirmed this is expected behavior per the data requirements: columns passed for drift detection must be non-empty. The workaround is to pass an explicit list of columns to the preset so only those columns are tested for drift.

Source: From GitHub issue evidentlyai/evidently#1480, answered by a maintainer with the docs and a workaround. If any column in the reference or current DataFrame is empty, the whole Report crashes with a ValueError about an empty column being provided for drift calculation, instead of warning and continuing with the other columns. The Data Quality preset, which could happily report the column as empty, goes down with it. The maintainer confirmed this is expected behavior per the data requirements: columns passed for drift detection must be non-empty. The workaround is to pass an explicit list of columns to the preset so only those columns are tested for drift.

Verified resolution: Before running an Evidently drift report on a DataFrame with sparse or new columns, filter out the empty ones or pass an explicit column list to the preset. Otherwise a single all-null column kills the whole run, including the quality checks that would have told you the column was empty.

Source URL: https://github.com/evidentlyai/evidently/issues/1480