Giskard: identical descriptions when comparing models, all columns in feature_names

TL;DR

When scanning multiple models with Giskard, pass the exact same description to each so the test cases are comparable. Include every dataset column in feature_names.

Steps

1. When scanning multiple models with Giskard, pass the exact same description to each so the test cases are comparable. Include every dataset column in feature_names. Invest in the description text; it is what makes the generated attacks relevant to your use case.

When to use

You are seeing this: And when comparing multiple models, the model description must be identical for each; otherwise the scan generates different test cases per model and the comparison is unfair. Use this skill when you run into "Giskard: identical descriptions when comparing models, all columns in feature_names".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Source: Official Giskard knowledge article (choosing an LLM for a chatbot): documents two comparison gotchas. Your feature_names list must include all columns in the dataset, or the scan cannot use them. And when comparing multiple models, the model description must be identical for each; otherwise the scan generates different test cases per model and the comparison is unfair. Make the description as informative as possible, since Giskard uses it to invent relevant attacks, like inducing the bot to reveal phone numbers or violate terms of service.

Verified resolution: When scanning multiple models with Giskard, pass the exact same description to each so the test cases are comparable. Include every dataset column in feature_names. Invest in the description text; it is what makes the generated attacks relevant to your use case.

Source URL: https://www.giskard.ai/knowledge/find-the-best-llm-for-your-customer-service-chatbot