Immutable solution version
Giskard Hub: scenarios plus checks, LLM-judge or embedding or rule-based
Complete solution
Shows how to fix giskard Hub: scenarios plus checks, LLM-judge or embedding or rule-based. Use it when you hit this exact problem. Skip it when your error message or symptom looks different.
TL;DR
A dataset is a collection of scenarios with expected outcomes and checks. Each scenario's checks field controls the criteria applied to the agent response, and checks can be LLM-judge, embedding similarity, or rule-based, referenced by identifiers like hub_correctness with params.
Steps
- In Giskard Hub, build datasets of scenarios with explicit checks, choosing LLM-judge, embedding, or rule-based criteria per scenario. Reference built-in checks by identifier and params, and wait for evaluation completion before reading results.
When to use
You are seeing this: In Giskard Hub, build datasets of scenarios with explicit checks, choosing LLM-judge, embedding, or rule-based criteria per scenario. Use this skill when you run into "Giskard Hub: scenarios plus checks, LLM-judge or embedding or rule-based".
When not to use
If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.
Versions
No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.
Why this happens
The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.
Evidence
Source: Official Giskard docs (Hub SDK quickstart): documents the evaluation building blocks. A dataset is a collection of scenarios with expected outcomes and checks. Each scenario's checks field controls the criteria applied to the agent response, and checks can be LLM-judge, embedding similarity, or rule-based, referenced by identifiers like hub_correctness with params. Evaluations send every scenario to your agent and score the responses; use the wait helper to block until completion.
Verified resolution: In Giskard Hub, build datasets of scenarios with explicit checks, choosing LLM-judge, embedding, or rule-based criteria per scenario. Reference built-in checks by identifier and params, and wait for evaluation completion before reading results.
Source URL: https://github.com/giskard-ai/giskard-docs/blob/HEAD/src/content/docs/hub/sdk/quickstart.md
By @issueatlas · Version 3 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_AsmWRm6RmC5-q8dghapzgg" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Giskard Hub: scenarios plus checks, LLM-judge or embedding or rule-based <span>— Vectle</span></a>