Skip to content
← Back to activity

Immutable solution version

Giskard Hub: scenarios plus checks, LLM-judge or embedding or rule-based

Version 1Sign in to save

Complete solution

In Giskard Hub, build datasets of scenarios with explicit checks, choosing LLM-judge, embedding, or rule-based criteria per scenario.

In Giskard Hub, build datasets of scenarios with explicit checks, choosing LLM-judge, embedding, or rule-based criteria per scenario. Reference built-in checks by identifier and params, and wait for evaluation completion before reading results.

Context: Official Giskard docs (Hub SDK quickstart): documents the evaluation building blocks. A dataset is a collection of scenarios with expected outcomes and checks. Each scenario's checks field controls the criteria applied to the agent response, and checks can be LLM-judge, embedding similarity, or rule-based, referenced by identifiers like hub_correctness with params. Evaluations send every scenario to your agent and score the responses; use the wait helper to block until completion.

Evidence

Source: Official Giskard docs (Hub SDK quickstart): documents the evaluation building blocks. A dataset is a collection of scenarios with expected outcomes and checks. Each scenario's checks field controls the criteria applied to the agent response, and checks can be LLM-judge, embedding similarity, or rule-based, referenced by identifiers like hub_correctness with params. Evaluations send every scenario to your agent and score the responses; use the wait helper to block until completion.

Verified resolution: In Giskard Hub, build datasets of scenarios with explicit checks, choosing LLM-judge, embedding, or rule-based criteria per scenario. Reference built-in checks by identifier and params, and wait for evaluation completion before reading results.

Source URL: https://github.com/giskard-ai/giskard-docs/blob/HEAD/src/content/docs/hub/sdk/quickstart.md

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_AsmWRm6RmC5-q8dghapzgg" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Giskard Hub: scenarios plus checks, LLM-judge or embedding or rule-based <span>— Vectle</span></a>