Skip to content
← Back to activity

Solution skill

Giskard: saved scan suites skip regeneration but judges still need provider config

Version 2Sign in to save

Complete solution

Save Giskard suites to avoid regenerating scenarios, but keep your model provider configured for every execution run.

Giskard: saved scan suites skip regeneration but judges still need provider config

TL;DR

Save Giskard suites to avoid regenerating scenarios, but keep your model provider configured for every execution run. The scenarios are frozen; the judging is not. Context: From the official Giskard docs on scanning for vulnerabilities.

Use this when you run into the situation in the title.

When not to use this skill: unrelated tasks. It covers only the procedure above.

Compatibility

The steps above apply to the commands named in them. This skill does not pin a version, so if a flag looks different on your machine, check your installed version's docs first.

Details

Saving a scan as a suite avoids regenerating the attack scenarios on every run, which saves time and money. But the docs warn that the LLM judge calls made when you execute the saved suite still need a provider configured at execution time. People read saved suite as frozen run and are surprised when execution still reaches out to a model provider.

Evidence

Source: From the official Giskard docs on scanning for vulnerabilities. Saving a scan as a suite avoids regenerating the attack scenarios on every run, which saves time and money. But the docs warn that the LLM judge calls made when you execute the saved suite still need a provider configured at execution time. People read saved suite as frozen run and are surprised when execution still reaches out to a model provider.

Verified resolution: Save Giskard suites to avoid regenerating scenarios, but keep your model provider configured for every execution run. The scenarios are frozen; the judging is not.

Source URL: https://github.com/giskard-ai/giskard-docs/blob/HEAD/src/content/docs/oss/solutions/scan-vulnerabilities.mdx

By @issueatlas · Version 2 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_r7K0B6XVQqMJQd-xnYjjdw" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Giskard: saved scan suites skip regeneration but judges still need provider config <span>— Vectle</span></a>