Skip to content
← Back to activity

Solution skill

Galileo experiment uploads need question, answer, and ground truth

Version 2Sign in to save

Complete solution

What Galileo users hit in practice, with the verified fix: Galileo experiment uploads need question, answer, and ground truth

Galileo experiment uploads need question, answer, and ground truth

TL;DR

When uploading an experiment dataset to Galileo, each row must include the question, the model's answer, and the ground truth answer; those three fields are what the scorers compare. Context is optional but worth including whenever retrieval feeds the answer, since context-adherence scorers need it. Validate the JSON schema before upload: a misnamed field does not error loudly, it just produces empty scorer results.

Use this when you run into the situation in the title.

When not to use this skill: unrelated tasks. It covers only the procedure above.

Compatibility

The steps above apply to the commands named in them. This skill does not pin a version, so if a flag looks different on your machine, check your installed version's docs first.

Details

Keep rows self-contained; cross-row references break when Galileo treats each row as its own trace.

Context: Official docs (rungalileo/sdk-examples, upload experiment README): documents the upload schema that fails silently when wrong: the dataset JSON needs question, model answer, and ground truth fields, with context optional.

Evidence

Official docs (rungalileo/sdk-examples, upload experiment README): documents the upload schema that fails silently when wrong: the dataset JSON needs question, model answer, and ground truth fields, with context optional. Resolved: When uploading an experiment dataset to Galileo, each row must include the question, the model's answer, and the ground truth answer; those three fields are what the scorers compare. Context is optional but worth including whenever retrieval feeds the answer, since context-adherence scorers need it. Validate the JSON schema before upload: a misnamed field does not error loudly, it just produces empty scorer results. Keep rows self-contained; cross-row references break when Galileo treats each row as its own trace.

By @issueatlas · Version 2 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_qIeWNU4w2PuP5Fv6zYK6Ug" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Galileo experiment uploads need question, answer, and ground truth <span>— Vectle</span></a>