agent hallucinating hyperparameters that were never logged and failing to cite the source run error

TL;DR

Attach a source run to every hyperparameter the agent states, and fetch instead of recalling: the invented values came from the model filling a table from plausibility, often blending values across runs. Build the report from a fetched table keyed by run id, render hyperparameters only from that table, and verify a random sample of cells back against the tracker before delivery.

agent hallucinating hyperparameters that were never logged and failing to cite the source run error

Use this when

• The run or call fails with this symptom: agent hallucinating hyperparameters that were never logged and failing to cite the source run.
• You are working on an agent summarising hyperparameters across experiment runs and the failure repeats on the same run or configuration.
• You need the failing run, call, or stage isolated before changing code or thresholds.

Not for this skill when

• Values that were logged as tags or notes rather than params, where the fix is looking in the right field, not treating the run as unlogged.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. For each hyperparameter in the draft, find the run id it is attributed to and fetch that run's recorded params. Verify: each value has a fetched counterpart or is flagged
2. List values with no counterpart and values that differ from the fetched record. Verify: invented and blended values are enumerated
3. Rebuild the table from the fetched records alone, leaving genuinely unlogged fields blank or marked not logged. Verify: the table contains only recorded values
4. Verify a sample of cells against fresh fetches and add run-id citations to the delivered report. Verify: the sample matches and citations resolve

Variant phrasings

agent hallucinated hyperparameters never logged source run

The verbatim phrasing.

agent invented hyperparameter values report citation fix

The citation phrasing.

agent hyperparameter table not matching tracker records

The mismatch phrasing.

Compatibility: Agent reporting over experiment trackers with citation requirements

Why it happens

Hyperparameter tables look like knowledge the model should have, so generation fills them fluently, mixing the run's real values with typical ones. The defence is structural: the report must be a rendering of fetched rows, with blanks where the record is blank.

Edge cases / pitfalls

• Blended values, the learning rate from run A with the batch size from run B, are subtler than pure invention; check cells jointly per run.
• Cite at the row level with run ids; a report-level note about sources does not make cells checkable.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_d9msZP77atUVYOEfYWbexQ

Resolved from the public thread: https://vectle.com/posts/pst_d9msZP77atUVYOEfYWbexQ

Thread query: agent hallucinating hyperparameters that were never logged and failing to cite the source run error

TL;DR: Attach a source run to every hyperparameter the agent states, and fetch instead of recalling: the invented values came from the model filling a table from plausibility, often blending values across runs. Build the report from a fetched table keyed by run id, render hyperparameters only from that table, and verify a random sample of cells back against the tracker before delivery.