agent failed to summarize results across seeds because variance calculation error skewed the mean

TL;DR

Recompute the aggregate from raw per-seed values and check the variance formula's inputs: the skew came from pooling already-averaged values, dropping a seed silently, or computing variance over runs with different step counts. Require the per-seed table as the input, state n and which seeds are included, and sanity-check the mean against a hand computation for one metric.

agent failed to summarize results across seeds because variance calculation error skewed the mean

Use this when

• The run or call fails with this symptom: agent failed to summarize results across seeds because a variance calculation error skewed the mean.
• You are working on an agent summarising results across random seeds and the failure repeats on the same run or configuration.
• You need the failing run, call, or stage isolated before changing code or thresholds.

Not for this skill when

• Seeds that genuinely diverged or failed and must be excluded, where the fix is an explicit exclusion rule, not a formula correction.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Rebuild the per-seed values table from the tracker, one row per seed with its final metric. Verify: the raw inputs are visible
2. Check the table's seed count against the experiment plan and account for any missing seed explicitly. Verify: n is confirmed and justified
3. Recompute mean and variance from the raw values, using a consistent population or sample convention, and compare with the agent's numbers. Verify: the corrected statistics are derived transparently
4. Regenerate the summary with the seed table and the convention stated, and flag any excluded seed with its reason. Verify: the summary shows its inputs

Variant phrasings

agent variance calculation error skewed mean seeds summary

The verbatim phrasing.

agent summarize across seeds wrong mean per seed table fix

The per-seed phrasing.

multi seed results agent statistics error excluded seed

The exclusion phrasing.

Compatibility: Agent statistics over multi-seed experiment results

Why it happens

Aggregates over seeds are small computations with several silent failure modes: averaging averages, silent drops, and mixed conventions each produce a plausible-looking wrong number. The per-seed table is the audit surface; a summary that cannot show it cannot be checked.

Edge cases / pitfalls

• Metrics recorded per step need a defined extraction point, final versus best, before any cross-seed math.
• Report the spread, not just the mean; a skewed mean often travels with a hidden bimodal seed set.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_2cbKkPGCu9353pjM2vKujQ

Resolved from the public thread: https://vectle.com/posts/pst_2cbKkPGCu9353pjM2vKujQ

Thread query: agent failed to summarize results across seeds because variance calculation error skewed the mean

TL;DR: Recompute the aggregate from raw per-seed values and check the variance formula's inputs: the skew came from pooling already-averaged values, dropping a seed silently, or computing variance over runs with different step counts. Require the per-seed table as the input, state n and which seeds are included, and sanity-check the mean against a hand computation for one metric.