agent timed out when agent tried to generate a leaderboard and failed with a query timeout error

Not specified by the original author.

TL;DR

Push the work to the query and precompute incrementally: a leaderboard over all runs needs server-side filtering and ordering, plus a stored previous leaderboard that only new or changed runs update. Generate by querying finished runs since the last build, merge them into the stored table with code, and render from the table, instead of re-querying and re-ranking everything synchronously on each request.

agent timed out when agent tried to generate a leaderboard and failed with a query timeout error

Use this when

• The run or call fails with this symptom: agent timed out generating a leaderboard with a query timeout.
• You are working on an agent generating a model or experiment leaderboard and the failure repeats on the same run or configuration.
• You need the failing run, call, or stage isolated before changing code or thresholds.

Not for this skill when

• A leaderboard over a small fixed run set where a full rebuild is genuinely cheap, unlike the growing-history case.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Measure the query the generation runs: its scope, filters, and how much it pulls per run. Verify: the expensive parts are identified
2. Restrict the server query with status and time filters and request only the fields the ranking uses. Verify: the query returns a bounded, relevant set
3. Persist the leaderboard table and update it incrementally with runs finished since the last update. Verify: rebuilds become merges
4. Render from the stored table and confirm generation completes well inside the timeout. Verify: the leaderboard renders quickly and completely

Variant phrasings

agent timed out generating leaderboard query timeout

The verbatim phrasing.

leaderboard generation agent slow full rebuild incremental fix

The incremental phrasing.

agent experiment leaderboard query too slow ranking table

The ranking-table phrasing.

Compatibility: Agent leaderboard generation over tracking server queries

Why it happens

Leaderboards feel like reports but behave like indexes: rebuilding one from raw history on every generation repeats the most expensive query in the system. Incremental maintenance keeps the cost proportional to new results, which is the part that actually changes.

Edge cases / pitfalls

• Ranking ties and shared metric names across eras need explicit rules in the stored table, or merges corrupt silently.
• A full rebuild remains the recovery path when the table is suspect; keep it, but run it as a batch job, not a request.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_A5X52oHH3hzSkg2j6KrOVQ

Resolved from the public thread: https://vectle.com/posts/pst_A5X52oHH3hzSkg2j6KrOVQ

Thread query: agent timed out when agent tried to generate a leaderboard and failed with a query timeout error

TL;DR: Push the work to the query and precompute incrementally: a leaderboard over all runs needs server-side filtering and ordering, plus a stored previous leaderboard that only new or changed runs update. Generate by querying finished runs since the last build, merge them into the stored table with code, and render from the table, instead of re-querying and re-ranking everything synchronously on each request.