churn-monitor agent timed out when querying the usage warehouse mid-scoring

TL;DR

The agent asks the warehouse one unbounded question (all usage, all accounts, full history) and the warehouse answers slowly or not at all. Split the work: score per account or per small batch, push aggregation into the warehouse query, set a statement timeout with retry, and checkpoint each finished account so a timeout loses minutes, not the run.

churn-monitor agent timed out when querying the usage warehouse mid-scoring

Use this when

• The scoring agent dies partway with warehouse timeout errors
• Larger or older accounts trigger the timeout most
• The agent holds all results in memory until the end

Not for this skill when

• The warehouse itself is down or overloaded by all tenants
• The timeout is in the CRM write step, not the warehouse read

Steps

1. Capture the exact query the agent sends and its runtime and row count in the warehouse history. Verify: the unbounded query is identified with its cost
2. Rewrite it to aggregate in the warehouse and return one row per account for a bounded window. Verify: the new query returns in seconds with a small result set
3. Process accounts in batches with a checkpoint write after each batch. Verify: a mid-run timeout preserves completed batches
4. Set a per-query timeout with bounded retries and jitter, and log slow queries separately. Verify: timeouts become handled events, not run-killers
5. Re-run the full scoring and compare wall time and completion against the old run. Verify: the run completes end to end with checkpoints to prove it

Variant phrasings

churn agent warehouse query timeout

A related phrasing for the same underlying problem.

scoring run dies querying usage

A related phrasing for the same underlying problem.

unbounded warehouse query in agent

A related phrasing for the same underlying problem.

Compatibility: Compatibility: agent frameworks (LangChain-style tools, custom Python agents) against Snowflake, BigQuery, or Redshift warehouses, current versions.

Why it happens

Agents compose queries optimistically and warehouses punish optimism at scale. The pattern that survives is database-first aggregation, small batches, and durable checkpoints, the same discipline as any long ETL job, because a scoring run is an ETL job with an LLM attached.

Edge cases / pitfalls

• Pushing aggregation into SQL changes semantics if the agent previously filtered in Python. Diff scores on 20 accounts before and after.
• Checkpoint writes must include the scoring config version, or resumed runs mix two models.
• A statement timeout without a retry cap turns the timeout into a loop. Bound the retries.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_88YdyCBh2qvbeqn8YbSGrA