agent failed to categorize expenses: ml model timed out with error

TL;DR

The model times out because it is called once per expense with no batching and no deadline. Batch expenses into single inference calls, set a per-call timeout, and fall back to merchant rules for anything the model does not answer in time.

Error

agent failed to categorize expenses: ml model timed out

Steps

1. Batch the inference: send expenses in groups (for example, 50 per call) instead of one call each. Expected: far fewer round trips and better throughput.
2. Set an explicit per-call timeout (for example, 30 seconds) so a hung call fails fast. Expected: timeouts become quick, catchable errors.
3. On timeout, retry once, then fall back to the merchant-rule categorizer for that batch. Expected: every expense gets a category, model or rules.
4. Log which expenses used the fallback. Expected: a list you can re-run through the model later for quality.
5. If timeouts persist, check model serving capacity: queue depth, cold starts, and instance size. Expected: you find whether it is a capacity or a code problem.

When to use

• An ML categorizer times out mid-run.
• The model works on small inputs but not on audit-scale volumes.

When not to use

• The model returns wrong categories quickly (a training or feature problem).
• Categorization is rule-based with no ML involved (see the merchant-rule skill).

Tool compatibility

• Custom ML categorization services; merchant-rule fallbacks in the agent.

Variant phrasings

Categorization model timed out

Same failure, shorter wording.

Inference deadline exceeded on expenses

The serving-infrastructure phrasing.

Why it happens

Per-expense inference calls multiply latency and connection overhead across thousands of expenses, and a model with no deadline can hang one call forever. Batching plus timeouts bounds the worst case.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_fUh3PgYrrQ5OjUl9wDMcHw