## TL;DR

The model times out because it is called once per expense with no batching and no deadline. Batch expenses into single inference calls, set a per-call timeout, and fall back to merchant rules for anything the model does not answer in time.

## Error

```text
agent failed to categorize expenses: ml model timed out
```

## Steps

1. Batch the inference: send expenses in groups (for example, 50 per call) instead of one call each. Expected: far fewer round trips and better throughput.
2. Set an explicit per-call timeout (for example, 30 seconds) so a hung call fails fast. Expected: timeouts become quick, catchable errors.
3. On timeout, retry once, then fall back to the merchant-rule categorizer for that batch. Expected: every expense gets a category, model or rules.
4. Log which expenses used the fallback. Expected: a list you can re-run through the model later for quality.
5. If timeouts persist, check model serving capacity: queue depth, cold starts, and instance size. Expected: you find whether it is a capacity or a code problem.

## When to use

- An ML categorizer times out mid-run.
- The model works on small inputs but not on audit-scale volumes.

## When not to use

- The model returns wrong categories quickly (a training or feature problem).
- Categorization is rule-based with no ML involved (see the merchant-rule skill).

## Tool compatibility

- Custom ML categorization services; merchant-rule fallbacks in the agent.

## Variant phrasings

### Categorization model timed out

Same failure, shorter wording.

### Inference deadline exceeded on expenses

The serving-infrastructure phrasing.

## Why it happens

Per-expense inference calls multiply latency and connection overhead across thousands of expenses, and a model with no deadline can hang one call forever. Batching plus timeouts bounds the worst case.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_fUh3PgYrrQ5OjUl9wDMcHw