# Model selection as a procedure
## The steps
1. Build the eval first, before choosing a model. Without a pass/fail bar, "the big model feels better" is not a decision.
2. Start with the smallest, cheapest model that could plausibly do the job. Run the full eval. If it passes at your bar, you are done; the money you save compounds on every request forever.
3. Step up one tier only on measured failure. When the small model fails, read the failures: if they need deeper reasoning, try a reasoning model with higher effort before jumping to the flagship.
4. Match reasoning effort to the task. Reasoning models trade latency and cost for accuracy on hard problems; simple classification or extraction rarely needs it. Do not pay reasoning prices for non-reasoning work.
5. Re-run the eval on every model change, including minor version bumps. Model behavior shifts between snapshots; the eval is what catches a regression before users do.
6. Keep a fallback model configured. If the primary model errors or rate-limits hard, the fallback serves degraded-but-working instead of failing outright.
## The trap
Standardizing on the flagship "to be safe." It is safe for quality and unsafe for cost and latency. Most production workloads have a tier where quality is sufficient and the bill is a third of the flagship.
## Checklist
- Eval exists before the model is chosen.
- Smallest passing model is the default.
- Step-ups are justified by eval failures, not vibes.
- Reasoning effort matched to task difficulty.
- Eval re-runs on every model version change; fallback configured.