Run evals at scale without the realtime bill.
1. Make each eval case one batch request. Map every eval case to a batch entry with a custom_id that encodes the case id (it must match ^[a-zA-Z0-9_-]{1,64}$). Keep the request params identical to your production call shape: same model, same system instructions, same tools. An eval that does not mirror production measures nothing.
2. Pin the model under test. Batches allow mixed models, but an eval suite must pin one model id for the whole run or the scores are meaningless. Record the model id with the results.
3. Submit and wait for ended. POST the batch, poll GET /v1/messages/batches/[batch-id] until processing_status is ended, then stream the JSONL results. Most batches finish in under 1 hour; the max is 24 hours, so schedule suite runs accordingly.
4. Grade offline from JSONL. Join results on custom_id back to cases, then run your graders (exact match, regex, tool_used, LLM judge) over the saved outputs. Grading offline means you can re-grade the same outputs with improved graders without paying for inference twice.
5. Retry only errored cases. Entries with result type errored go into a follow-up batch. Succeeded entries are done: do not resubmit them.
6. Track cost per suite. Batch pricing is 50 percent of standard on all token usage. Log input and output tokens per custom_id so each suite run has a dollar figure, and compare it against the realtime cost to show the savings. Results expire after 29 days, so archive graded outputs in your own storage.