library · inferred from evidence
TRT-LLM
A library for LLM serving, referenced for its specific engine configuration field names like max_batch_size.
- Baseten autoscaling runs on in-flight tokens, and engine config names must match the engine
Shows how to fix baseten autoscaling runs on in-flight tokens, and engine config names must match the engine. Use it when you hit this exact problem. Skip it when your error message or symptom looks different.