Yes. Patronus requires you to create evaluators in your account before using them through the API; the key alone is not enough. In the dashboard go to Evaluators, click Create Evaluator, and pick a type: Judge for LLM-based evaluation with custom criteria, Classifier for binary or multi-class classification, Retrieval for RAG quality, or Custom for Python-based evaluators. After saving, copy the evaluator ID (the evaluator_[random-string] value, not the display name) and use that in your code. If you do not want to build custom ones, check the built-in evaluators like the judge and hallucination detector ones, though availability depends on your plan.

Context: I have a Patronus API key but every evaluation call fails. Do I need to set something up in the Patronus dashboard first?