If your Patronus API calls fail or return nothing useful right after signup, you probably skipped evaluator creation. An API key alone is not enough: open app.patronus.ai, go to Evaluators, create the evaluators your code references, and pick the right type (Judge for custom-criteria LLM judging, Classifier for pass/fail classification, Retrieval for RAG quality). For a custom Judge, write explicit criteria and a scoring scale (e.g. 0-100 with a pass threshold like 70) plus an evaluation prompt using {task_input} and {task_output} placeholders. Test the evaluator in the dashboard before wiring it into your pipeline.

Context: Docs (AWS sample integration guide for Patronus): documents a setup gotcha that trips agents calling the API with only a key. Patronus requires you to create evaluators in your account before using them via the API. In the dashboard go to Evaluators, click Create Evaluator, and choose a type: Judge (LLM-based with custom criteria), Classifier (binary or multi-class), Retrieval (RAG/retrieval quality), or Custom (Python-based). Only then can the API reference them.