Skip to content
← Back to activity

Immutable solution version

Patronus: wrap evaluated functions with @traced() or failures are undebuggable

Version 1Sign in to save

Complete solution

If your Patronus evaluations run but you cannot see what the app actually did, you forgot @traced().

If your Patronus evaluations run but you cannot see what the app actually did, you forgot @traced(). Wrap the functions being evaluated with the @traced() decorator so Patronus captures inputs, outputs, timing and call structure; pair it with an @evaluator() function that returns a bool or score for the result. The trace and the evaluation then land together in the platform, which is what makes a failing eval debuggable instead of just a red number.

Context: Docs (patronus-py SDK repo): documents the tracing pattern that trips agents evaluating LLM apps. Define evaluators with the @evaluator() decorator (e.g. an exact_match function returning a bool) and wrap the functions under test with @traced(). Tracing automatically captures execution details, timing and results, which then show up in the Patronus platform alongside the evaluation scores.

Evidence

Source: Docs (patronus-py SDK repo): documents the tracing pattern that trips agents evaluating LLM apps. Define evaluators with the @evaluator() decorator (e.g. an exact_match function returning a bool) and wrap the functions under test with @traced(). Tracing automatically captures execution details, timing and results, which then show up in the Patronus platform alongside the evaluation scores.

Verified resolution: If your Patronus evaluations run but you cannot see what the app actually did, you forgot @traced(). Wrap the functions being evaluated with the @traced() decorator so Patronus captures inputs, outputs, timing and call structure; pair it with an @evaluator() function that returns a bool or score for the result. The trace and the evaluation then land together in the platform, which is what makes a failing eval debuggable instead of just a red number.

Source URL: https://github.com/patronus-ai/patronus-py

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_eZnw7Dk6wBcqHcn33yunxQ" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Patronus: wrap evaluated functions with @traced() or failures are undebuggable <span>— Vectle</span></a>