during run evaluation of a test case inferences could be NoneType which has no len attribute and error is

Not specified by the original author.

TL;DR

Upgrade ADK past the release containing that fix. Reported in Google Agent Development Kit: during run evaluation of a test case inferences could be NoneType which has no len attribute and error is not caught.

Steps

1. ADK eval crashes with len() on NoneType when a test case's inference failed: in localevalservice.py the code calls len(inferenceresult.inferences) without a None check. When inference fails (session service errors, network failures), InferenceResult has status=FAILURE and errormessage set but inferences stays None. The fix (PR #3805): in evaluate(), filter out failed inferences first, i.e. if inferenceresult.status == InferenceStatus.FAILURE or inferenceresult.inferences is None, log a warning and return EvalStatus.NOT_EVALUATED instead of evaluating. Upgrade ADK past the release containing that fix.

When to use

You are seeing this: ADK eval crashes with len() on NoneType when a test case's inference failed: in localevalservice.py the code calls len(inference_result.inferences) without a None check. Use this skill when you run into "during run evaluation of a test case inferences could be NoneType which has no len attribute and error is".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

In gh:google/adk-python#2729 (https://github.com/google/adk-python/issues/2729), the reporter ran into this: Reported in Google Agent Development Kit: during run evaluation of a test case inferences could be NoneType which has no len attribute and error is not caught. The verified resolution: ADK eval crashes with len() on NoneType when a test case's inference failed: in localevalservice.py the code calls len(inferenceresult.inferences) without a None check. When inference fails (session service errors, network failures), InferenceResult has status=FAILURE and errormessage set but inferences stays None. The fix (PR #3805): in evaluate(), filter out failed inferences first, i.e. if inferenceresult.status == InferenceStatus.FAILURE or inferenceresult.inferences is None, log a warning and return EvalStatus.NOT_EVALUATED instead of evaluating. Upgrade ADK past the release containing that fix.