grpcio 1.68 breaks concurrent pymilvus requests with failed to connect to all addresses

TL;DR

For "failed to connect to all addresses": maintainer XuanYang-cn traced it to grpc/grpc#39894 and confirmed it is fixed in grpcio 1.71.x, 1.72.x and 1.73.x. The fix on the pymilvus side: upgrade grpcio to 1.71 or newer, but below 2.0, in your requirements and the concurrent connection failures go away.

failed to connect to all addresses

Steps

1. What was reported:

Issue milvus-io/pymilvus#2358 (closed, 8 comments): After upgrading grpcio to 1.68.0, multi-threaded concurrent DQL requests fail with "failed to connect to all addresses". Rolling back to grpcio 1.60.0 makes the errors disappear. Reproducible under concurrent search load, so it looked like a pymilvus bug at first.

2. What works:

Upstream grpcio bug, not pymilvus. Maintainer XuanYang-cn traced it to grpc/grpc#39894 and confirmed it is fixed in grpcio 1.71.x, 1.72.x and 1.73.x. The fix on the pymilvus side: upgrade grpcio to 1.71 or newer, but below 2.0, in your requirements and the concurrent connection failures go away. Newer pymilvus releases (from 2.6.0rc159 on) also loosened their grpcio pin so you stop getting forced onto the broken 1.68 line. If you are stuck on an older pymilvus for other reasons, grpcio 1.60.0 is the last known-good fallback.

When to use

You are seeing this: What was reported: Issue milvus-io/pymilvus#2358 (closed, 8 comments): After upgrading grpcio to 1.68.0, multi-threaded concurrent DQL requests fail with "failed to connect to all addresses". Use this skill when you run into "grpcio 1.68 breaks concurrent pymilvus requests with failed to connect to all addresses".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

Versions mentioned in the source: 1.68.0, grpcio 1.60.0. If you are on something much newer or older, the details may have shifted.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

In the GitHub issue, the reporter asked: grpcio 1.68 breaks concurrent pymilvus requests with failed to connect to all addresses. The working answer that came out of the thread: Upstream grpcio bug, not pymilvus. Maintainer XuanYang-cn traced it to grpc/grpc#39894 and confirmed it is fixed in grpcio 1.71.x, 1.72.x and 1.73.x. The fix on the pymilvus side: upgrade grpcio to 1.71 or newer, but below 2.0, in your requirements and the concurrent connection failures go away. Newer pymilvus releases (from 2.6.0rc159 on) also loosened their grpcio pin so you stop getting forced onto the broken 1.68 line. If you are stuck on an older pymilvus for other reasons, grpcio 1.60.0 is the last known-good fallback. Source: https://github.com/milvus-io/pymilvus/issues/2358