GCS Node SDK: lots of socket hang up errors on batch requests

TL;DR

Upgrade @google-cloud/storage to a version containing the fix and retry the batch checks. Fixed upstream in the @google-cloud/storage SDK: the reporter re-tested with 7 files and all streams succeeded after the fix.

Steps

1. Fixed upstream in the @google-cloud/storage SDK: the reporter re-tested with 7 files and all streams succeeded after the fix. Upgrade @google-cloud/storage to a version containing the fix and retry the batch checks. If you still see socket hang ups on an old version, the practical mitigation is to serialize the checks or increase the HTTP agent's maxSockets.

When to use

You are seeing this: Batch file existence checks against GCS (bucket.file(fileName).exists() in batches of 20) produce lots of socket hang up errors from the @google-cloud/storage Node SDK. Use this skill when you run into "GCS Node SDK: lots of socket hang up errors on batch requests".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

A user asked: Batch file existence checks against GCS (bucket.file(fileName).exists() in batches of 20) produce lots of socket hang up errors from the @google-cloud/storage Node SDK. The SDK was not reusing connections properly, so bursts of requests exhausted sockets. The verified answer: Fixed upstream in the @google-cloud/storage SDK: the reporter re-tested with 7 files and all streams succeeded after the fix. Upgrade @google-cloud/storage to a version containing the fix and retry the batch checks. If you still see socket hang ups on an old version, the practical mitigation is to serialize the checks or increase the HTTP agent's maxSockets. Thread: https://vectle.com/threads/thrhj9-QOFP6YNZt-Wph7n2A