fake-gcs-server: invalid character '\x1f' looking for beginning of value from Java client

TL;DR

Upgrade fake-gcs-server past the 2021-02-01 fix. The Java SDK gzip-compresses request bodies; the fake server enabled compressed responses but never implemented decompression of gzipped request bodies, so it tried to parse the compressed bytes as JSON.

Steps

1. The Java client sends gzipped request bodies by default, and fake-gcs-server did not decompress them (compressed requests are non-standard and had to be implemented manually). The maintainer fixed the server to handle gzipped request payloads and verified the fix against the reporter's reproducer (both tests passing, with a regression test added). Upgrade fake-gcs-server past the 2021-02-01 fix. As a workaround on old versions, disable gzip in the client.

When to use

You are seeing this: Creating a bucket on fake-gcs-server using the Java Storage client failed with: invalid character '\x1f' looking for beginning of value, raised from HttpStorageRpc.translate. Use this skill when you run into "fake-gcs-server: invalid character '\x1f' looking for beginning of value from Java client".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

In the thread https://vectle.com/threads/thr_uCdcbsqATihQw62ghyGOcg, the question was: Creating a bucket on fake-gcs-server using the Java Storage client failed with: invalid character '\x1f' looking for beginning of value, raised from HttpStorageRpc.translate. Pointing the client at the fake server (custom StorageOptions with NoCredentials and a custom host) was fine, and creating the bucket via the startup flag worked, but SDK calls failed. Another user traced it: \x1f is the first byte of a gzipped payload. The Java SDK gzip-compresses request bodies; the fake server enabled compressed responses but never implemented decompression of gzipped request bodies, so it tried to parse the compressed bytes as JSON. The verified resolution was: The Java client sends gzipped request bodies by default, and fake-gcs-server did not decompress them (compressed requests are non-standard and had to be implemented manually). The maintainer fixed the server to handle gzipped request payloads and verified the fix against the reporter's reproducer (both tests passing, with a regression test added). Upgrade fake-gcs-server past the 2021-02-01 fix. As a workaround on old versions, disable gzip in the client.