OpenSearch 3.x snapshot repository fails on S3-compatible storage: "trailing checksum is not supported"
Shows how to fix openSearch 3.x snapshot repository fails on S3-compatible storage: "trailing checksum is not supported". Use it when you hit this exact problem. Skip it when your error message or symptom looks different.
TL;DR
The repository-s3 plugin in 3.0.0 upgraded the AWS SDK to 2.30.31, and testing a repository failed with: "Caused by: software.amazon.awssdk.services.s3.model.S3Exception: trailing checksum is not supported (Service: S3, Status Code: 400...)". Users confirmed the bug persisted in 3.1.0 and 3.2.0, blocking upgrades.
When to use
You are seeing this: After upgrading to OpenSearch 3.0.0, snapshot repositories on S3-compatible storage (e.g. Use this skill when you run into "OpenSearch 3.x snapshot repository fails on S3-compatible storage: "trailing checksum is not supported"".
When not to use
If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.
Versions
Versions mentioned in the source: OpenSearch 3.0.0, 2.30.31, 3.1.0, 3.2.0, 3.3.2, SDK 2.29.52. If you are on something much newer or older, the details may have shifted.
What was going on
After upgrading to OpenSearch 3.0.0, snapshot repositories on S3-compatible storage (e.g. Outscale) stopped working. The repository-s3 plugin in 3.0.0 upgraded the AWS SDK to 2.30.31, and testing a repository failed with: "Caused by: software.amazon.awssdk.services.s3.model.S3Exception: trailing checksum is not supported (Service: S3, Status Code: 400 ...)". The documented workaround of disabling checksum calculation via env vars did not help. Users confirmed the bug persisted in 3.1.0 and 3.2.0, blocking upgrades.
The fix that worked
The 3.0.0 repository-s3 plugin's newer AWS SDK sends trailing checksums that S3-compatible stores reject. Verified workaround (confirmed by multiple users): set legacy MD5 checksum calculation per repository via the create-repository API, e.g. settings {"bucket": "bucketname", "basepath": "pathinbucket", "legacymd5checksumcalculation": "true"}. Permanent fix: maintainer cwperks raised PR #19788 to make this settable in opensearch.yml; it shipped in 3.3.2, where users confirmed s3.client.default.legacymd5checksumcalculation works without per-repo config. Resolution: upgrade to OpenSearch 3.3.2+ and set s3.client.default.legacymd5checksumcalculation: true, or apply the per-repository setting on older 3.x. (An earlier community workaround was rebuilding the repository-s3 plugin zip against AWS SDK 2.29.52.) Source: https://github.com/opensearch-project/OpenSearch/issues/18240
Where this came from
https://github.com/opensearch-project/OpenSearch/issues/18240
Maintainer review
No maintainer verification is recorded for this version.
This records the version a maintainer checked. It does not assert that the version is the latest upstream release.