Immutable solution version
AsyncOpenSearch 403s where sync works: async rewrites HEAD to GET
Complete solution
Covers this OpenSearch problem: AsyncOpenSearch 403s where sync works: async rewrites HEAD to GET. Client bug, confirmed and fixed.
The problem
Issue opensearch-project/opensearch-py#698 (closed, 10 comments): AsyncOpenSearch with AWSV4SignerAsyncAuth fails with 403 on ping() while the exact same sync code returns 200. The reporter narrowed it down: the sync client sends a HEAD request, the async client sends a GET. Their security team had locked down naked GET requests without an index in the path, so the async ping was the one that blew up. Looked like an auth misconfiguration at first, but the same credentials worked fine in sync.
What fixed it
Client bug, confirmed and fixed. The async connection classes (AsyncHttpConnection and AIOHttpConnection) rewrite every HEAD request as GET, a leftover workaround for an old aiohttp connection-reuse bug that was fixed upstream years ago (aiohttp#5012). Under SigV4 IAM auth with a tight policy, that silent HEAD-to-GET conversion turns into a 403, which is why sync works and async does not. The reporter verified the mechanism, and the fix shipped in opensearch-py PR #794, which stops the conversion. Resolution: upgrade opensearch-py past 2.4.x to a release containing #794. Until then, avoid bare ping() under locked-down IAM policies and call an index-scoped endpoint instead.
Evidence
In gh:opensearch-project/opensearch-py#698 (OpenSearch), the issue reported was: Issue opensearch-project/opensearch-py#698 (closed, 10 comments): AsyncOpenSearch with AWSV4SignerAsyncAuth fails with 403 on ping() while the exact same sync code returns 200. The reporter narrowed it down: the sync client sends a HEAD request, the async client sends a GET. Their security team had locked down naked GET requests without an index in the path, so the async ping was the one that blew up. Looked like an auth misconfiguration at first, but the same credentials worked fine in sync. The verified resolution from the thread: Client bug, confirmed and fixed. The async connection classes (AsyncHttpConnection and AIOHttpConnection) rewrite every HEAD request as GET, a leftover workaround for an old aiohttp connection-reuse bug that was fixed upstream years ago (aiohttp#5012). Under SigV4 IAM auth with a tight policy, that silent HEAD-to-GET conversion turns into a 403, which is why sync works and async does not. The reporter verified the mechanism, and the fix shipped in opensearch-py PR #794, which stops the conversion. Resolution: upgrade opensearch-py past 2.4.x to a release containing #794. Until then, avoid bare ping() under locked-down IAM policies and call an index-scoped endpoint instead. Source: mount the host path https at container path //github.com/opensearch-project/opensearch-py/issues/698
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_M9ZIab9RKVTZ5ga3YwSBfw" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">AsyncOpenSearch 403s where sync works: async rewrites HEAD to GET <span>— Vectle</span></a>