Skip to content
← Back to activity

Immutable solution version

AsyncTogether is not reusing HTTP sessions - creates new session for each request

Version 1Sign in to save

Complete solution

How to fix 'AsyncTogether is not reusing HTTP sessions - creates new session for each request' in Together AI, straight from the verified resolution in gh:togethercomputer/together-python#346. Root cause plus what to change.

The problem

Issue togethercomputer/together-python#346 (closed, 2 comments): AsyncTogether created a new aiohttp ClientSession for every API request instead of reusing one, adding connection/setup overhead on each call and hurting performance in async workloads.

The verified resolution

Contributor blainekasten: 'This should be addressed in our new 2.0 SDK. I'm going to close this issue for now' - the fix path is upgrading to the 2.0 SDK.

Rule of thumb

for heavy async Together usage, upgrade to the 2.x SDK to get session reuse; on the old 1.x SDK per-request sessions are a known cost.

Evidence

Asked in gh:togethercomputer/together-python#346 (https://github.com/togethercomputer/together-python/issues/346): AsyncTogether is not reusing HTTP sessions - creates new session for each request. The issue report: Issue togethercomputer/together-python#346 (closed, 2 comments): AsyncTogether created a new aiohttp ClientSession for every API request instead of reusing one, adding connection/setup overhead on each call and hurting performance in async workloads. The verified answer: Contributor blainekasten: 'This should be addressed in our new 2.0 SDK. I'm going to close this issue for now' - the fix path is upgrading to the 2.0 SDK. Lesson: for heavy async Together usage, upgrade to the 2.x SDK to get session reuse; on the old 1.x SDK per-request sessions are a known cost.

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_-JRDocU1TXJqnLo_JU34vA" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">AsyncTogether is not reusing HTTP sessions - creates new session for each request <span>— Vectle</span></a>