Immutable solution version
Stream error after 5 minutes: Connection broken InvalidChunkLength
Complete solution
Stream error after 5 minutes: Connection broken InvalidChunkLength. If your streamed completions die at almost exactly 5 minutes with `Connection broken: InvalidChunkLength(got length b'', 0 bytes read)`, you are hitting a streaming timeout that existed in the v0 client.
The problem
Streaming long GPT-4 responses kept dying at exactly the 5-minute mark with Connection broken: InvalidChunkLength(got length b'', 0 bytes read). The reporter reproduced it every time by asking for a long document rewrite, and a non-streaming request with a 10-minute timeout only failed at the 10-minute mark, which pointed at a streaming-specific timeout. Maintainer rattrayalex confirmed the client side was fixed in v1 of the library, and noted the server-side limit had been improved too.
The fix
If your streamed completions die at almost exactly 5 minutes with Connection broken: InvalidChunkLength(got length b'', 0 bytes read), you are hitting a streaming timeout that existed in the v0 client. Upgrade to v1 of the openai python library, where the client side was fixed. The maintainers also said the server-side limit was improved, so recent reports mention hitting a wall closer to 10 minutes instead. If you still see failures on v1, check whether the timeout moved to the longer mark before digging into your own socket code.
Evidence
The thread asked: Streaming long GPT-4 responses kept dying at exactly the 5-minute mark with Connection broken: InvalidChunkLength(got length b'', 0 bytes read). The reporter reproduced it every time by asking for a long document rewrite, and a non-streaming request with a 10-minute timeout only failed at the 10-minute mark, which pointed at a streaming-specific timeout. Maintainer rattrayalex confirmed the client side was fixed in v1 of the library, and noted the server-side limit had been improved too.
The verified answer: If your streamed completions die at almost exactly 5 minutes with Connection broken: InvalidChunkLength(got length b'', 0 bytes read), you are hitting a streaming timeout that existed in the v0 client. Upgrade to v1 of the openai python library, where the client side was fixed. The maintainers also said the server-side limit was improved, so recent reports mention hitting a wall closer to 10 minutes instead. If you still see failures on v1, check whether the timeout moved to the longer mark before digging into your own socket code.
Thread: https://vectle.com/threads/thr4HWfpbTuqHs-viPYdECbQ
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_ATLNcCeCxDoJdP4ZHFUcSA" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Stream error after 5 minutes: Connection broken InvalidChunkLength <span>— Vectle</span></a>