Stream error after 5 minutes: Connection broken InvalidChunkLength

Not specified by the original author.

TL;DR: if your streamed completions die at almost exactly 5 minutes with Connection broken: InvalidChunkLength(got length b'', 0 bytes read), you are hitting a streaming timeout that existed in the v0 client. Upgrade to v1 of the openai python library, where the client side was fixed. The server-side limit was improved too, so recent reports hit the wall closer to 10 minutes instead.

Connection broken: InvalidChunkLength(got length b'', 0 bytes read)

Steps

1. Upgrade the client:

pip install --upgrade openai

Expected: v1 of the library.

2. Re-run the long streaming request.

   Expected: the stream survives past 5 minutes. Success check: the full document rewrite completes.

3. If it still fails on v1, note where it dies; the wall moved toward 10 minutes after the server-side improvement, so check the new mark before digging into your own socket code.

When to use this

• Streamed responses reproducibly die at ~5 minutes; non-streaming with a 10-minute timeout only fails at 10 minutes.
• You run the v0 client.

When NOT to use this

• Connections fail immediately; that is networking or auth.
• Short responses fail; this timeout only bites long streams.

Compatibility

• openai python library v1+ (client fix); server-side limit also improved.

Variant phrasings

• "openai stream dies 5 minutes InvalidChunkLength"
• "openai python streaming timeout long response"

Root cause

The v0 client had a streaming-specific timeout the non-streaming path did not share, so long streams always cut out at the same mark.

Edge cases

• A longer server-side wall (~10 minutes) still exists; truly huge generations need chunking regardless.

The thread asked: Streaming long GPT-4 responses kept dying at exactly the 5-minute mark with Connection broken: InvalidChunkLength(got length b'', 0 bytes read). The reporter reproduced it every time by asking for a long document rewrite, and a non-streaming request with a 10-minute timeout only failed at the 10-minute mark, which pointed at a streaming-specific timeout. Maintainer rattrayalex confirmed the client side was fixed in v1 of the library, and noted the server-side limit had been improved too.

The verified answer: If your streamed completions die at almost exactly 5 minutes with Connection broken: InvalidChunkLength(got length b'', 0 bytes read), you are hitting a streaming timeout that existed in the v0 client. Upgrade to v1 of the openai python library, where the client side was fixed. The maintainers also said the server-side limit was improved, so recent reports mention hitting a wall closer to 10 minutes instead. If you still see failures on v1, check whether the timeout moved to the longer mark before digging into your own socket code.

Thread: https://vectle.com/threads/thr4HWfpbTuqHs-viPYdECbQ

Not specified by the original author.

Model Studio chat API: stream for long outputs, usage only arrives in the last chunk

TL;DR

Always stream chat completions that can run long; a non-streaming call that times out gives you truncated output with no error to catch. When you need token counts from a stream, set includeusage and read only the final chunk, the earlier chunks will not carry it. Context: Official docs (Model Studio OpenAI-compatible chat reference): documents that stream=true is recommended for long outputs because non-streaming calls can hit timeouts (the service may interrupt the request and return partial content instead of an error), and that token usage with includeusage only appears in the last data chunk of a streamed response.

Use this when you run into the situation in the title.

When not to use this skill: unrelated tasks. It covers only the procedure above.

Compatibility

The steps above apply to the commands named in them. This skill does not pin a version, so if a flag looks different on your machine, check your installed version's docs first.

Source: Official docs (Model Studio OpenAI-compatible chat reference): documents that stream=true is recommended for long outputs because non-streaming calls can hit timeouts (the service may interrupt the request and return partial content instead of an error), and that token usage with include_usage only appears in the last data chunk of a streamed response.

Verified resolution: Always stream chat completions that can run long; a non-streaming call that times out gives you truncated output with no error to catch. When you need token counts from a stream, set include_usage and read only the final chunk, the earlier chunks will not carry it.

Source URL: https://help.aliyun.com/en/model-studio/qwen-api-via-openai-chat-completions

Not specified by the original author.

TL;DR

Use it where partial progress matters to the user. Stream for user experience, not for speed. Streaming does not change total generation time; it reduces time to first token. Set timeouts at two levels: time to first token (catches a hung request early) and total stream time (catches a stream that trickles forever).

When to use

You are seeing this: Stream for user experience, not for speed. Use this skill when you run into "Stream OpenAI responses robustly: assemble events, handle cutoffs, never trust partial state".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Streaming without the footguns

The steps

1. Stream for user experience, not for speed. Streaming does not change total generation time; it reduces time to first token. Use it where partial progress matters to the user.
2. Assemble the full message from events before acting on it. Buffer text deltas, tool call deltas, and annotations; only when the stream signals completion do you have a message. Executing a tool call from a partial delta is a classic bug.
3. Handle the cutoff: if the connection drops mid-stream, you have a partial response and no completion signal. Decide the policy up front: resume, retry the whole request, or surface the partial with a clear "incomplete" marker. Never silently present a truncated answer as complete.
4. Set timeouts at two levels: time to first token (catches a hung request early) and total stream time (catches a stream that trickles forever). Both need values before production.
5. Do not stream structured outputs into parsers incrementally. Parse once, after assembly, with the same validation and refusal checks as a non-streamed call.
6. Log stream anomalies: unexpected event order, missing completion signals, mid-stream errors. These are the diagnostics that explain the one-in-a-thousand weird response.

The trap

Treating the stream as the message. Until the completion event arrives, everything is provisional. Code that acts on deltas as they arrive will eventually act on half a thought.

Checklist

• Full assembly before any action on the content.
• Dropped-connection policy defined and tested.
• First-token and total-stream timeouts set.
• Structured outputs parsed after assembly, validated as usual.
• Stream anomalies logged for diagnosis.

OpenAI's production best practices guide documents that stream true reduces time to first token without changing total time. Verified live at https://developers.openai.com/api/docs/guides/production-best-practices.