Flask: stream OpenAI Responses API output as SSE without hanging workers

TL;DR

Use gevent or eventlet workers for SSE routes, or run a separate gunicorn with more workers for the streaming blueprint. - Create the OpenAI client once at app startup, not per request. The sync client is safe to share across threads; per-request clients just add TLS handshake cost.

Steps

1. • With gunicorn sync workers, one open SSE stream holds one worker for the whole

   session. Four workers means the fifth concurrent user waits. Use gevent or eventlet workers for SSE routes, or run a separate gunicorn with more workers for the streaming blueprint.

• Create the OpenAI client once at app startup, not per request. The sync client is safe to share across threads; per-request clients just add TLS handshake cost.

2. • Open two browser tabs and confirm both streams progress at once (catches the

   single-worker stall).

• Kill a client mid-stream and confirm the generator exits instead of running to response.completed in the background, burning tokens.

When to use

You are seeing this: Flask routes are sync, which is actually convenient: the sync OpenAI client's stream=True iterator works directly in a generator. Use this skill when you run into "Flask: stream OpenAI Responses API output as SSE without hanging workers".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

OpenAI's streaming guide defines the typed SSE events (response.outputtext.delta, response.completed) for client.responses.create(stream=True). Flask's streamwith_context is required to keep request context alive in streaming generators; gunicorn sync workers block one worker per open stream.