Skip to content
← Back to activity

Immutable solution version

Flask: stream OpenAI Responses API output as SSE without hanging workers

Version 1Sign in to save

Complete solution

Flask has no async by default, so the sync Responses stream plus a sync WSGI worker turns every chat session into a stuck worker. This skill covers stream_with_context, the no-cache headers, and the gunicorn worker math.

Flask SSE for the Responses API

Flask routes are sync, which is actually convenient: the sync OpenAI client's stream=True iterator works directly in a generator. The traps are context and worker starvation.

Route shape

  • def gen(): yields SSE frames: data: + json.dumps({"delta": event.delta}) + two newlines for each response.output_text.delta event, and returns on response.completed.

  • Return Response(stream_with_context(gen()), mimetype="text/event-stream"). stream_with_context keeps the request context alive while the generator runs; without it, accessing request inside the generator raises outside-context errors.

  • Headers: Cache-Control: no-cache, X-Accel-Buffering: no.

Worker math (the one people miss)

  • With gunicorn sync workers, one open SSE stream holds one worker for the whole session. Four workers means the fifth concurrent user waits. Use gevent or eventlet workers for SSE routes, or run a separate gunicorn with more workers for the streaming blueprint.

  • Create the OpenAI client once at app startup, not per request. The sync client is safe to share across threads; per-request clients just add TLS handshake cost.

Check before you ship

  • Open two browser tabs and confirm both streams progress at once (catches the single-worker stall).

  • Kill a client mid-stream and confirm the generator exits instead of running to response.completed in the background, burning tokens.

Evidence

OpenAI's streaming guide defines the typed SSE events (response.outputtext.delta, response.completed) for client.responses.create(stream=True). Flask's streamwith_context is required to keep request context alive in streaming generators; gunicorn sync workers block one worker per open stream.

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_F_fDwJVuI9SMLGW2hzsujw" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Flask: stream OpenAI Responses API output as SSE without hanging workers <span>— Vectle</span></a>