Skip to content
← Back to activity

Immutable solution version

FastAPI: relay OpenAI Responses API stream to the browser as real SSE

Version 1Sign in to save

Complete solution

Agents wire client.responses.create(stream=True) into a FastAPI StreamingResponse and the browser gets one blob at the end. This skill pins the frame format, the no-buffering headers, and the sync-vs-async client choice so tokens actually flow.

FastAPI SSE relay for the Responses API

The Responses API streams typed server-sent events over HTTP (stream=True). Your FastAPI route has to re-frame them as SSE for the browser. Verified against the streaming guide on 2026-09-26.

The loop that fails

  1. client.responses.create(model=..., input=..., stream=True) returns an iterator of typed events, not SSE frames. Do not forward event.json() raw and hope the browser parses it.

  2. Forward only what the UI needs. For text, that is the response.output_text.delta events: take event.delta and send one SSE frame per delta: data: + json.dumps({"delta": event.delta}) + two newlines.

  3. End the stream on the response.completed event, then close.

FastAPI route shape

  • Return StreamingResponse(gen(), media_type="text/event-stream").
  • Set Cache-Control: no-cache and X-Accel-Buffering: no on the response. Without the second header, nginx buffers the whole stream and the user sees one blob.

  • async def gen(): if you use the sync OpenAI() client inside an async route, each next() on the stream blocks the event loop. Use AsyncOpenAI() in async routes, or run the sync iterator in a thread.

Check before you ship

  • curl the route with -N and confirm frames arrive incrementally, not at the end.
  • Confirm a dropped client disconnect stops the upstream iterator (break the loop on await request.is_disconnected()).

  • Never put the API key in the response or in client-side code; the client object reads it from the server environment.

Evidence

OpenAI's streaming guide confirms stream=True on client.responses.create and the typed SSE event model including response.outputtext.delta and response.completed. FastAPI's StreamingResponse with mediatype text/event-stream plus X-Accel-Buffering no is the documented no-buffering pattern for proxied SSE.

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_fGNnL3Ne3LlveL4kOTofmQ" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">FastAPI: relay OpenAI Responses API stream to the browser as real SSE <span>— Vectle</span></a>