# FastAPI Realtime websocket relay

Use a relay when your server captures audio or must keep the project API key
server-side while the browser only holds an ephemeral secret. Verified against
the Realtime websocket docs on 2026-09-26.

## Connection setup

1. Open a server-side websocket to the Realtime endpoint with the header
   `Authorization: Bearer` plus your server-side key. (Keep the key out of any
   client-visible code; the relay socket lives entirely on the server.)
2. Send `session.start` as the FIRST message, with model, instructions, audio
   format, and voice inside the `session` object.
3. Wait for `session.started` before sending audio or commands. It carries the
   resolved session config and session id. Sending audio first is silently
   dropped.

## Audio framing

- Websocket audio is manual: send JSON events with base64-encoded PCM16 mono
  audio at 24 kHz via `input_audio_buffer.append`.
- Read `response.audio.delta` events from OpenAI and forward the base64 chunks
  to the browser client over your own websocket.
- In FastAPI, use the `websockets` library (or any ASGI websocket) for the
  upstream socket, and pair it with the client websocket in an async relay loop.

## Traps

- Base64 your audio; raw bytes in a JSON event corrupt the frame.
- Handle backpressure: if the browser is slow, pause reading upstream rather
  than buffering unbounded audio in memory.
- On either socket closing, close the other one. A half-open relay holds the
  OpenAI session (and its cost) open.

## Check before you ship

- Log the first five upstream messages; the sequence must be session.start,
  then session.started, then audio.
- Test a mid-call disconnect and confirm the upstream socket closes within
  seconds.