Express: pair browser and OpenAI sockets for the Realtime API

TL;DR

1. npm install ws (and openai if you use the SDK helpers). Create an HTTP server from your Express app and attach a ws WebSocketServer with noServer: true; handle the upgrade event and route by path, for example /realtime. Forgetting the timer is a slow memory leak, one per call. - Backpressure: check socket.bufferedAmount before forwarding large audio bursts; pause the upstream reader when the browser falls behind.

When to use

You are seeing this: When a browser client connects, open the upstream OpenAI websocket from the server with the Authorization Bearer header carrying your server-side key. Use this skill when you run into "Express: pair browser and OpenAI sockets for the Realtime API".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Express Realtime websocket relay

Setup

1. npm install ws (and openai if you use the SDK helpers).

2. Create an HTTP server from your Express app and attach a ws WebSocketServer with noServer: true; handle the upgrade event and route by path, for example /realtime.

3. When a browser client connects, open the upstream OpenAI websocket from the server with the Authorization Bearer header carrying your server-side key.

4. Send session.start first, wait for session.started, then begin relaying frames in both directions.

Frame relay rules

• Browser audio arrives as JSON events with base64 PCM16 24kHz chunks; forward them upstream unchanged.

• Upstream response.audio.delta events go straight back to the browser socket.

• Never parse-and-rebuild frames you do not need to inspect; forward the raw message to avoid corrupting base64 payloads.

Lifecycle (the Node traps)

• ws sockets do not auto-ping. Send ping frames on an interval and terminate sockets that miss pong; otherwise dead mobile clients hold sessions open.

• On close of either socket, close() the other immediately and clear the heartbeat timer. Forgetting the timer is a slow memory leak, one per call.

• Backpressure: check socket.bufferedAmount before forwarding large audio bursts; pause the upstream reader when the browser falls behind.

Check before you ship

• Two concurrent calls must not cross audio: tag each relay pair with the session id from session.started and assert isolation in a test.

• Confirm the server-side key never reaches the browser; the browser socket carries only audio and control frames.

OpenAI's Realtime websocket guide documents the session.start then session.started ordering, base64 audio buffer events, and keeping the project API key on the trusted server while relaying. The ws package upgrade pattern is the standard Express websocket approach.