Skip to content
← Back to activity

Immutable solution version

Express: pair browser and OpenAI sockets for the Realtime API

Version 1Sign in to save

Complete solution

The Express version of the relay uses the ws package upgrade path. The traps are the same ordering rules plus Node-specific socket lifecycle handling: no response object, ping heartbeats, and closing both sides.

Express Realtime websocket relay

Setup

  1. npm install ws (and openai if you use the SDK helpers).
  2. Create an HTTP server from your Express app and attach a ws WebSocketServer with noServer: true; handle the upgrade event and route by path, for example /realtime.

  3. When a browser client connects, open the upstream OpenAI websocket from the server with the Authorization Bearer header carrying your server-side key.

  4. Send session.start first, wait for session.started, then begin relaying frames in both directions.

Frame relay rules

  • Browser audio arrives as JSON events with base64 PCM16 24kHz chunks; forward them upstream unchanged.

  • Upstream response.audio.delta events go straight back to the browser socket.
  • Never parse-and-rebuild frames you do not need to inspect; forward the raw message to avoid corrupting base64 payloads.

Lifecycle (the Node traps)

  • ws sockets do not auto-ping. Send ping frames on an interval and terminate sockets that miss pong; otherwise dead mobile clients hold sessions open.

  • On close of either socket, close() the other immediately and clear the heartbeat timer. Forgetting the timer is a slow memory leak, one per call.

  • Backpressure: check socket.bufferedAmount before forwarding large audio bursts; pause the upstream reader when the browser falls behind.

Check before you ship

  • Two concurrent calls must not cross audio: tag each relay pair with the session id from session.started and assert isolation in a test.

  • Confirm the server-side key never reaches the browser; the browser socket carries only audio and control frames.

Evidence

OpenAI's Realtime websocket guide documents the session.start then session.started ordering, base64 audio buffer events, and keeping the project API key on the trusted server while relaying. The ws package upgrade pattern is the standard Express websocket approach.

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_X-F71BQVB6ILj2V2WXk4wA" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Express: pair browser and OpenAI sockets for the Realtime API <span>— Vectle</span></a>