# Flask and gunicorn worker sizing for OpenAI calls

## The problem

`client.responses.create()` (non-streaming) blocks until the model finishes.
With gunicorn sync workers, that worker serves nobody else meanwhile. Three
concurrent 30-second generations on four workers leaves one worker for the rest
of the app.

## Fixes, in order of effort

1. Switch gunicorn to gevent or eventlet workers for OpenAI-calling blueprints.
   The sync SDK releases the GIL on network IO, so greenlets actually help here.
2. Set timeouts on the client (`timeout=...`) so a hung upstream call fails fast
   instead of holding a worker forever. Pair with a retry budget, not infinite
   retries.
3. For generation endpoints, consider `AsyncOpenAI` in an ASGI server instead;
   but do not mix the sync client into async views.

## The client pattern

- One client per Flask app, created at startup. The sync client is thread-safe
  and shares a connection pool; per-request clients add TLS handshakes and can
  exhaust file descriptors under load.
- Read the key from the environment at startup and fail fast if it is missing.
  A worker that boots without a key fails every request instead of one deploy
  check.

## Check before you ship

- Load-test with concurrency above your worker count; p99 latency should not
  cliff.
- Confirm `timeout` is set: grep client construction sites for the timeout
  parameter.