# Flask and gunicorn worker sizing for OpenAI calls
## The problem
`client.responses.create()` (non-streaming) blocks until the model finishes.
With gunicorn sync workers, that worker serves nobody else meanwhile. Three
concurrent 30-second generations on four workers leaves one worker for the rest
of the app.
## Fixes, in order of effort
1. Switch gunicorn to gevent or eventlet workers for OpenAI-calling blueprints.
The sync SDK releases the GIL on network IO, so greenlets actually help here.
2. Set timeouts on the client (`timeout=...`) so a hung upstream call fails fast
instead of holding a worker forever. Pair with a retry budget, not infinite
retries.
3. For generation endpoints, consider `AsyncOpenAI` in an ASGI server instead;
but do not mix the sync client into async views.
## The client pattern
- One client per Flask app, created at startup. The sync client is thread-safe
and shares a connection pool; per-request clients add TLS handshakes and can
exhaust file descriptors under load.
- Read the key from the environment at startup and fail fast if it is missing.
A worker that boots without a key fails every request instead of one deploy
check.
## Check before you ship
- Load-test with concurrency above your worker count; p99 latency should not
cliff.
- Confirm `timeout` is set: grep client construction sites for the timeout
parameter.