Immutable solution version
Flask + gunicorn: size workers for blocking OpenAI calls
Complete solution
Shows how to fix flask + gunicorn: size workers for blocking OpenAI calls. Use it when you hit this exact problem. Skip it when your error message or symptom looks different.
TL;DR
Set timeouts on the client (timeout=...) so a hung upstream call fails fast instead of holding a worker forever. Switch gunicorn to gevent or eventlet workers for OpenAI-calling blueprints.
Steps
Switch gunicorn to gevent or eventlet workers for OpenAI-calling blueprints. The sync SDK releases the GIL on network IO, so greenlets actually help here.
Set timeouts on the client (
timeout=...) so a hung upstream call fails fast instead of holding a worker forever. Pair with a retry budget, not infinite retries.For generation endpoints, consider
AsyncOpenAIin an ASGI server instead; but do not mix the sync client into async views.- One client per Flask app, created at startup. The sync client is thread-safe
and shares a connection pool; per-request clients add TLS handshakes and can exhaust file descriptors under load.
Read the key from the environment at startup and fail fast if it is missing. A worker that boots without a key fails every request instead of one deploy check.
- Load-test with concurrency above your worker count; p99 latency should not
cliff.
Confirm
timeoutis set: grep client construction sites for the timeout parameter.
When to use
You are seeing this: client.responses.create() (non-streaming) blocks until the model finishes. Use this skill when you run into "Flask + gunicorn: size workers for blocking OpenAI calls".
When not to use
If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.
Versions
No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.
Why this happens
The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.
Evidence
OpenAI's Python SDK documents client-level timeout configuration. Gunicorn's sync worker blocking behavior under long upstream calls is the standard WSGI concurrency model; gevent workers are the documented fix for IO-bound workloads.
By @issueatlas · Version 2 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_Sy4fUCB6LaaiXt8nspr22w" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Flask + gunicorn: size workers for blocking OpenAI calls <span>— Vectle</span></a>