Ship an OpenAI integration: the pre-launch checklist agents skip

TL;DR

Set timeouts and retries on every call: a request timeout, a retry policy with backoff and jitter, and a cap on retries. Unpinned dependencies drift; a silent SDK or model update changing behavior in production is avoidable. No prompt change, model change, or retrieval change ships without the eval passing. Set cost alerts before traffic arrives: project budgets, per-feature usage logging, and an alert at a fraction of budget.

When to use

You are seeing this: Pin the SDK version and the model version. Use this skill when you run into "Ship an OpenAI integration: the pre-launch checklist agents skip".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

The steps

1. Pin the SDK version and the model version. Unpinned dependencies drift; a silent SDK or model update changing behavior in production is avoidable.
2. Set timeouts and retries on every call: a request timeout, a retry policy with backoff and jitter, and a cap on retries. The rate-limit skill covers 429s; this covers everything else that can hang.
3. Gate deploys on the eval. No prompt change, model change, or retrieval change ships without the eval passing. The eval is the test suite for behavior.
4. Set cost alerts before traffic arrives: project budgets, per-feature usage logging, and an alert at a fraction of budget. The first bill should never be a surprise.
5. Scope keys and separate projects per environment. Production traffic runs on production keys in the production project; nothing else touches them.
6. Plan observability: log request ids, token usage, latency, and error classes per feature. When something degrades, these logs are the difference between a ten-minute diagnosis and a ten-hour one.
7. Document the fallback: what happens when the API is down or rate-limited hard. Degraded mode, cached responses, or a clean error to the user; pick one per feature before you need it.
8. Review the data flow for privacy: what user data goes into prompts, what gets logged, what gets cached. Strip what you do not need before it leaves your infrastructure.

The trap

Launching the prototype. Prototypes skip every item above because "we will harden it later." Later is during the incident. The checklist takes an afternoon; the incident takes a week.

Checklist

• SDK and model versions pinned.
• Timeouts, retries, and caps on every call.
• Eval gates every deploy.
• Cost alerts live before traffic.
• Keys scoped, projects separated.
• Observability per feature; fallback documented; data flow reviewed.

OpenAI's production best practices guide covers streaming, MLOps lifecycle, and data/model management for production. Verified live at https://developers.openai.com/api/docs/guides/production-best-practices.