DeepInfra with the OpenAI SDK: point base_url at api.deepinfra.com

TL;DR

Set base_url to https://api.deepinfra.com/v1/openai and pass your DeepInfra API key - existing OpenAI SDK code for chat completions works unchanged. Use DeepInfra model IDs like meta-llama/Meta-Llama-3.1-70B-Instruct instead of OpenAI model names. DeepInfra exposes an OpenAI-compatible endpoint.

Fix

1. Set base_url to https://api.deepinfra.com/v1/openai and pass your DeepInfra API key - existing OpenAI SDK code for chat completions works unchanged. Expected: You get the expected result; the problem is gone.

2. Use DeepInfra model IDs like meta-llama/Meta-Llama-3.1-70B-Instruct instead of OpenAI model names. Expected: You get the expected result; the problem is gone.

When to use

• You are setting up or using this DeepInfra with the OpenAI SDK feature.
• The symptom matches: point base_url at api.deepinfra.com.

When NOT to use

• Unrelated DeepInfra with the OpenAI SDK issues (different feature, different failure).
• You need general documentation for the tool; check the official docs instead.

Compatibility

Reported against DeepInfra with the OpenAI SDK (mentions 3.1).

Variant phrasings

point base_url at api.deepinfra.com

Why it happens

No new SDK needed. Swap the base URL and keep your OpenAI client code.

Edge cases

• If your error message differs even slightly, this is probably a different issue; search the exact text.
• Behavior can change between releases; the linked source reflects the versions above.
• If the fix does not help, capture the full error output and check the source link for updates.

Source: No new SDK needed. Swap the base URL and keep your OpenAI client code.

Verified resolution: [DeepInfra docs]: DeepInfra exposes an OpenAI-compatible endpoint. Set base_url to https://api.deepinfra.com/v1/openai and pass your DeepInfra API key: existing OpenAI SDK code for chat completions works unchanged. Use DeepInfra model IDs like meta-llama/Meta-Llama-3.1-70B-Instruct instead of OpenAI model names. If you get a model-not-found error, the ID is wrong or the model is not served on DeepInfra: check the model list first.

Source URL: https://deepinfra.com/docs/openai_api