cohere.ClientV2 returns Resource not found error against an Azure-deployed Cohere model

TL;DR: if cohere.Client works but cohere.ClientV2 returns Resource not found error against an Azure deployment of a Cohere model, the deployment itself is the problem, not your code. The model must be deployed as a serverless endpoint: the "Deploy models to Azure AI model inference service" setting under Preview Features has to be OFF when you deploy. Also do not append /v1 to the base URL. The Cohere maintainer reproduced this on Azure and confirmed the toggle-off deployment fixes it.

Resource not found error

Steps

1. Redeploy the model with the "Deploy models to Azure AI model inference service" preview toggle set to OFF.
2. Point the client's base_url at the deployment URL without appending /v1.

3. Retry the ClientV2 call.

   Expected: embeddings return. Success check: no more resource-not-found.

When to use this

• Azure-deployed Cohere model (e.g. Cohere-embed-v3-english), ClientV2 fails while Client works.

When NOT to use this

• Non-Azure deployments; the toggle is Azure-specific.
• Auth errors; check the key.

Compatibility

• cohere-python ClientV2 against Azure AI deployments.

Variant phrasings

• "cohere ClientV2 azure resource not found"
• "cohere azure serverless endpoint ClientV2"

Root cause

With the preview toggle on, the model deploys to the AI model inference service, whose routing the Cohere client cannot address directly; the serverless endpoint shape is what ClientV2 expects.

Edge cases

• The maintainer confirmed deploying Embed V4 with the toggle off works; the pattern holds across embed models.

Source: https://github.com/cohere-ai/cohere-python/issues/661

In the thread, the asker reported: Creating embeddings worked with cohere.Client but cohere.ClientV2 returned "Resource not found error" against an Azure deployment of Cohere-embed-v3-english. Maintainer mkozakov reproduced it against Azure and got the fix from the Azure side: the model must be deployed as a serverless endpoint, which means the "Deploy models to Azure AI model inference service" setting under Preview Features has to be OFF when you deploy. Also do not append /v1 to the base URL. The maintainer confirmed deploying Embed V4 without that toggle enabled works. The verified answer that resolved it: When cohere.Client works but cohere.ClientV2 returns "Resource not found error" on Azure, the deployment itself is the problem, not your code. Redeploy the model with the "Deploy models to Azure AI model inference service" preview toggle set to OFF, which gives you a serverless endpoint the Cohere client can talk to directly. Point the client's baseurl at the deployment URL without appending /v1. The Cohere maintainer reproduced this on Azure and confirmed the toggle-off deployment fixes it. Source: https://github.com/cohere-ai/cohere-python/issues/661 Thread: https://vectle.com/threads/thrZoR7CORZURzjl2eGun3eQg