cohere.ClientV2 returns Resource not found error against an Azure-deployed Cohere model

cohere.ClientV2 returns Resource not found error against an Azure-deployed Cohere model

What is going on

Creating embeddings worked with cohere.Client but cohere.ClientV2 returned "Resource not found error" against an Azure deployment of Cohere-embed-v3-english. Maintainer mkozakov reproduced it against Azure and got the fix from the Azure side: the model must be deployed as a serverless endpoint, which means the "Deploy models to Azure AI model inference service" setting under Preview Features has to be OFF when you deploy. Also do not append /v1 to the base URL. The maintainer confirmed deploying Embed V4 without that toggle enabled works.

What actually works

When cohere.Client works but cohere.ClientV2 returns "Resource not found error" on Azure, the deployment itself is the problem, not your code. Redeploy the model with the "Deploy models to Azure AI model inference service" preview toggle set to OFF, which gives you a serverless endpoint the Cohere client can talk to directly. Point the client's base_url at the deployment URL without appending /v1. The Cohere maintainer reproduced this on Azure and confirmed the toggle-off deployment fixes it. Source: https://github.com/cohere-ai/cohere-python/issues/661

Source

Mined from a verified thread: https://vectle.com/threads/thr_ZoR7CORZURzjl2eGun3eQg (original: https://github.com/cohere-ai/cohere-python/issues/661)

In the thread, the asker reported: Creating embeddings worked with cohere.Client but cohere.ClientV2 returned "Resource not found error" against an Azure deployment of Cohere-embed-v3-english. Maintainer mkozakov reproduced it against Azure and got the fix from the Azure side: the model must be deployed as a serverless endpoint, which means the "Deploy models to Azure AI model inference service" setting under Preview Features has to be OFF when you deploy. Also do not append /v1 to the base URL. The maintainer confirmed deploying Embed V4 without that toggle enabled works. The verified answer that resolved it: When cohere.Client works but cohere.ClientV2 returns "Resource not found error" on Azure, the deployment itself is the problem, not your code. Redeploy the model with the "Deploy models to Azure AI model inference service" preview toggle set to OFF, which gives you a serverless endpoint the Cohere client can talk to directly. Point the client's baseurl at the deployment URL without appending /v1. The Cohere maintainer reproduced this on Azure and confirmed the toggle-off deployment fixes it. Source: https://github.com/cohere-ai/cohere-python/issues/661 Thread: https://vectle.com/threads/thrZoR7CORZURzjl2eGun3eQg