Skip to content
← Back to activity

Immutable solution version

[BUG] LiteLLM issues with local ollama and Anthropic

Version 1Sign in to save

Complete solution

CrewAI: Collaborator lorenzejay: CrewAI moved off LiteLLM after v1 - route local models through an OpenAI-compatible endpoint instead, e.g. LLM(base_url='YOUR_HOST port 11434/v1', api_key [your value], model='...').

The problem

Issue crewAIInc/crewAI#3811 (closed, 19 comments): ### Description In version 0.203.1, I used to have multiple LLM configured I could switch to like the following and they were all working. I don't think I changed anything and these 2 don't work anymore in 1.2.0. The OpenAI vLLM endpoint for gpt-oss is the only one working. `` # Also have ANTHROPIC_API_KEY set in my .env lm_claude = LLM( model="anthropic/claude-sonnet-4-5" ) llm = llm_claude llm_local_ollama = LLM( # model="gpt-4o", api_key [your value], model="ollama/gpt-oss:20b", base_url="YOUR_HOST port 11434" ) llm = llm_local_ollama `` ### Steps to Reproduce Use the LLM in th

The verified fix

Collaborator lorenzejay: CrewAI moved off LiteLLM after v1 - route local models through an OpenAI-compatible endpoint instead, e.g. LLM(baseurl='YOURHOST port 11434/v1', apikey [your value], model='...'). Contributor Vidit-Ostwal: passing islitellm=True to LLM() makes it fall back to LiteLLM, and 'Fallback to LiteLLM is not available' is fixed by pip install litellm.

Evidence

In gh:crewAIInc/crewAI#3811 (https://github.com/crewAIInc/crewAI/issues/3811), someone reported: Issue crewAIInc/crewAI#3811 (closed, 19 comments): ### Description In version 0.203.1, I used to have multiple LLM configured I could switch to like the following and they were all working. I don't think I changed anything and these 2 don't work anymore in 1.2.0. The OpenAI vLLM endpoint for gpt-oss is the only one working. `` # Also have ANTHROPIC_API_KEY set in my .env lm_claude = LLM( model="anthropic/claude-sonnet-4-5" ) llm = llm_claude llm_local_ollama = LLM( # model="gpt-4o", api_key [your value], model="ollama/gpt-oss:20b", base_url="YOUR_HOST port 11434" ) llm = llm_local_ollama `` ### Steps to Reproduce Use the LLM in th The verified resolution from the thread: Collaborator lorenzejay: CrewAI moved off LiteLLM after v1 - route local models through an OpenAI-compatible endpoint instead, e.g. LLM(baseurl='YOURHOST port 11434/v1', apikey [your value], model='...'). Contributor Vidit-Ostwal: passing islitellm=True to LLM() makes it fall back to LiteLLM, and 'Fallback to LiteLLM is not available' is fixed by pip install litellm.

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_u1s3FVPSfHYsj_tdpbYgMg" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">[BUG] LiteLLM issues with local ollama and Anthropic <span>— Vectle</span></a>