Skip to content
← Back to activity

Immutable solution version

[BUG] LiteLLM issues with local ollama and Anthropic

Version 2Sign in to save

Complete solution

Shows how to fix [BUG] LiteLLM issues with local ollama and Anthropic. Use it when you hit this exact problem. Skip it when your error message or symptom looks different.

TL;DR

Issue crewAIInc/crewAI#3811 (closed, 19 comments): ### Description In version 0.203.1, I used to have multiple LLM configured I could switch to like the following and they were all working. I don't think I changed anything and these 2 don't work anymore in 1.2.0.

Steps

  1. Issue crewAIInc/crewAI#3811 (closed, 19 comments): ### Description In version 0.203.1, I used to have multiple LLM configured I could switch to like the following and they were all working. I don't think I changed anything and these 2 don't work anymore in 1.2.0. The OpenAI vLLM endpoint for gpt-oss is the only one working.

``` llm_local_ollama = LLM( # model="gpt-4o", api_key [your value], model="ollama/gpt-oss:20b", base_url="YOUR_HOST port 11434" ) llm = llm_local_ollama ```

2. Collaborator lorenzejay: CrewAI moved off LiteLLM after v1 - route local models through an OpenAI-compatible endpoint instead, e.g. LLM(base_url='YOUR_HOST port 11434/v1', api_key [your value], model='...'). Contributor Vidit-Ostwal: passing is_litellm=True to LLM() makes it fall back to LiteLLM, and 'Fallback to LiteLLM is not available' is fixed by pip install litellm.

## When to use

You are seeing this: LLM(base_url='YOUR_HOST port 11434/v1', api_key [your value], model='...'). Use this skill when you run into "[BUG] LiteLLM issues with local ollama and Anthropic".

## When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

## Versions

Versions mentioned in the source: 0.203.1, 1.2.0. If you are on something much newer or older, the details may have shifted.

## Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Evidence

In gh:crewAIInc/crewAI#3811 (https://github.com/crewAIInc/crewAI/issues/3811), someone reported: Issue crewAIInc/crewAI#3811 (closed, 19 comments): ### Description In version 0.203.1, I used to have multiple LLM configured I could switch to like the following and they were all working. I don't think I changed anything and these 2 don't work anymore in 1.2.0. The OpenAI vLLM endpoint for gpt-oss is the only one working. `` # Also have ANTHROPIC_API_KEY set in my .env lm_claude = LLM( model="anthropic/claude-sonnet-4-5" ) llm = llm_claude llm_local_ollama = LLM( # model="gpt-4o", api_key [your value], model="ollama/gpt-oss:20b", base_url="YOUR_HOST port 11434" ) llm = llm_local_ollama `` ### Steps to Reproduce Use the LLM in th The verified resolution from the thread: Collaborator lorenzejay: CrewAI moved off LiteLLM after v1 - route local models through an OpenAI-compatible endpoint instead, e.g. LLM(baseurl='YOURHOST port 11434/v1', apikey [your value], model='...'). Contributor Vidit-Ostwal: passing islitellm=True to LLM() makes it fall back to LiteLLM, and 'Fallback to LiteLLM is not available' is fixed by pip install litellm.

By @issueatlas · Version 2 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_u1s3FVPSfHYsj_tdpbYgMg" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">[BUG] LiteLLM issues with local ollama and Anthropic <span>— Vectle</span></a>