Immutable solution version
Ollama example errors: raise num_ctx for tool calling
Complete solution
A practical guide for Hugging Face smolagents. If the smolagents Ollama tool-calling example errors out, raise Ollama's context window: it defaults to 2048 tokens, which breaks the tool-calling prompts. Pass num_ctx to the LiteLLMModel, e.g.
The problem
Issue huggingface/smolagents#36 (closed, 10 comments): I tweaked example code using my external Ollama server URL (on LightningAI) and llama3.2:latest. model = LiteLLMModel(modelid="ollamachat/llama3.2:latest", apibase=os.getenv("OLLAMABASEURL"), apikey value [redacted]). The toolcallingagent_ollama.py example errors out.
The fix
If the smolagents Ollama tool-calling example errors out, raise Ollama's context window: it defaults to 2048 tokens, which breaks the tool-calling prompts. Pass numctx to the LiteLLMModel, e.g. LiteLLMModel(modelid='ollamachat/llama3.2:latest', apibase=..., num_ctx=32768). The reporter confirmed this fixed it. You can pass other Ollama request options the same way; see the Ollama API docs for the list.
Notes
Thread: https://github.com/huggingface/smolagents/issues/36. Verified against the closed issue and the maintainer/accepted answer there.
Evidence
In https://github.com/huggingface/smolagents/issues/36, the reporter asked: Issue huggingface/smolagents#36 (closed, 10 comments): I tweaked example code using my external Ollama server URL (on LightningAI) and llama3.2:latest. model = LiteLLMModel(modelid="ollamachat/llama3.2:latest", apibase=os.getenv("OLLAMABASEURL"), apikey value [redacted]). The toolcallingagentollama.py example errors out. The verified answer: If the smolagents Ollama tool-calling example errors out, raise Ollama's context window: it defaults to 2048 tokens, which breaks the tool-calling prompts. Pass numctx to the LiteLLMModel, e.g. LiteLLMModel(modelid='ollamachat/llama3.2:latest', apibase=..., numctx=32768). The reporter confirmed this fixed it. Source thread: https://github.com/huggingface/smolagents/issues/36
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_OaXqhUCCE53yayHh-2FOmw" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Ollama example errors: raise num_ctx for tool calling <span>— Vectle</span></a>