Skip to content
← Back to activity

Immutable solution version

Ollama example errors: raise num_ctx for tool calling

Version 1Sign in to save

Complete solution

A practical guide for Hugging Face smolagents. If the smolagents Ollama tool-calling example errors out, raise Ollama's context window: it defaults to 2048 tokens, which breaks the tool-calling prompts. Pass num_ctx to the LiteLLMModel, e.g.

The problem

Issue huggingface/smolagents#36 (closed, 10 comments): I tweaked example code using my external Ollama server URL (on LightningAI) and llama3.2:latest. model = LiteLLMModel(modelid="ollamachat/llama3.2:latest", apibase=os.getenv("OLLAMABASEURL"), apikey value [redacted]). The toolcallingagent_ollama.py example errors out.

The fix

If the smolagents Ollama tool-calling example errors out, raise Ollama's context window: it defaults to 2048 tokens, which breaks the tool-calling prompts. Pass numctx to the LiteLLMModel, e.g. LiteLLMModel(modelid='ollamachat/llama3.2:latest', apibase=..., num_ctx=32768). The reporter confirmed this fixed it. You can pass other Ollama request options the same way; see the Ollama API docs for the list.

Notes

Thread: https://github.com/huggingface/smolagents/issues/36. Verified against the closed issue and the maintainer/accepted answer there.

Evidence

In https://github.com/huggingface/smolagents/issues/36, the reporter asked: Issue huggingface/smolagents#36 (closed, 10 comments): I tweaked example code using my external Ollama server URL (on LightningAI) and llama3.2:latest. model = LiteLLMModel(modelid="ollamachat/llama3.2:latest", apibase=os.getenv("OLLAMABASEURL"), apikey value [redacted]). The toolcallingagentollama.py example errors out. The verified answer: If the smolagents Ollama tool-calling example errors out, raise Ollama's context window: it defaults to 2048 tokens, which breaks the tool-calling prompts. Pass numctx to the LiteLLMModel, e.g. LiteLLMModel(modelid='ollamachat/llama3.2:latest', apibase=..., numctx=32768). The reporter confirmed this fixed it. Source thread: https://github.com/huggingface/smolagents/issues/36

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_OaXqhUCCE53yayHh-2FOmw" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Ollama example errors: raise num_ctx for tool calling <span>— Vectle</span></a>