Ollama example errors: raise num_ctx for tool calling

TL;DR

Pass num_ctx to the LiteLLMModel, e.g. The reporter confirmed this fixed it.

Steps

1. If the smolagents Ollama tool-calling example errors out, raise Ollama's context window: it defaults to 2048 tokens, which breaks the tool-calling prompts. Pass numctx to the LiteLLMModel, e.g. LiteLLMModel(modelid='ollamachat/llama3.2:latest', apibase=..., num_ctx=32768). The reporter confirmed this fixed it. You can pass other Ollama request options the same way; see the Ollama API docs for the list.

When to use

You are seeing this: Issue huggingface/smolagents#36 (closed, 10 comments): I tweaked example code using my external Ollama server URL (on LightningAI) and llama3.2:latest. model = LiteLLMModel(modelid="ollamachat/llama3.2:latest", apibase=os.getenv("OLLAMABASEURL"),... Use this skill when you run into "Ollama example errors: raise numctx for tool calling".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Sources

• https://github.com/huggingface/smolagents/issues/36

In https://github.com/huggingface/smolagents/issues/36, the reporter asked: Issue huggingface/smolagents#36 (closed, 10 comments): I tweaked example code using my external Ollama server URL (on LightningAI) and llama3.2:latest. model = LiteLLMModel(modelid="ollamachat/llama3.2:latest", apibase=os.getenv("OLLAMABASEURL"), apikey value [redacted]). The toolcallingagentollama.py example errors out. The verified answer: If the smolagents Ollama tool-calling example errors out, raise Ollama's context window: it defaults to 2048 tokens, which breaks the tool-calling prompts. Pass numctx to the LiteLLMModel, e.g. LiteLLMModel(modelid='ollamachat/llama3.2:latest', apibase=..., numctx=32768). The reporter confirmed this fixed it. Source thread: https://github.com/huggingface/smolagents/issues/36