Immutable solution version
Model resilience for agents: retry and fallback middleware
Complete solution
Make agents survive provider blips with ModelRetryMiddleware (exponential backoff on retryable model errors) and ModelFallbackMiddleware (try the next model when the primary fails). Both are middleware, so they compose with the rest of the stack.
Retry and fallback middleware
from langchain.agents import create_agent
from langchain.agents.middleware import ModelRetryMiddleware, ModelFallbackMiddleware
agent = create_agent(
model="gpt-5.5",
tools=[...],
middleware=[
ModelRetryMiddleware(
max_retries=3,
backoff_factor=2.0,
initial_delay=1.0,
),
ModelFallbackMiddleware(
"gpt-5.4-mini",
"claude-3-5-sonnet-20241022",
),
],
)How they decide
- Retry respects each exception's
is_retryableattribute by default: rate limits, timeouts, connection errors, and server errors retry; auth errors, not-found, bad requests, and context overflow do not. on_failurecontrols exhaustion:continuereturns an AIMessage with the error details so the agent can handle it,errorre-raises, or pass a callable for custom content.- Fallback tries the models in order when the primary fails. Model identifiers can be strings like
openai:gpt-5.4-minior model instances.
Rules
- Retry handles transient failures; fallback handles outages. Use both for anything user-facing.
- Backoff has jitter on by default; leave it on so a fleet of agents does not thundering-herd the provider.
- Fallback models should be cheaper or more available than the primary, and they must support the features the agent uses (tool calling at minimum). A fallback that cannot call tools just fails differently.
Evidence
Workflow grounded in the official middleware docs (model retry, model fallback), verified 2026-09-26. Middleware: https://docs.langchain.com/oss/python/langchain/middleware/built-in
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_sE0paAJkAHEGW38fSemApw" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Model resilience for agents: retry and fallback middleware <span>— Vectle</span></a>