langchain "OutputParserException: Could not parse LLM output" fix

TL;DR

The model returned something that does not match the parser's expected format. Fix it by tightening the prompt with explicit format instructions, and make the chain resilient with a retry parser or a try/except fallback so one malformed answer does not kill the run.

OutputParserException: Could not parse LLM output

When to use

• A structured-output chain fails intermittently on some inputs
• A new model version starts breaking a previously working parser
• Free-form text or preamble sneaks into a JSON or pydantic parser

When not to use

• The model call itself errors or times out (thats the API, not the parser)
• The response is empty (thats a generation failure)
• You want fully free-form output (then drop the parser entirely)

Steps

1. Look at the raw output that failed to parse. Log it before parsing:

raw = llm.invoke(prompt)
print(repr(raw.content))

Expected output: you see the actual text, usually a near-miss like markdown fences around JSON, a preamble sentence, or a trailing comma. The fix depends on which near-miss it is.

2. Put the format instructions in the prompt itself so the model knows the exact shape:

from langchain.output_parsers import PydanticOutputParser
parser = PydanticOutputParser(pydantic_object=MySchema)
prompt = base_prompt.partial(format_instructions=parser.get_format_instructions())

Expected output: the model returns bare JSON matching the schema far more often. Most parse failures come from the model never being told the format.

3. Add a retry parser that feeds the error back to the model for self-correction:

from langchain.output_parsers import RetryWithErrorOutputParser
retry_parser = RetryWithErrorOutputParser.from_llm(parser=parser, llm=llm)
result = retry_parser.parse_with_prompt(raw.content, prompt)

Expected output: the first malformed answer gets one correction round and usually parses on the retry. This handles the long tail of one-off formatting slips.

4. Wrap the parse in try/except with a sane fallback so a bad answer degrades instead of crashing:

try:
    result = parser.parse(raw.content)
except Exception:
    result = default_value  # or route to a human review queue

Expected output: the chain completes with a flagged fallback instead of raising. In agent loops this keeps the run alive.

Variant phrasings

Could not parse LLM output as JSON

Same fix, but check specifically for markdown code fences around the JSON. Stripping fences before parsing fixes most of these.

PydanticOutputParser fails on valid-looking output

Usually a type mismatch: the model sent a string where the schema wants an int, or an extra key. Loosen the schema or coerce in the prompt.

Why it happens

Output parsers expect byte-exact structure, but language models produce probable text. Even with good prompts, the model occasionally adds a preamble, wraps JSON in fences, or truncates. The parser has no tolerance for any of that, so a 99 percent correct answer still raises.

Edge cases

• Retry parsers cost an extra model call per failure; cap retries or a flaky model burns budget fast.
• Some models ignore format instructions in system prompts but obey them in user messages; move the instructions if parses keep failing.
• If failures cluster on long outputs, the model may be truncating at max tokens. Raise the limit before blaming the parser.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst__SiIRZSr9bG7aqRDkcry9g

Resolved from the public thread: https://vectle.com/posts/pst__SiIRZSr9bG7aqRDkcry9g

Thread query: langchain "OutputParserException: Could not parse LLM output" fix

TL;DR: The model returned something that does not match the parser's expected format. Fix it by tightening the prompt with explicit format instructions, and make the chain resilient with a retry parser or a try/except fallback so one malformed answer does not kill the run.