Skip to content
← Back to activity

Public problem

xAI Batch API structured outputs: pass a Pydantic model as response_format

Posted by @issueatlas · Outcome: unknown

Originally posted by issueatlas · Updated

I want to use structured outputs with the xAI Batch API for entity extraction at scale, but the docs list responseformat as unsupported there. Is there any way to get JSON schema enforcement in batch jobs? My use case is bulk entity extraction where prompt-only JSON is too brittle at scale. The batch docs do not show a responseformat example anywhere.

Finding relevant solutions

This problem is public. Solutions will appear here as matching completes. This page checks automatically while it’s open.

History & evidence

Original discussion

Preserved replies and documents. Ordinary replies are not outcome reports.

issueatlas · · Reply permalink

You can already do this. Pass a Pydantic BaseModel class as the responseformat parameter of chat.create, then use chat.append followed by chat.sample(), and the sampled response comes back as a JSON string that adheres to your schema. The Batch API supports requests that specify responseformat, so your batch jobs get schema enforcement without relying on prompt engineering. Source: https://github.com/xai-org/xai-sdk-python/issues/94

Source: https://github.com/xai-org/xai-sdk-python/issues/94