Braintrust CLI ignores response_format when pushing prompts
Shows how to fix braintrust CLI ignores response_format when pushing prompts. Use it when you hit this exact problem. Skip it when your error message or symptom looks different.
TL;DR
This causes the invoke function to fail during schema validation because it receives text instead of the expected JSON structure.
Steps
- The Braintrust CLI doesn't accept zod schemas directly when pushing prompts with responseformat: a zod schema gets silently ignored and the prompt falls back to text output, failing schema validation on invoke. Fix: convert the zod schema to a JSON schema with zod-to-json-schema first, then push: import { zodToJsonSchema } from 'zod-to-json-schema' and build jsonschema as { name: 'schema', schema: zodToJsonSchema(YourZodSchema) }. If invoke still gets text instead of JSON, manually set the output format to JSON output in the UI as a sanity check.
When to use
You are seeing this: Issue braintrustdata/braintrust-sdk-javascript#765 (closed, 6 comments): When using the Braintrust CLI to push prompts with a structured response\format configuration, the CLI appears to ignore the json\schema settings and defaults to text output. Use this skill when you run into "Braintrust CLI ignores response_format when pushing prompts".
When not to use
If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.
Versions
Versions mentioned in the source: 0.0.208, 0.0.209. If you are on something much newer or older, the details may have shifted.
Why this happens
(closed, 6 comments): When using the Braintrust CLI to push prompts with a structured response\format configuration, the CLI appears to ignore the json\schema settings and defaults to text output.
Sources
- https://github.com/braintrustdata/braintrust-sdk-javascript/issues/765