Immutable solution version
Braintrust CLI ignores response_format when pushing prompts
Complete solution
A practical guide for Braintrust. The Braintrust CLI doesn't accept zod schemas directly when pushing prompts with response_format: a zod schema gets silently ignored and the prompt falls back to text output, failing schema validation on invoke. Fix: convert the zod schema to a JSON schema with zod-to-json-schema first, then push: import { zodToJsonSchema } from 'zod-to-json-schema' and build json_schema as { name: 'schema', schema: zodToJsonSchema(YourZodSchema) }.
The problem
Issue braintrustdata/braintrust-sdk-javascript#765 (closed, 6 comments): When using the Braintrust CLI to push prompts with a structured response_format configuration, the CLI appears to ignore the json_schema settings and defaults to text output. This causes the invoke function to fail during schema validation because it receives text instead of the expected JSON structure. However, when manually setting the output format to "JSON output" in the Braintrust UI after pushing, the prompts work correctly. ## Environment Braintrust CLI version: 0.0.208 and 0.0.209 (tested both) Node.js version: 22 Package manager*: Bun 1.2.
The fix
The Braintrust CLI doesn't accept zod schemas directly when pushing prompts with responseformat: a zod schema gets silently ignored and the prompt falls back to text output, failing schema validation on invoke. Fix: convert the zod schema to a JSON schema with zod-to-json-schema first, then push: import { zodToJsonSchema } from 'zod-to-json-schema' and build jsonschema as { name: 'schema', schema: zodToJsonSchema(YourZodSchema) }. If invoke still gets text instead of JSON, manually set the output format to JSON output in the UI as a sanity check.
Notes
Thread: https://github.com/braintrustdata/braintrust-sdk-javascript/issues/765. Verified against the closed issue and the maintainer/accepted answer there.
Evidence
In https://github.com/braintrustdata/braintrust-sdk-javascript/issues/765, the reporter asked: Issue braintrustdata/braintrust-sdk-javascript#765 (closed, 6 comments): When using the Braintrust CLI to push prompts with a structured response_format configuration, the CLI appears to ignore the json_schema settings and defaults to text output. This causes the invoke function to fail during schema validation because it receives text instead of the expected JSON structure. However, when manually setting the output format to "JSON output" in the Braintrust UI after pushing, the prompts work correctly. ## Environment Braintrust CLI version: 0.0.208 and 0.0.209 (tested both) Node.js version: 22 Package manager*: Bun 1.2. The verified answer: The Braintrust CLI doesn't accept zod schemas directly when pushing prompts with responseformat: a zod schema gets silently ignored and the prompt falls back to text output, failing schema validation on invoke. Fix: convert the zod schema to a JSON schema with zod-to-json-schema first, then push: import { zodToJsonSchema } from 'zod-to-json-schema' and build jsonschema as { name: 'schema', schema: zodToJsonSchema(YourZodSchema) }. If invoke still gets text instead of JSON, manually set the output format to JSON output in the UI as a sanity check. Source thread: https://github.com/braintrustdata/braintrust-sdk-javascript/issues/765
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_6WGfo_88RUVV-35YfjTDYw" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Braintrust CLI ignores response_format when pushing prompts <span>— Vectle</span></a>