Braintrust CLI ignores response_format when pushing prompts

TL;DR

This causes the invoke function to fail during schema validation because it receives text instead of the expected JSON structure.

Steps

1. The Braintrust CLI doesn't accept zod schemas directly when pushing prompts with responseformat: a zod schema gets silently ignored and the prompt falls back to text output, failing schema validation on invoke. Fix: convert the zod schema to a JSON schema with zod-to-json-schema first, then push: import { zodToJsonSchema } from 'zod-to-json-schema' and build jsonschema as { name: 'schema', schema: zodToJsonSchema(YourZodSchema) }. If invoke still gets text instead of JSON, manually set the output format to JSON output in the UI as a sanity check.

When to use

You are seeing this: Issue braintrustdata/braintrust-sdk-javascript#765 (closed, 6 comments): When using the Braintrust CLI to push prompts with a structured response\format configuration, the CLI appears to ignore the json\schema settings and defaults to text output. Use this skill when you run into "Braintrust CLI ignores response_format when pushing prompts".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

Versions mentioned in the source: 0.0.208, 0.0.209. If you are on something much newer or older, the details may have shifted.

Why this happens

(closed, 6 comments): When using the Braintrust CLI to push prompts with a structured response\format configuration, the CLI appears to ignore the json\schema settings and defaults to text output.

Sources

• https://github.com/braintrustdata/braintrust-sdk-javascript/issues/765

In https://github.com/braintrustdata/braintrust-sdk-javascript/issues/765, the reporter asked: Issue braintrustdata/braintrust-sdk-javascript#765 (closed, 6 comments): When using the Braintrust CLI to push prompts with a structured response_format configuration, the CLI appears to ignore the json_schema settings and defaults to text output. This causes the invoke function to fail during schema validation because it receives text instead of the expected JSON structure. However, when manually setting the output format to "JSON output" in the Braintrust UI after pushing, the prompts work correctly. ## Environment Braintrust CLI version: 0.0.208 and 0.0.209 (tested both) Node.js version: 22 Package manager*: Bun 1.2. The verified answer: The Braintrust CLI doesn't accept zod schemas directly when pushing prompts with responseformat: a zod schema gets silently ignored and the prompt falls back to text output, failing schema validation on invoke. Fix: convert the zod schema to a JSON schema with zod-to-json-schema first, then push: import { zodToJsonSchema } from 'zod-to-json-schema' and build jsonschema as { name: 'schema', schema: zodToJsonSchema(YourZodSchema) }. If invoke still gets text instead of JSON, manually set the output format to JSON output in the UI as a sanity check. Source thread: https://github.com/braintrustdata/braintrust-sdk-javascript/issues/765