braintrust cli

TL;DR

The Braintrust CLI is called bt and ships as the npm package @braintrust/bt, separate from the braintrust SDK. Install it with npm install -g @braintrust/bt, set your API key through the BRAINTRUSTAPIKEY environment variable, then run evals with bt eval path/to/foo.eval.ts. Most first-run failures are missing auth or running from the wrong directory.

Error

$ bt eval evals/smoke.eval.ts
zsh: command not found: bt

Steps

1. Install the CLI: npm install -g @braintrust/bt. Expected: bt --version prints a version number.
2. Authenticate by setting the BRAINTRUSTAPIKEY environment variable to your Braintrust API key (find it in the Braintrust UI under your account settings). Expected: bt commands stop reporting missing credentials.
3. Run an eval file from your project: bt eval path/to/foo.eval.ts. Expected: the eval executes and prints a link to the experiment in the Braintrust UI.
4. To run without sending logs to Braintrust, add the local flag: bt eval --local path/to/foo.eval.ts. Expected: results stay local, nothing uploads.
5. To push prebuilt logs into a project: bt sync push project_logs:"My Project" --in ./out/logs --no-input. Expected: the CLI reports the rows synced and they appear under that project in the UI.

When to use

• You need to run a Braintrust eval file from the terminal or CI.
• You want to push existing logs into Braintrust without writing SDK code.
• You are scripting eval runs (nightly jobs, CI gates) and need a non-interactive command.

When not to use

• You are defining the eval itself (task function, scorers, dataset); that lives in SDK code, the CLI only runs it.
• You want to query or summarize past experiments; the web UI or MCP server fits better.
• You have not installed the package; npx @braintrust/bt --help works without a global install.

Tool compatibility

• @braintrust/bt from npm, any recent release; verify with bt --version.
• Node.js 18 or newer.
• Works alongside the braintrust Python SDK (pip install braintrust) and TypeScript SDK (npm install braintrust).

Variant phrasings

braintrust command line

The binary is named bt, not braintrust. The braintrust npm package is the SDK; the CLI is the separate @braintrust/bt package.

run braintrust eval from terminal

bt eval takes a path to an eval file (.eval.ts). It is not braintrust eval, and it does not take an eval name; the file defines the eval.

braintrust cli login

There is no interactive login subcommand in current releases; auth is the BRAINTRUSTAPIKEY environment variable. The env var is the reliable path in automation and CI.

Why it happens

Braintrust split the CLI out of the SDK so terminal workflows (CI, scripts, log pushes) do not need project code. That split is also why braintrust eval fails: the SDK package ships no such binary, and most agents guess the command name.

Edge cases

• In CI, prefer npx @braintrust/bt@VERSION pinned to a release over a global install, so runs are reproducible.
• bt eval --local still needs your model provider keys for the task under test; it only skips the Braintrust upload.
• If the CLI and the SDK disagree on the API endpoint, set BRAINTRUSTAPIURL for both; the default points at the public Braintrust API.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstoPMRHep6X9AwzkhEK42cw