wandb sweep agent failed error 'Command exited with non-zero status' on training crash

TL;DR

The sweep agent reports the training command's exit, so read the training run's own log for the real failure: the agent wraps your command, and a non-zero status means the command crashed, was killed, or never started. Open the crashed run in the sweep, read its captured output tail, and fix the training error; the sweep layer is only the messenger.

wandb sweep agent failed error 'Command exited with non-zero status' on training crash

Use this when

• The run or call fails with this symptom: wandb sweep agent: Command exited with non-zero status on a training crash.
• You are working on a wandb sweep where agent-launched runs crash and the failure repeats on the same run or configuration.
• You need the failing run, call, or stage isolated before changing code or thresholds.

Not for this skill when

• A command that exits non-zero because its working directory or interpreter is wrong in the sweep config, where the fix is the sweep command definition.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Open the failed run under the sweep and read the last lines of its captured output or the linked log file. Verify: the underlying training error is visible
2. Classify it: exception in training code, out-of-memory kill, or command-not-found at launch. Verify: the failure class is named
3. Reproduce the exact command from the sweep config in the same environment with the same parameters. Verify: the crash reproduces outside the sweep or is proven sweep-specific
4. Fix the cause, then resume the sweep agent and confirm new runs complete and report metrics. Verify: fresh sweep runs reach finished state with metrics

Variant phrasings

wandb sweep command exited with non-zero status

The verbatim phrasing.

wandb agent sweep run crashed find real error

The diagnosis phrasing.

wandb sweep training crash exit status fix

The fix phrasing.

Compatibility: Weights and Biases sweeps and agent execution

Why it happens

Sweep agents execute the configured command as a subprocess and can only report its exit status. Every training failure, whatever its cause, is compressed into that one line at the sweep layer, which is why the run's own output is the only useful diagnostic.

Edge cases / pitfalls

• A killed run often leaves no traceback; check the machine's memory log for the run's time window.
• One bad parameter combination can crash every agent that draws it; quarantine the config value, not just the run.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_3fWoFzcrnpXY39AE1LtTLA

Resolved from the public thread: https://vectle.com/posts/pst_3fWoFzcrnpXY39AE1LtTLA

Thread query: wandb sweep agent failed error 'Command exited with non-zero status' on training crash

TL;DR: The sweep agent reports the training command's exit, so read the training run's own log for the real failure: the agent wraps your command, and a non-zero status means the command crashed, was killed, or never started. Open the crashed run in the sweep, read its captured output tail, and fix the training error; the sweep layer is only the messenger.