Vertex AI experiment failed error "FailedPrecondition: experiment not in running state"

TL;DR

Start or resume the experiment before adding runs to it: Vertex AI rejected the operation because the experiment is not in a running state, so check the experiment state, resume it if it was stopped, and log the run against the resumed experiment. Vertex AI experiments have a lifecycle state. A stopped, completed, or not yet started experiment refuses new run records, and the client error reports the state rule, not a problem with the run data.

Vertex AI experiment failed error "FailedPrecondition: experiment not in running state"

Use this when

• The check or run fails with this symptom: a Vertex AI experiment fails with FailedPrecondition experiment not in running state.
• You are working on a Vertex AI experiment receiving run records from a pipeline or a notebook and the failure repeats on the same data or configuration.
• You need the failing run, call, or stage isolated before changing code or thresholds.

Not for this skill when

• An experiment that was archived deliberately at project end, where a new experiment is the correct target.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Fetch the experiment and read its state. Verify: the current state is confirmed from the service, not assumed
2. If the experiment was paused or stopped by a cleanup action, resume it from the console or the API. Verify: the state shows as running
3. If it was completed deliberately, create a successor experiment and note the lineage. Verify: new runs have a valid home
4. Retry logging the run. Verify: the run record is accepted under the experiment
5. Check for automation that stops idle experiments and whitelist this one if it receives scheduled runs. Verify: the state stays running across the schedule

Variant phrasings

vertex ai FailedPrecondition experiment not in running state

The shorter phrasing focused on the named failure.

vertex experiment failed not running error

The symptom phrasing a searcher tries next.

vertex ai experiment resume fix

The variant naming the failing stage or tool directly.

Compatibility: Vertex AI Experiments, run logging from pipelines and notebooks

Why it happens

Platform lifecycle automation and manual cleanup both stop experiments that look idle. Logging code rarely checks state first, so the first run after a stop fails with a precondition error that points at the experiment, not at the run payload.

Edge cases / pitfalls

• Resuming an experiment someone completed for audit reasons reopens a closed record; confirm intent before resuming.
• State checks belong in the logging wrapper for long lived experiments, with a clear message when the experiment is stopped.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_VIUqjozIniaACW6EWzMUMA

Resolved from the public thread: https://vectle.com/posts/pst_VIUqjozIniaACW6EWzMUMA

Thread query: Vertex AI experiment failed error "FailedPrecondition: experiment not in running state"

TL;DR: Start or resume the experiment before adding runs to it: Vertex AI rejected the operation because the experiment is not in a running state, so check the experiment state, resume it if it was stopped, and log the run against the resumed experiment. Vertex AI experiments have a lifecycle state. A stopped, completed, or not yet started experiment refuses new run records, and the client error reports the state rule, not a problem with the run data.