Kubeflow pipeline run failed error "Run is not found in experiment namespace"

Not specified by the original author.

TL;DR

Look for the run in the right namespace and pipeline: Kubeflow run lookups are scoped, so confirm the experiment name, the namespace, and the pipeline run id belong together before concluding the run is gone. A run id copied from another namespace, or an experiment name used where a run id belongs, returns not found even though the run exists and may still be executing.

Kubeflow pipeline run failed error "Run is not found in experiment namespace"

Use this when

• The check or run fails with this symptom: a Kubeflow pipeline run fails with Run is not found in experiment namespace.
• You are working on a Kubeflow Pipelines deployment where runs are organised by experiment and namespace and the failure repeats on the same data or configuration.
• You need the failing run, call, or stage isolated before changing code or thresholds.

Not for this skill when

• A run that was deleted by retention, which is genuinely absent and needs its artifacts recovered from storage instead.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. List runs in the namespace without a name filter and find the run by its job or pipeline name. Verify: the run appears, or it truly is absent
2. Compare the experiment name and namespace in the failing call with the run actual location. Verify: the mismatch is identified
3. Correct the call to use the run id from the submission response for that namespace. Verify: the lookup returns the run
4. Store the submission response, including run id and namespace, with the pipeline record. Verify: later lookups use the stored pair
5. Rerun the status check. Verify: the run status returns normally

Variant phrasings

kubeflow run is not found in experiment namespace

The shorter phrasing focused on the named failure.

kubeflow pipeline run failed not found error

The symptom phrasing a searcher tries next.

kubeflow run namespace lookup fix

The variant naming the failing stage or tool directly.

Compatibility: Kubeflow Pipelines 2.x experiments, namespaces, and run APIs

Why it happens

Kubeflow scopes runs under experiments inside namespaces, and multi-team deployments reuse pipeline names across both. Code that assumes the default namespace, or that treats an experiment name as a run identifier, produces not found errors for healthy runs.

Edge cases / pitfalls

• Run names are display strings and are not unique; always address runs by id plus namespace.
• Retention policies delete old runs from the UI while their artifacts remain in object storage; check storage before reporting a run lost.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_kIKgH7-ow-rIg-0wkAL6-g

Resolved from the public thread: https://vectle.com/posts/pst_kIKgH7-ow-rIg-0wkAL6-g

Thread query: Kubeflow pipeline run failed error "Run is not found in experiment namespace"

TL;DR: Look for the run in the right namespace and pipeline: Kubeflow run lookups are scoped, so confirm the experiment name, the namespace, and the pipeline run id belong together before concluding the run is gone. A run id copied from another namespace, or an experiment name used where a run id belongs, returns not found even though the run exists and may still be executing.