Buildkite dynamic pipeline: a failed upload silently continues the build

TL;DR

Add set -euo pipefail to the top of every pipeline generator script so a failed pipeline upload fails the build step. When debugging unexpected steps, download the captured artifact instead of re-running the generator locally and guessing.

Steps

1. Add set -euo pipefail to the top of every pipeline generator script so a failed pipeline upload fails the build step. For auditability, save the generated YAML to a file, validate it with pipeline upload --dry-run, upload it as a build artifact, then do the real upload. When debugging unexpected steps, download the captured artifact instead of re-running the generator locally and guessing.

When to use

You are seeing this: For auditability, save the generated YAML to a file, validate it with pipeline upload --dry-run, upload it as a build artifact, then do the real upload. Use this skill when you run into "Buildkite dynamic pipeline: a failed upload silently continues the build".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Source: Official Buildkite dynamic pipelines docs: in a dynamic pipeline, a rejected pipeline upload (invalid YAML, validation error) or transient failure happens mid-build while it is already running. pipeline upload exits nonzero, but Bash does not exit on errors by default, so the step succeeds and the build continues with no record that steps were expected.

Verified resolution: Add set -euo pipefail to the top of every pipeline generator script so a failed pipeline upload fails the build step. For auditability, save the generated YAML to a file, validate it with pipeline upload --dry-run, upload it as a build artifact, then do the real upload. When debugging unexpected steps, download the captured artifact instead of re-running the generator locally and guessing.

Source URL: https://github.com/buildkite/docs/blob/HEAD/pages/pipelines/configure/dynamic_pipelines.md