canary deploys for agent-initiated changes

TL;DR

Canary deploys are the right default for agent-initiated changes because they bound the blast radius of machine-speed mistakes: roll the change to a small slice of traffic, watch the metrics the agent might not think to check, and promote or roll back automatically on the numbers. The canary watches what the agent cannot: real user impact.

The query

canary deploys for agent-initiated changes

Use this when

• AI agents ship changes to production
• Designing deploy pipelines for agent workflows
• You want automatic rollback for bad agent changes
• Agent changes have caused incidents before

Not for when

• Human-driven canary processes (similar, but the reviewer differs)
• Feature flag rollouts (complementary, not the same)
• Database migrations (need their own safety patterns)

Steps

Step 1: Define canary stages and traffic slices

Start small (1-5% of traffic or one availability zone), then step up (25%, 50%, 100%) with bake time at each stage. The stages should be automatic; the agent proposes the change, the pipeline owns the rollout shape. Expected output: a staged rollout definition the agent cannot skip or reshape.

Step 2: Pick guardrail metrics the agent might miss

Canary analysis needs metrics beyond "no errors": latency percentiles, business metrics (conversion, checkout success), and resource saturation. Agents optimize what they are told; the canary checks what matters to users. Expected output: a metric set that would catch the last three agent-caused incidents.

Step 3: Automate promotion and rollback on the numbers

Set automatic promotion when guardrails hold for the bake period and automatic rollback when they breach. Human approval can sit at the final promotion gate, but the rollback must not wait for a human. Expected output: bad canaries roll back in minutes without anyone being paged first.

Step 4: Keep the agent out of the canary controls

The agent that made the change must not be able to approve its own canary, skip stages, or silence the guardrails. Separation of duties applies to agents exactly as it does to humans. Expected output: canary policy enforced by the pipeline, not by the agent's goodwill.

Step 5: Record canary outcomes as agent feedback

Feed canary results back to the agent: which changes passed cleanly, which rolled back and why. This is how the agent learns what safe changes look like. A canary that only gates, without teaching, wastes half its value. Expected output: a feedback loop where repeated rollback causes improve over time.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstMSTpcdp-ZQaMAR3XSL6fw