human-in-the-loop patterns for agent-run deploys

TL;DR

Human-in-the-loop for agent deploys works when approvals are specific, contextual, and rare: approve the plan before execution, approve production promotion, and let everything else run. Approvals on every step become rubber stamps; approvals on nothing is just autonomy with extra steps. Design the gates around blast radius, not around anxiety.

The query

human-in-the-loop patterns for agent-run deploys

Use this when

• Agents execute deployments
• Deciding which deploy steps need human approval
• Approval gates have become rubber stamps
• Balancing deploy velocity with safety

Not for when

• Fully autonomous deployment design
• Human-only deploy processes
• Incident response command structures

Steps

Step 1: Approve the plan, not each command

Have the agent present its deploy plan (what will change, in what order, how it rolls back) and get approval on the plan. Then let it execute without per-step prompts. Plan-level approval catches bad strategies; step-level approval just slows good ones. Expected output: a plan document the human actually read, approved with understanding.

Step 2: Gate on blast radius, not on routine

Require approval for production promotion, data migrations, and irreversible actions. Do not require approval for staging deploys, plan generation, or read-only investigation. The gate list should fit on an index card. Expected output: a short, stable list of what needs a human; everything else flows.

Step 3: Give approvers real context

An approval prompt with "deploy now?" and no context gets clicked without thought. Include the diff summary, the blast radius, the rollback plan, and what changed since the last deploy. Approvers who understand the change approve meaningfully. Expected output: approvals backed by context, with rejections that cite specific concerns.

Step 4: Make approval fast and mobile-friendly

If approval takes 20 minutes of VPN and laptop, people batch-approve or skip. Approvals should work from a phone in under a minute for the common case, with escalation when the approver is unavailable. Expected output: median approval latency measured in minutes, not hours.

Step 5: Audit the approvals

Review approval logs: approval rate near 100% with zero rejections means the gate is a rubber stamp and the context is insufficient. Healthy gates reject or question a meaningful fraction of plans. Expected output: a rejection/question rate that proves humans are actually reviewing.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_fph-PcoqrgDyAKzTgoIoAQ