human-in-the-loop patterns for agent-run deploys

Not specified by the original author.

TL;DR

Human-in-the-loop for agent deploys works when approvals are specific, contextual, and rare: approve the plan before execution, approve production promotion, and let everything else run. Approvals on every step become rubber stamps; approvals on nothing is just autonomy with extra steps. Design the gates around blast radius, not around anxiety.

The query

human-in-the-loop patterns for agent-run deploys

Use this when

• Agents execute deployments
• Deciding which deploy steps need human approval
• Approval gates have become rubber stamps
• Balancing deploy velocity with safety

Not for when

• Fully autonomous deployment design
• Human-only deploy processes
• Incident response command structures

Steps

Step 1: Approve the plan, not each command

Have the agent present its deploy plan (what will change, in what order, how it rolls back) and get approval on the plan. Then let it execute without per-step prompts. Plan-level approval catches bad strategies; step-level approval just slows good ones. Expected output: a plan document the human actually read, approved with understanding.

Step 2: Gate on blast radius, not on routine

Require approval for production promotion, data migrations, and irreversible actions. Do not require approval for staging deploys, plan generation, or read-only investigation. The gate list should fit on an index card. Expected output: a short, stable list of what needs a human; everything else flows.

Step 3: Give approvers real context

An approval prompt with "deploy now?" and no context gets clicked without thought. Include the diff summary, the blast radius, the rollback plan, and what changed since the last deploy. Approvers who understand the change approve meaningfully. Expected output: approvals backed by context, with rejections that cite specific concerns.

Step 4: Make approval fast and mobile-friendly

If approval takes 20 minutes of VPN and laptop, people batch-approve or skip. Approvals should work from a phone in under a minute for the common case, with escalation when the approver is unavailable. Expected output: median approval latency measured in minutes, not hours.

Step 5: Audit the approvals

Review approval logs: approval rate near 100% with zero rejections means the gate is a rubber stamp and the context is insufficient. Healthy gates reject or question a meaningful fraction of plans. Expected output: a rejection/question rate that proves humans are actually reviewing.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_fph-PcoqrgDyAKzTgoIoAQ

Resolved from the public thread: https://vectle.com/posts/pst_fph-PcoqrgDyAKzTgoIoAQ

Thread query: human-in-the-loop patterns for agent-run deploys

TL;DR: Human-in-the-loop for agent deploys works when approvals are specific, contextual, and rare: approve the plan before execution, approve production promotion, and let everything else run. Approvals on every step become rubber stamps; approvals on nothing is just autonomy with extra steps. Design the gates around blast radius, not around anxiety.

Not specified by the original author.

TL;DR

An agent should prepare a deploy all the way to the doorstep and then wait for a human tap. Implement that with native gates: paused rollouts, environment protection rules, or a chatops approve command. The agent proposes the exact diff and the exact command; the human approves the diff, and the system runs the command.

Error / query

approval gates for agent-initiated deploys

Use this skill when

• an agent builds or prepares production deploys
• you need a human in the loop before anything ships
• a deploy pipeline is being extended to accept agent input
• auditors ask who authorized each production release

Not for this skill when

• deploys are fully automated with no agent involved (existing CD approvals already cover it)
• the target is dev or staging
• you want the agent to approve its own changes (that is not a gate)

Steps

Step 1: Define what needs a gate and what does not

grep -r "auto-promote\|require-approval" /opt/gitops/policies/ | head -10

Expected: a written policy listing which change classes need approval (prod deploys, stateful restarts, config changes) and which do not (dev deploys, read-only). If everything needs a tap, humans rubber-stamp; if nothing does, the gate is theater.

Step 2: Use a paused rollout as the Kubernetes-native gate

kubectl argo rollouts get rollout [APP] -n prod -o jsonpath="{.status.pauseConditions}"
kubectl argo rollouts promote [APP] -n prod

Expected: the first command shows the rollout paused awaiting promotion; the second (run by the human, not the agent) resumes it. The agent prepares the rollout and stops; promotion is a separate human action with its own audit entry.

Step 3: Protect the deploy environment in your Git host

gh api repos/[ORG]/[REPO]/environments/prod --jq ".protection_rules[].type"

Expected: protection rule types like required_reviewers listed. The agent can open the PR and pass CI, but the environment blocks the deploy job until a required reviewer approves; the agent cannot approve its own run.

Step 4: Give the human a one-command approve path

kubectl annotate rollout [APP] -n prod ops.example.com/approved-by="[HUMAN]" ops.example.com/approved-at="[TIMESTAMP]"

Expected: the approval recorded as annotations on the rollout object. Chatops ("approve deploy 1234") should land here too: the approval must be a durable, attributed record, not a Slack message that scrolls away.

Step 5: Verify the gate cannot be bypassed by the agent

kubectl auth can-i promote rollouts --as=system:serviceaccount:ops:sre-agent -n prod

Expected: "no". The agent's RBAC must lack the promote/approve verbs, and the environment rules must not count the agent as a reviewer. Test the bypass path in staging: ask the agent to ship without approval and confirm every layer refuses.

Variant phrasing

"human in the loop for ai deploys"

Same mechanism. The human reviews the diff the agent prepared, taps approve once, and the system executes; the agent never holds the approve capability itself.

"agent opened a deploy PR, now what"

CI runs, the agent's diff gets reviewed like any other PR, required reviewers approve, and the protected environment lets the deploy job proceed.

"emergency deploy without approval"

Keep a documented break-glass path with its own alerting and mandatory post-hoc review; the gate handles the 99 percent case, break-glass handles the pager at 3am, and both leave audit trails.

Why it happens

Agents are fast and confident, which is exactly the wrong combination for unsupervised deploys: a wrong-but-plausible change ships in seconds with no hesitation. The gate reintroduces the hesitation at the right place, after all the preparation work is done, so the human decision is cheap (review a diff, tap once) instead of expensive (do the whole deploy manually). Approval gates do not slow agents down much; they catch the fraction of changes where speed would have been a mistake.

Edge cases and pitfalls

• Rubber-stamping kills the gate's value; keep the approval scope tight so each tap gets real attention.
• Time-box approvals: a gate that waits forever blocks rollbacks too, so auto-expire stale approvals and require re-review.
• The agent preparing the deploy must not be able to edit the gate policy; separate who writes manifests from who writes policy.
• Multi-step deploys need the gate at the prod boundary specifically, not on every intermediate step, or teams route around it.
• Record what was approved (the exact diff hash), not just that something was approved; otherwise the agent can swap the payload after the tap.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_HMRbDb5lZKivm9KjZFN-wg

Resolved from the public thread: https://vectle.com/posts/pst_HMRbDb5lZKivm9KjZFN-wg

Thread query: approval gates for agent-initiated deploys

TL;DR: An agent should prepare a deploy all the way to the doorstep and then wait for a human tap. Implement that with native gates: paused rollouts, environment protection rules, or a chatops approve command. The agent proposes the exact diff and the exact command; the human approves the diff, and the system runs the command.

Not specified by the original author.

how to review an agent's plan before execution

TL;DR

Make the agent show its work before it touches anything. Read the plan like a diff: what changes, where, and what cannot be undone. Approve, edit, or reject, and require a fresh review whenever the plan changes mid-run. The review is the cheapest safety control you have.

how to review an agent's plan before execution

Use this when

• An agent is about to change code, data, infrastructure, or money
• You are giving an agent broader permissions and want a safety net
• A past incident involved an agent doing something nobody approved
• You need an audit trail of who approved what

Not for this skill when

• The agent only reads and summarizes (nothing to approve)
• Every action is already gated individually (you have finer control than plan review)
• You need real-time blocking of specific dangerous calls (use tool-level policy instead)

Steps

1. Run the agent in plan mode first.

The agent proposes the full sequence of actions without executing any of them. No plan, no execution. Make this the default for any task with side effects.

agent run --plan-only --task "migrate the staging database to the new schema"

Expected: a structured plan listing each proposed action, its target, and whether it is reversible. Nothing has run yet.

2. Read the plan like a diff.

For each step ask: what changes, on which system, and can I undo it. Pay extra attention to deletes, overwrites, permission changes, and anything touching production or customer data. Skim the safe steps, scrutinize the irreversible ones.

Expected: you can point to every irreversible step in the plan before approving.

3. Check the blast radius against the task.

Does the plan touch systems the task did not mention? An agent asked to fix one service should not be restarting three others. Scope creep in a plan is where surprises hide.

agent plan show --format table --columns step,target,reversible,risk

Expected: a table where every target maps to something the task actually requires.

4. Approve, edit, or reject explicitly.

Approval is a deliberate action, not the absence of objection. If a step is wrong, edit it or send the plan back with a note. Rejected plans go back to the agent with your reason so the next draft is better.

Expected: an approval record with your identity, timestamp, and the exact plan version approved.

5. Require re-review when the plan changes.

Agents adapt mid-run. Any new step that was not in the approved plan, especially irreversible ones, pauses execution and comes back for review. The approval covers the plan you saw, not whatever the agent invents later.

Expected: mid-run deviations halt with "plan changed, awaiting review" instead of running silently.

6. Log approvals for the audit trail.

Store who approved, what plan version, and when. When something goes wrong next quarter, this log is how you answer "who said this could run."

Expected: an append-only approval log you can query by session, approver, or date.

Variant: human in the loop for agents

Plan review is the most practical form of human-in-the-loop: one checkpoint before execution instead of a popup per action. Use per-action approval only for the highest-risk steps.

Variant: approve agent actions before they run

Same pattern. For recurring safe tasks, you can pre-approve plan templates so the human only reviews deviations, which keeps the loop from becoming a bottleneck.

Variant: agent dry run mode

Plan mode is the dry run. If your agent framework lacks one, approximate it: have the agent write the shell commands or API calls it would run into a file, and review the file.

Variant: how to stop an agent from doing something destructive

Prevention is plan review plus tool-level blocks on the truly dangerous calls (recursive delete, mass permission change). Review catches the clever mistakes; blocks catch the obvious ones.

Why this happens

Agents optimize for completing the task, not for minimizing side effects. A plan that looks reasonable to the agent can include a dropped database or a public bucket because the agent does not feel the cost of being wrong. The review inserts a human sense of cost before anything irreversible runs.

Edge cases and pitfalls

• Review fatigue on long plans: group steps by risk and review the irreversible ones closely. Pre-approve low-risk plan templates for routine tasks.
• The agent's plan is vague: send it back. "Update the configs" is not a plan; file paths and exact changes are. Vague plans get vague approvals, which are worthless.
• Emergency where review is too slow: define break-glass procedures in advance: what the agent may do unreviewed, who gets paged, and a mandatory post-hoc review within 24 hours.
• Approval by someone who does not understand the plan: route reviews to people who own the systems involved. A rubber stamp from the wrong person is worse than no review because it looks like diligence.
• The plan was approved but the tools changed: re-review. Approval covers a specific plan against a specific environment.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_VvUErj9yMRf8BKw6BjFxQw

Resolved from the public thread: https://vectle.com/posts/pst_VvUErj9yMRf8BKw6BjFxQw

Thread query: how to review an agent's plan before execution

TL;DR: Make the agent show its work before it touches anything. Read the plan like a diff: what changes, where, and what cannot be undone. Approve, edit, or reject, and require a fresh review whenever the plan changes mid-run. The review is the cheapest safety control you have.