kill switches for runaway agents

kill switches for runaway agents

TL;DR

Define what "runaway" means before it happens, then build two things: a big red button a human can hit, and automatic circuit breakers that trip on their own. A runaway agent burns money, spams APIs, or corrupts data by the minute, so the kill switch has to work in seconds, not tickets.

kill switches for runaway agents

Use this when

• An agent is stuck in a loop, burning API budget or hammering a service
• You are launching an agent with broad permissions for the first time
• Leadership asks what stops the AI if it goes wrong
• You are writing the runbook for agent operations

Not for this skill when

• The agent is compromised by an attacker (that is incident response: isolate and investigate)
• A single bad action already happened (that is rollback and review)
• You want to prevent bad actions entirely (use plan review and tool-level policy)

Steps

1. Define "runaway" in numbers.

Pick thresholds: tool calls per run, cost per run, actions per minute, consecutive errors, repeat identical actions. "It looks wrong" is not a trigger a machine can use; numbers are.

guardrails:
  max_tool_calls_per_run: 200
  max_cost_per_run_usd: 25
  max_identical_actions_in_a_row: 5
  on_breach: halt_and_alert

Expected: a config where every runaway pattern you fear maps to a number that trips.

2. Build the manual kill switch.

One command, known to everyone on the team, that halts all of an agent's sessions and revokes its credentials immediately. It must not require the agent's cooperation, a deploy, or three approvals.

agent kill --all --reason "runaway loop detected"

Expected: all sessions halted within seconds, credentials revoked, and a confirmation listing what was stopped.

3. Add automatic circuit breakers.

The manual button needs a human awake. Breakers trip on their own: cost caps, rate limits, error-loop detection, and anomaly alerts that halt first and ask questions later. Automatic halt, manual resume.

Expected: a run that blows past the cost cap stops by itself and pages the owner.

4. Make resume deliberate, not automatic.

After a halt, the agent stays down until a human reviews what tripped the breaker and re-enables it. Auto-resume turns your circuit breaker into a circuit slower-downer.

Expected: re-enabling requires an explicit command plus a note about what was found.

5. Practice the drill.

Run a game day: simulate a runaway in staging and time how long it takes someone to notice, hit the switch, and confirm the halt. Fix whatever was slow. An untested kill switch is a hope.

Expected: a drill log with detection time, kill time, and confirmation time, all in minutes or less.

6. Review every trip before closing it out.

Each halt gets a short postmortem: what tripped it, was it a real runaway or a bad threshold, what changed. Tune thresholds from evidence, not from annoyance.

Expected: a one-paragraph write-up per trip and threshold changes where the data supports them.

Variant: stop an AI agent loop

Loops are the most common runaway. Detect them with the identical-actions counter and a max-steps cap, and make sure the halt actually kills the underlying session rather than just the chat window.

Variant: emergency stop for agents

That is step 2: one command, no dependencies, known to the whole team, tested in a drill. Put it in the runbook next to the database failover procedure, not buried in a wiki.

Variant: agent spending too much how to halt

Cost caps per run and per day, enforced at the billing layer if your provider supports it. Alert at 50 percent, halt at 100. Money is the runaway metric executives understand fastest.

Variant: circuit breaker for agent actions

Same as step 3. The key design choice: breakers halt and alert rather than just alert. An alert nobody acts on for an hour is how a loop spends the budget.

Why this happens

Agents do not get tired, bored, or embarrassed. A human stuck in a loop stops after the third identical failure; an agent will happily retry a thousand times, each one billable, each one hammering the same API. The kill switch exists because persistence is the agent's best trait right up until it is the worst one.

Edge cases and pitfalls

• The kill switch needs the agent platform to work: if the platform itself is down, have a provider-level kill: revoke the API key at the vendor, which stops everything regardless of your tooling.
• Thresholds set from guesswork: start from observed normal runs (p99 of cost and steps), then set caps at a multiple. Too tight and every long task trips; too loose and the breaker never fires.
• Partial halt: killing the chat session but leaving scheduled jobs or webhooks running. The kill switch must cover every way the agent can act, including cron-like triggers.
• Someone re-enables without review: require the postmortem note as part of the resume command. Process beats discipline.
• Breakers that only alert: audit that every breaker ends in a halt. If any path is alert-only, it is a monitoring gap wearing a circuit-breaker costume.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_SCkSXQmAJmXYg-gN69F5xA