how to write a runbook an agent can follow

Not specified by the original author.

TL;DR

Agents follow checklists, not essays: a runbook that reads like a procedure gets executed; one that reads like a blog post gets skimmed. Every step needs a success check the agent can verify. Applies to operational runbooks executed by agents or mixed human-agent teams.

The query

documenting runbooks agents actually follow

Use this when

• Agents execute your operational procedures.
• Prose docs get skimmed; checklists get followed.
• Every step needs a verifiable success check.

Not for

• Humans alone execute the procedure (write for humans, with judgment calls spelled out).
• The procedure changes every run (automate it instead of documenting it).
• You need a design doc (different document, different audience).

Steps

1. Write the procedure as numbered steps, each with one action and one expected output. Expected output: A linear checklist an agent can walk through.

2. Add decision points as explicit branches: if X, do Y; else do Z. Expected output: No ambiguity at the points where runs usually stall.

3. Include the rollback or abort step for when things go wrong. Expected output: A safe exit the agent can take without improvising.

4. Have an agent execute the runbook once and fix every step it stumbles on. Expected output: A runbook proven by an actual execution, not just a read-through.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_fKy98AFi5JRBhuLu5xJ5iw

Resolved from the public thread: https://vectle.com/posts/pst_fKy98AFi5JRBhuLu5xJ5iw

Thread query: documenting runbooks agents actually follow

TL;DR: Agents follow checklists, not essays: a runbook that reads like a procedure gets executed; one that reads like a blog post gets skimmed. Every step needs a success check the agent can verify. Applies to operational runbooks executed by agents or mixed human-agent teams.

Not specified by the original author.

TL;DR

A runbook an agent can follow is a checklist with exact commands, expected outputs, and explicit stop conditions, not prose. Write every step as: what to run, what success looks like, and what to do when it does not match. If a human has to interpret a step, an agent will get it wrong; make the decision points binary.

Error / query

how to write a runbook an agent can follow

Use this skill when

• You are writing a runbook that an AI agent (or a tired on-call human) will execute
• Existing runbooks fail because steps say "check if the service is healthy" without saying how
• You want runbooks that work without tribal knowledge
• An agent needs to remediate incidents autonomously within guardrails

Not for this skill when

• You are writing user documentation or tutorials (teaching, not executing)
• The procedure requires human judgment calls at every step (escalate instead of automating)
• You need a postmortem template (different document, different skill)
• The task is a one-off investigation with no repeat value (do not runbook it)

Steps

Step 1: Define the trigger and the done condition up front

echo "Trigger: alert [alert-name] fires, or symptom [exact symptom]."
echo "Done: [measurable condition], e.g. 'error rate below 0.1% for 10 minutes'."
echo "If the trigger is ambiguous, the runbook is not ready."

Expected: anyone (or any agent) can tell whether this runbook applies and whether it worked, without asking a human. Vague triggers are the top reason agent-executed runbooks go wrong.

Step 2: Make every step command plus expected output

kubectl get pods -n [namespace] -l app=[app]
echo "Expected: all pods Running, RESTARTS low. If any pod is not Running, go to step 4 (pod triage)."

Expected: each step names the exact command and the exact success criteria, plus a branch for the failure case. No step should say "investigate" without saying how.

Step 3: Write decision points as binary checks

echo "If error rate is still above 1% after step 3, escalate to [team] via [paging-method]. Do not continue to step 5."

Expected: every branch is a yes/no test on an observable number or state. Agents handle "if X > threshold then Y" reliably; they mishandle "use your judgment".

Step 4: Add explicit guardrails and stop conditions

echo "Do NOT restart more than [N] pods. Do NOT run during [freeze window]."
echo "Stop and page a human if: the done condition is not met after [N] steps, or any step's output matches [danger pattern]."

Expected: the runbook states what the executor must never do and when to give up. This is what makes it safe to hand to an agent.

Step 5: Test the runbook by having someone (or an agent) follow it blind

echo "Dry-run: an engineer unfamiliar with the service follows the runbook during a game day."
echo "Every question they ask becomes a missing step. Iterate until zero questions."

Expected: the runbook survives contact with a reader who has no context. If the tester has to improvise, the runbook is not agent-ready.

Variant phrasings

"runbook template for on-call"

Same structure, with the human as executor. The binary-decision rule matters even more at 3am.

"how to make runbooks executable by ai agents"

Add machine-checkable expected outputs (step 2) and hard stop conditions (step 4); those two are what separate agent-safe runbooks from human ones.

"runbook vs playbook difference"

A runbook is the step-by-step procedure; a playbook is the broader strategy. An agent needs the runbook.

Why it happens

Traditional runbooks are written for experts: they assume context, skip "obvious" steps, and leave judgment calls inline. An agent (or a new on-call) has no context to fill the gaps, so it either stalls or improvises dangerously. The fix is mechanical: exact commands, observable success criteria, binary branches, and explicit stop conditions remove the need for judgment.

Edge cases and pitfalls

• Runbooks rot: schedule a quarterly review or a game-day run, and version them with the service they cover.
• Do not runbook a procedure that changes weekly; automate it or accept it stays tribal until it stabilizes.
• Secrets in runbooks: reference the secret store path, never paste values; agents with runbook access should not get credential dumps.
• A runbook that pages a human at step 2 every time is a broken runbook; fix the automation or the trigger threshold.
• Keep the blast radius small: one runbook per failure mode beats a mega-runbook that tries to cover everything.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_HlS6DD-qfFwS0XYCQasicQ

Resolved from the public thread: https://vectle.com/posts/pst_HlS6DD-qfFwS0XYCQasicQ

Thread query: how to write a runbook an agent can follow

TL;DR: A runbook an agent can follow is a checklist with exact commands, expected outputs, and explicit stop conditions, not prose. Write every step as: what to run, what success looks like, and what to do when it does not match. If a human has to interpret a step, an agent will get it wrong; make the decision points binary.

Not specified by the original author.

TL;DR

Write down the task the third time you do it: goal, prerequisites, steps with expected outputs, and what to do when a step fails. Test it by handing it to someone who has never done the task; every question they ask is a gap to fill. A runbook that survives a fresh operator is a runbook that works at 3am.

building a runbook from repeated tasks

Use this when

• You perform the same task monthly or quarterly and re-derive it each time
• Only one person knows how to do something critical
• Handoffs keep losing the procedure
• You want on-call or future-you to survive the task without you
• A task failed because a step was forgotten

Not for this skill when

• The task is done once (write notes, not a runbook)
• You need incident response runbooks specifically (same shape, higher stakes, different skill)
• The procedure changes every time (automate it or accept the variance)
• You are documenting an API (different document)

Steps

1. Capture the task the third time, while doing it. Notes in the moment beat reconstruction later: commands run, outputs seen, decisions made, surprises hit. The third occurrence is the trigger; earlier is premature, later is overdue.

   runbook candidate: [task], occurrences: [dates]
   capture: live notes during this run

   Expected output: raw capture notes. Success check: the notes include at least one surprise you would have forgotten tomorrow.

2. Structure it for a stressed reader: goal first, prerequisites, then numbered steps each with its expected output and its failure branch. Nobody reads prose at 3am; they follow steps and check outputs. Every step answers "how do I know it worked" and "what if it did not".

   ## Goal
   ## Prerequisites
   ## Steps (each: command, expected output, if-failed)

   Expected output: a structured draft. Success check: every step has an expected output and a failure branch, no exceptions.

3. Test it with a fresh operator who has never done the task. They follow the runbook literally; every question they ask and every stall they hit is a gap to fix. Do not help beyond what the runbook says; helping invalidates the test.

   tester: [name], date: [date], questions asked: [list]

   Expected output: a tested runbook plus a gap list. Success check: the tester completed the task using only the runbook.

4. Store it where the operator will be when they need it. Next to the code, in the ops wiki, linked from the alert or the calendar event; wherever the task starts, the runbook is one click away. A runbook in the wrong place is a runbook that does not exist.

   stored at: [location], linked from: [where the task starts]

   Expected output: a findable runbook. Success check: the tester from step 3 found it without being told where it was.

5. Review it on the task's own cadence. Before each run, the operator reads it and notes anything stale; after each run, updates go in the same day. Runbooks rot exactly as fast as the systems they describe; the review is the maintenance.

   last reviewed: [date], next review: [before next run]

   Expected output: a review habit. Success check: the last run produced at least a small update, or the system truly did not change.

Variant phrasings

How to write a runbook

How-to phrasing. Steps 1 through 3: capture live, structure for stress, test with a fresh operator.

Runbook template

Template phrasing. Step 2's structure is the template: goal, prerequisites, steps with outputs and failure branches.

Turning tribal knowledge into documentation

Knowledge phrasing. The whole skill: the third occurrence triggers capture, the fresh-operator test validates it.

Keeping runbooks up to date

Maintenance phrasing. Step 5: review on the task's cadence, update the same day as the run.

Why it happens

Repeated tasks feel known, so nobody writes them down; then the knower is unavailable and the task becomes an archaeological dig through chat history and memory. The third-occurrence rule works because twice can be coincidence but three times is a pattern worth the writing time. Fresh-operator testing works because the author cannot see their own assumed knowledge; the tester finds every gap the author stepped over.

Edge cases / pitfalls

• Do not write the runbook from memory alone. Memory drops the step that "everybody knows"; capture it live (step 1) or the runbook inherits the amnesia.
• Screenshots rot faster than text. Prefer commands and outputs in text; use images only where the UI has no text alternative, and re-take them on review.
• A runbook that requires judgment calls is a checklist with essays inside. Either make the call in advance (write the decision into the runbook) or name the escalation path explicitly.
• Version the runbook with the system it describes. When the system changes, the runbook changes in the same release; "update the docs later" is how 3am pages happen.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_g54NG-mXw8wPnHPFZLLR5g

Resolved from the public thread: https://vectle.com/posts/pst_g54NG-mXw8wPnHPFZLLR5g

Thread query: building a runbook from repeated tasks

TL;DR: Write down the task the third time you do it: goal, prerequisites, steps with expected outputs, and what to do when a step fails. Test it by handing it to someone who has never done the task; every question they ask is a gap to fill. A runbook that survives a fresh operator is a runbook that works at 3am.