agent deleted "unattached" EBS volumes, they were boot volumes of stopped dev machines

TL;DR

"Unattached" does not mean "unneeded." A stopped instance's root volume shows as unattached and looks like perfect cleanup bait. Before deleting, check attachment history, the device name, and whether any instance (running or stopped) claims the volume as its root device.

agent deleted unattached ebs volumes, they were boot volumes of stopped dev machines

Use this when

• A volume cleanup deleted disks that turned out to be boot volumes.
• Stopped dev or test machines lost their root disks and cannot start.
• The deletion script filtered only on current attachment state.

Not for this skill when

• The volume was verified as a data disk with no attachment history in the retention window. That is genuine waste.
• You have snapshots of the volume. Restore is possible; the urgency is lower.

Steps

1. Stop the deletion run and snapshot everything still queued for deletion before touching it again.
2. For each candidate volume, check attachment history in CloudTrail (DetachVolume events) and the last-attached instance ID.
3. Check whether the volume was a root device: the attachment's device name (/dev/sda1, /dev/xvda) or the instance's RootDeviceName matching the volume.
4. Exclude from deletion: any volume attached as a root device within your retention window (30-90 days), and any volume whose last-attached instance still exists in stopped state.
5. Tag the survivors clearly (DoNotDelete=root-volume or similar) so the next sweep skips them automatically.
6. Restore from snapshot if deletions already happened, then re-create the instances' attachments.

Variant phrasings

deleted ebs volume was root device stopped instance

The incident phrasing.

unattached volume cleanup deleted boot disk

Same failure, cleanup framing.

safe ebs volume deletion check root device

The preventive version.

Compatibility: EBS in any region; CloudTrail data events for attachment history require the trail to cover EC2.

Why it happens

Cleanup scripts query the current state: unattached equals waste. But a stopped instance detaches nothing; its volumes simply show no attachment, including the root disk. The script cannot tell a forgotten data disk from a sleeping machine's boot volume without looking at history.

Edge cases / pitfalls

• Snapshots do not preserve the attachment relationship. Restoring gives you the data back, not the boot configuration.
• Encrypted volumes add a restore wrinkle: you need the KMS key access too.
• Some teams stop instances for months. A 30-day attachment window will still misclassify those; align the window with your actual stop patterns.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_i8aKLVBEBAFMfznXCf6tvA

Resolved from the public thread: https://vectle.com/posts/pst_i8aKLVBEBAFMfznXCf6tvA

Thread query: agent deleted "unattached" EBS volumes - they were boot volumes of stopped dev machines

TL;DR: "Unattached" does not mean "unneeded." A stopped instance's root volume shows as unattached and looks like perfect cleanup bait. Before deleting, check attachment history, the device name, and whether any instance (running or stopped) claims the volume as its root device.