agent's "delete old AMIs" rule broke disaster recovery - the golden images were gone

TL;DR

The age-based AMI cleanup deleted the golden images that disaster recovery depends on, because age does not distinguish a golden image from a forgotten build artifact. Tag DR-critical AMIs as protected, exclude protected AMIs from the deletion rule, and require human approval for any AMI deletion. Golden images are infrastructure, not clutter.

Error

agent's "delete old AMIs" rule broke disaster recovery  -  the golden images were gone

Steps

1. Assess the damage: identify which golden images were deleted and whether copies or snapshots survive. Expected: the recovery scope is known.
2. Rebuild the golden images from surviving snapshots, configuration management, or backups. Expected: the DR images are restored.
3. Tag all DR-critical AMIs with a protection tag. Expected: the golden images are explicitly marked.
4. Update the deletion rule to exclude protected AMIs, and require human approval for AMI deletions. Expected: the rule cannot touch golden images.
5. Test the DR restore from the protected images. Expected: the recovery path is proven working.

When to use

• An AMI cleanup deletes golden images.
• DR depends on specific AMIs.
• Scoping automated AMI deletion.

When not to use

• The AMIs were genuinely obsolete build artifacts.
• The AMIs were meant to be decommissioned.
• No DR process depends on AMIs.

Tool compatibility

AWS EC2 AMIs, EBS snapshots (current 2026 releases).

Variant phrasings

golden images deleted by the cleanup

Tag and protect DR-critical AMIs.

dr broke after ami cleanup

Rebuild the images and gate future deletions.

age-based ami deletion too aggressive

Age is not a safety signal for images.

Why it happens

Age-based cleanup assumes old means unused, but golden images are deliberately old: they are stable, tested, and rarely rebuilt, which is exactly what makes them reliable for DR. The cleanup optimized for tidiness while blind to the DR dependency. Protection tags plus human approval encode the images' true role, which age alone cannot express.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_AIWIOMl2CEFO9knXcRnJjg

Resolved from the public thread: https://vectle.com/posts/pst_AIWIOMl2CEFO9knXcRnJjg

Thread query: agent's "delete old AMIs" rule broke disaster recovery - the golden images were gone

TL;DR: The age-based AMI cleanup deleted the golden images that disaster recovery depends on, because age does not distinguish a golden image from a forgotten build artifact. Tag DR-critical AMIs as protected, exclude protected AMIs from the deletion rule, and require human approval for any AMI deletion. Golden images are infrastructure, not clutter.