cost agent deregistered the wrong AMIs - its age filter ignored the "do-not-delete" tag the platform team uses

TL;DR: Check AMI tags before the age filter runs, not after. Add a tag guard that skips any AMI carrying the do-not-delete tag (exact key match), and also skip AMIs referenced by launch templates. The age filter only ever sees what is left, so protected images can never be picked.

Deregistered ami-0abc123def456789 (age 214 days exceeds 180-day policy; tags not evaluated)

1. Find out what the agent touched: search CloudTrail for DeregisterImage events from the agent's role. Expected: a list of AMI ids the agent deregistered, with timestamps.
2. Recover what you can: if the underlying snapshot still exists, re-register it with aws ec2 register-image using the original name, architecture, and block device mapping. Expected: a working AMI under a NEW id. The old AMI id is gone permanently, so update any references.
3. Add the tag guard to the cleanup: call aws ec2 describe-images --owners self, then drop every image whose tag set contains the do-not-delete key. Normalize the key to lowercase before comparing so casing drift does not slip through. Expected: protected AMIs never appear in the candidate list.
4. Also exclude AMIs referenced by launch templates: check aws ec2 describe-launch-template-versions for the image id before deregistering. Expected: no deregistration breaks an autoscaling group or launch template.
5. Delete snapshots only after deregistration, and only when no remaining AMI references them. Expected: no 'snapshot in use' errors and no orphaned-but-needed deletions. Then dry-run the agent in report-only mode. Expected: protected AMIs show up under 'skipped', never under 'delete'.

Use this when

• AMI cleanup by age deleted images the platform team marked protected
• Your org uses tag-based deletion protection and the agent does not honor it
• You want age-based AMI pruning that respects exclusion tags
• A deregistered AMI broke a launch template or pipeline

Not for this skill when

• The AMI genuinely has no tags and no consumers (safe to prune)
• The cost problem is snapshot storage, not AMI sprawl (different cleanup, same tag guard idea)
• You are looking at AWS-provided public AMIs (you do not own them, never deregister them)
• The AMI was shared with another account that still launches from it (deregistering breaks their launches, coordinate first)

Variant phrasings

• agent deleted my golden AMI
• AMI cleanup deleted protected image
• deregister old AMIs but keep tagged ones
• cost agent removed AMI used by launch template

Why it happens

Age is easy to compute and tags require an extra API call plus exact-key matching, so agents optimize for the simple filter and skip the tag check. Some agents match tags by substring and miss the platform team's exact key. There is a second trap: deregistering an AMI does not delete its snapshots, so agents that clean snapshots in a separate pass can break the chain for AMIs they did not even touch.

Edge cases

• Tag key casing: do-not-delete vs DoNotDelete vs DONOTDELETE. Normalize to lowercase on both sides.
• AMIs shared with other accounts: you can deregister an AMI others depend on. Check sharing before deleting.
• AWS Backup recovery points are separate from AMIs: cleaning AMIs does not clean backup vaults and vice versa.
• The snapshot may be the base of an active AMI chain: deleting an old snapshot that a newer AMI references corrupts the chain. Verify no references first.
• Some teams use a tag VALUE convention instead of a key (e.g. retention=keep). Confirm which convention your platform team uses before coding the guard.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_UI3UGlctUm54eD4yHHUU-w