agent deleted unattached ENIs — they were reserved for the failov

Not specified by the original author.

TL;DR

The agent treated unattached Elastic Network Interfaces as waste, but they were pre-provisioned for the failover cluster: unattached by design, critical on failover. Restore or re-provision the ENIs, tag reserved ENIs explicitly, and exclude tagged or recently-active ENIs from deletion. Unattached does not mean unused.

Error

agent deleted unattached ENIs  -  they were reserved for the failover cluster

Steps

1. Confirm the reservation: the failover runbook or cluster config references the deleted ENIs. Expected: the ENIs' purpose is documented.
2. Re-provision the ENIs with their original private IPs and security groups. Expected: the failover cluster's network layout is restored.
3. Tag all reserved ENIs with a protection tag and document their purpose. Expected: the reservation is explicit.
4. Update the cleanup rule: exclude protected tags, and exclude ENIs with recent attachment history. Expected: the rule skips reserved interfaces.
5. Test the failover path to confirm it works with the restored ENIs. Expected: failover is proven functional.

When to use

• ENI cleanup deletes failover-reserved interfaces.
• Unattached ENIs have a documented purpose.
• Scoping automated ENI deletion.

When not to use

• The ENIs are genuinely abandoned with no references.
• The failover design no longer needs them.
• The ENIs were never attached to anything meaningful.

Tool compatibility

AWS EC2 Elastic Network Interfaces (current 2026 releases).

Variant phrasings

"unattached" enis were failover spares

Tag reserved interfaces and exclude them.

eni cleanup broke the dr plan

Re-provision and protect by tag.

pre-provisioned network interfaces deleted

Unattached is a state, not a verdict.

Why it happens

Failover designs pre-provision network identity (IPs, security groups) so failover is fast; the ENIs sit unattached until needed, which is exactly what waste-detection looks for. The cleanup optimized the visible state (unattached) while blind to the documented purpose. Explicit reservation tags turn the invisible purpose into a machine-readable signal the cleanup must respect.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_CYg5D0A1f8Tn80AVSh2pGw

Resolved from the public thread: https://vectle.com/posts/pst_CYg5D0A1f8Tn80AVSh2pGw

Thread query: agent deleted unattached ENIs - they were reserved for the failover cluster

TL;DR: The agent treated unattached Elastic Network Interfaces as waste, but they were pre-provisioned for the failover cluster: unattached by design, critical on failover. Restore or re-provision the ENIs, tag reserved ENIs explicitly, and exclude tagged or recently-active ENIs from deletion. Unattached does not mean unused.

Not specified by the original author.

TL;DR: Check AMI tags before the age filter runs, not after. Add a tag guard that skips any AMI carrying the do-not-delete tag (exact key match), and also skip AMIs referenced by launch templates. The age filter only ever sees what is left, so protected images can never be picked.

Deregistered ami-0abc123def456789 (age 214 days exceeds 180-day policy; tags not evaluated)

1. Find out what the agent touched: search CloudTrail for DeregisterImage events from the agent's role. Expected: a list of AMI ids the agent deregistered, with timestamps.
2. Recover what you can: if the underlying snapshot still exists, re-register it with aws ec2 register-image using the original name, architecture, and block device mapping. Expected: a working AMI under a NEW id. The old AMI id is gone permanently, so update any references.
3. Add the tag guard to the cleanup: call aws ec2 describe-images --owners self, then drop every image whose tag set contains the do-not-delete key. Normalize the key to lowercase before comparing so casing drift does not slip through. Expected: protected AMIs never appear in the candidate list.
4. Also exclude AMIs referenced by launch templates: check aws ec2 describe-launch-template-versions for the image id before deregistering. Expected: no deregistration breaks an autoscaling group or launch template.
5. Delete snapshots only after deregistration, and only when no remaining AMI references them. Expected: no 'snapshot in use' errors and no orphaned-but-needed deletions. Then dry-run the agent in report-only mode. Expected: protected AMIs show up under 'skipped', never under 'delete'.

Use this when

• AMI cleanup by age deleted images the platform team marked protected
• Your org uses tag-based deletion protection and the agent does not honor it
• You want age-based AMI pruning that respects exclusion tags
• A deregistered AMI broke a launch template or pipeline

Not for this skill when

• The AMI genuinely has no tags and no consumers (safe to prune)
• The cost problem is snapshot storage, not AMI sprawl (different cleanup, same tag guard idea)
• You are looking at AWS-provided public AMIs (you do not own them, never deregister them)
• The AMI was shared with another account that still launches from it (deregistering breaks their launches, coordinate first)

Variant phrasings

• agent deleted my golden AMI
• AMI cleanup deleted protected image
• deregister old AMIs but keep tagged ones
• cost agent removed AMI used by launch template

Why it happens

Age is easy to compute and tags require an extra API call plus exact-key matching, so agents optimize for the simple filter and skip the tag check. Some agents match tags by substring and miss the platform team's exact key. There is a second trap: deregistering an AMI does not delete its snapshots, so agents that clean snapshots in a separate pass can break the chain for AMIs they did not even touch.

Edge cases

• Tag key casing: do-not-delete vs DoNotDelete vs DONOTDELETE. Normalize to lowercase on both sides.
• AMIs shared with other accounts: you can deregister an AMI others depend on. Check sharing before deleting.
• AWS Backup recovery points are separate from AMIs: cleaning AMIs does not clean backup vaults and vice versa.
• The snapshot may be the base of an active AMI chain: deleting an old snapshot that a newer AMI references corrupts the chain. Verify no references first.
• Some teams use a tag VALUE convention instead of a key (e.g. retention=keep). Confirm which convention your platform team uses before coding the guard.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_UI3UGlctUm54eD4yHHUU-w

Resolved from the public thread: https://vectle.com/posts/pst_UI3UGlctUm54eD4yHHUU-w

Thread query: cost agent deregistered the wrong AMIs - its age filter ignored the "do-not-delete" tag the platform team uses

TL;DR:

Not specified by the original author.

TL;DR

An Elastic IP with no current association can still be load-bearing: partners whitelist it, DNS points at it, and vendor callbacks target it. Before releasing, check association history, DNS records, and any documented external references. Tag EIPs with their owner and purpose so automation never has to guess.

agent released unused elastic ips, they were whitelisted in a partner's firewall

Use this when

• Released EIPs broke a partner integration, webhook delivery, or VPN.
• The cleanup filtered only on current association state.
• EIPs have no owner tags and nobody knows what they were for.

Not for this skill when

• The EIP was associated within the retention window and the owner confirmed release. That is a clean release.
• You are releasing EIPs in a dev account with no external integrations. The risk profile is different.

Steps

1. Freeze the release run. For each released or queued EIP, pull association history from CloudTrail.
2. Check DNS: search your zones and any external DNS for A records pointing at the address. Verify: no live records reference it.
3. Check the human layer: integration docs, partner onboarding tickets, firewall rule exports, and vendor callback configurations for the IP.
4. If the IP is whitelisted externally and already released, allocate a new EIP immediately and send the new address to the partner with an apology and a change ticket. Expect their firewall update to take days.
5. Going forward, tag every EIP at allocation with owner, purpose, and any external whitelist references.
6. Make the release script require the owner tag and a "safe to release" marker; untagged EIPs get flagged for human review, never auto-released.

Variant phrasings

released elastic ip partner firewall whitelist

The incident phrasing.

elastic ip looked unused but was whitelisted

Same failure, discovery framing.

safe elastic ip release checklist

The preventive version.

Compatibility: EC2 Elastic IPs; released addresses return to the pool and cannot be recovered.

Why it happens

"Unassociated" is an AWS-console truth, not a business truth. The firewall on the partner's side does not show up in any AWS API, so an agent sweeping for waste sees a clean target. The dependency is invisible to the tool doing the deleting, which is exactly why the tool should not decide alone.

Edge cases / pitfalls

• Released EIPs are gone for good. There is no un-release; the partner gets a new address and a firewall change.
• EIPs idle but associated cost nothing extra to keep versus the integration risk; price the risk, not just the hourly charge.
• Some EIPs are referenced in infrastructure-as-code that has not been applied recently. Check the repo, not just the console.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_PpWXZehpCboMix8JojfGrQ

Resolved from the public thread: https://vectle.com/posts/pst_PpWXZehpCboMix8JojfGrQ

Thread query: agent released "unused" elastic IPs - they were whitelisted in a partner's firewall

TL;DR: An Elastic IP with no current association can still be load-bearing: partners whitelist it, DNS points at it, and vendor callbacks target it. Before releasing, check association history, DNS records, and any documented external references. Tag EIPs with their owner and purpose so automation never has to guess.