Finding what keeps changing your infra outside Terraform: "Objects have changed" every run

Finding what keeps changing your infra outside Terraform

Symptom

Every terraform plan reports Objects have changed outside of Terraform on the same resources, even right after a clean apply.

Cause

Something with API access is mutating the resources: an autoscaler, a Kubernetes controller, a console user, another IaC tool, or a provider-side defaulting behavior.

Confirmation

1. Read the drift note: which attributes change? Scaling counts point at autoscalers; tags point at humans or policy tools; timestamps/version fields point at controllers.
2. Check cloud audit logs (CloudTrail, Activity Log, Audit Logs) for the resource ARN around the drift window. The userIdentity field names the writer.
3. Check for controllers: does a Kubernetes operator, ASG, or external system own this resource? Resource tags and descriptions often name the owner.
4. Rule out Terraform itself: terraform plan immediately after terraform apply should be clean. If it is not clean even with no external actor, the provider is defaulting something (see the inconsistent-result diagnostic).

Fix

1. Human writers: move their workflow into Terraform or into the config. Console hotfixes during incidents should be codified afterwards.
2. Controllers/autoscalers: either manage the resource entirely outside Terraform (remove it, use a data source), or add a scoped ignore_changes on exactly the controller-managed attributes, with a comment naming the controller.
3. Another IaC tool: split ownership clearly. Two writers for one resource is the bug; decide who owns it.

Verification

1. Apply, wait a full controller cycle (or a day for human-driven drift), then plan. Clean means the writer is handled.
2. ignore_changes should name specific attributes, never whole resources. Verify the plan still catches changes to the non-ignored attributes.

Refresh reports externally changed objects as a note; -refresh=false hides it; ignore_changes scopes per attribute. Verified against live docs 2026-09-26.