## TL;DR
Never delete an exposed service account key before its replacement is live everywhere; do create, roll out, then delete. Create the new key, update every consumer, verify, and only then delete the old key. Deleting first causes outages; deleting never leaves the exposure open.

## Error
```text
exposed gcp service account key: rotation failed
```

## Steps
1. Create the replacement: run `gcloud iam service-accounts keys create` for the service account, saving the new key securely. Expected: a new key ID alongside the old one.
2. Inventory consumers: search CI configs, servers, and apps for the old key's usage (key ID or file). Expected: a complete list of everything that must move.
3. Roll out the new key to every consumer, one environment at a time, verifying each. Expected: all consumers authenticate with the new key.
4. Verify no traffic uses the old key: check audit logs for authentications with the old key ID. Expected: zero recent uses before you delete.
5. Delete the old key: run `gcloud iam service-accounts keys delete` with the old key ID. Expected: the old key is gone and cannot authenticate.

## When to use
- A GCP service account key was exposed and must be rotated
- Planned key rotation for hygiene
- Migrating consumers off a shared key

## When not to use
- Deciding to move to workload identity (migration project, then delete keys)
- Key already deleted but consumers still reference it (restore rollout order)

## Tool compatibility
- Google Cloud IAM service account keys (`gcloud iam service-accounts keys`); Cloud Audit Logs

## Variant phrasings
### gcp key rotation caused outage
The old key was deleted before rollout finished; recreate the rollout order and keep both keys live during migration.

### find where service account key is used
Audit logs keyed by key ID show recent usage; combine with a config search for full coverage.

## Why it happens
Service account keys are long-lived bearer credentials copied to many places. Rotation fails when teams delete first (outage) or forget a consumer (the old key stays valid somewhere), so the create-rollout-verify-delete sequence is the only safe order.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_5qosguxECD9cqZY3nO-xVg