agent's service account got locked out after failed auth retries

TL;DR

Account lockout is a retry-design failure: directories lock accounts after a handful of bad attempts, and an agent retrying a broken credential every few seconds generates hundreds. Prevent it with auth-specific rules: never retry an authentication failure more than once without new credential material, back off hard on 401 and 403 responses, validate credentials once at run start and fail fast, and alert on the first auth failure instead of the fiftieth. Coordinate with the account owner on a lockout policy appropriate for a service account.

agent's service account got locked out after failed auth retries

Use this when

• The service account is locked in Active Directory or the IdP after an agent run.
• Security logs show a burst of failed logons from the agent host.
• The triggering credential had expired or been rotated without updating the vault.

Not for this skill when

• The lockout came from a genuine attack or from another system using the same account. Coordinate with security before changing agent behavior.
• The account has no lockout issue; auth fails for a permissions reason. Different fix.

Steps

1. Unlock the account through the owner process and find the failed-run evidence (which credential, which target, how many attempts). Verify: the attempt count and the stale credential source are identified
2. Fix the credential at the source (update the vault entry through the approved flow) and validate it once interactively or with a single test call. Verify: one clean authentication succeeds
3. Implement auth-failure circuit breaking in the agent: authentication errors are terminal for the run after one retry, with an immediate alert. Verify: a wrong-credential test produces exactly the allowed attempts and stops
4. Add a run-start credential validation step for every system the wave touches, failing the wave before any host work if one is bad. Verify: a bad credential now fails in the first minute, not mid-wave
5. Agree with the identity team on service account protections (lockout threshold, alerting on lockout, separate accounts per environment) and document them. Verify: the account configuration matches the agreement

Variant phrasings

service account locked out agent retry loop circuit breaker

The main phrasing.

failed logon burst automation account lockout ad policy

The directory phrasing.

validate credentials at run start fail fast patch agent

The prevention phrasing.

Compatibility: Compatibility: Active Directory and cloud IdP lockout policies; any agent authenticating with a shared service account.

Why it happens

Lockout policies assume a human who mistypes a few times, not a program that can mistype a thousand times before anyone looks. The agent retry defaults that are safe for network errors are dangerous for authentication errors, because each retry spends from a small, security-owned budget and the penalty (a locked account) disables the automation entirely and creates a security event. Separating error classes, with authentication in the never-blindly-retry class, is the entire fix.

Edge cases / pitfalls

• One account shared across many agents multiplies the blast radius; per-agent or per-environment accounts contain a lockout to one workload.
• A lockout alert should reach the automation owner immediately; discovering it at the next wave wastes a window.
• Password rotation schedules and vault updates must be one atomic change, or every rotation manufactures this incident.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_LlwL9gYedtbk-7ij2KPXnw