Kubernetes "service account token has expired" pod errors

TL;DR

Pods get API tokens via projected volumes that should refresh automatically; "token has expired" means the refresh is not happening. Causes: ancient clusters without bound tokens, tokens with explicit short lifetimes that the app caches, or the kubelet failing to rotate the file. Check the token file's freshness first, then whether the app re-reads it.

The query

Kubernetes "service account token has expired" pod errors

Use this when

• In-pod API calls start failing with token expiry
• Long-running pods lose API access after hours or days
• Apps cache the token instead of re-reading the file
• After changing token lifetime settings

Not for when

• kubectl from your laptop (user credential expiry, different fix)
• RBAC forbidden errors (auth works, authz denies)
• Token file missing entirely (mount problem)

Steps

Step 1: Check the token file's age and content

Read the token's expiry from inside the pod and compare with the mounted file's modification time. If the file is fresh but the app fails, the app cached an old token. If the file itself is stale, rotation is broken. Expected output: file-freshness vs app-behavior isolated as the problem layer.

Step 2: Verify the app re-reads the token

Many clients read the token once at startup. Bound tokens rotate roughly hourly; apps must re-read the file per request or on a timer. Check the client library's token reload behavior. Expected output: the app confirmed to reload the token, or the caching identified.

Step 3: Check kubelet token rotation

If the mounted file is stale, check kubelet logs for token rotation errors. Rotation failures usually trace to API server connectivity or clock skew between node and control plane. Expected output: rotation working, or the kubelet-side failure named.

Step 4: Consider explicit token lifetime settings

If tokens were issued with a short explicit lifetime (for security), either lengthen it to match the pod's expected lifetime or ensure the app handles rotation. Short lifetimes plus non-reloading apps equal guaranteed outages. Expected output: token lifetime aligned with the app's reload capability.

Step 5: Restart as remediation, fix as prevention

Restarting the pod gets a fresh token immediately. Then fix the reload or rotation issue so it does not recur. Do not let "just restart it monthly" become the runbook. Expected output: immediate recovery plus a permanent fix for the rotation path.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_mkJ5s23EknDx6q5PwRhnpQ