Kubernetes secret not mounting: volume debug steps

TL;DR

A secret that will not mount is usually one of four things: the secret does not exist in the pod's namespace, the name in the volume does not match, RBAC denies the kubelet or service account from reading it, or the pod spec has a typo in the volume mount path. Check them in that order and you will find it fast.

The query

Kubernetes secret not mounting: volume debug steps

Use this when

• A pod fails to start with a secret volume error
• Environment variables from a secret come up empty
• Mounted secret files contain old values
• The secret exists but the pod cannot see it

Not for when

• Creating secrets for the first time
• External secret operators (External Secrets, Vault injector)
• Sealed Secrets decryption issues

Steps

Step 1: Confirm the secret exists in the pod's namespace

Describe or get the secret using the exact name from the pod spec, in the pod's namespace. Secrets are namespaced; the number one cause is the secret living in default while the pod runs elsewhere. Expected output: the secret is found, or you have identified that it is missing or in the wrong namespace.

Step 2: Match names between the volume and the secret

Compare the secretName in the pod's volume definition character by character with the actual secret name, including the key names if you use items mapping. Typos and case differences are common. Expected output: names match exactly, or you found the mismatch to fix.

Step 3: Check RBAC for the pod's service account

The kubelet reads secrets on the pod's behalf using the pod's service account credentials. If a restrictive RBAC policy denies secret reads, the mount fails. Check the roles bound to the service account in that namespace. Expected output: the service account can get the named secret, or you found the denying policy.

Step 4: Inspect the events on the pod

Pod events describe mount failures plainly: "failed to mount", "secret not found", permission errors. Read the events before theorizing; they usually name the exact problem. Expected output: an event message that confirms one of the causes above, narrowing the fix.

Step 5: For stale values, restart the pod

Mounted secret volumes update automatically, but there is a delay (kubelet sync period plus cache TTL), and environment variables from secrets never update without a restart. If values look stale, check whether you are reading env vars (restart needed) or files (wait for sync). Expected output: fresh values after the appropriate action; a rollout strategy for future secret rotations.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_etClKF12XASmn27nHHMR2A