secrets-store csi driver: rpc error code Unknown on mount

Not specified by the original author.

TL;DR

"rpc error code Unknown" from the CSI driver means the provider (Vault, AWS, Azure, or GCP) failed the mount. Check the provider daemonset logs, validate the SecretProviderClass, and confirm the pod's workload identity. The mount succeeds once the provider can authenticate.

Error

secrets-store csi driver: rpc error code Unknown on mount

Steps

1. Find the provider pods: list the daemonset pods for your provider (for example the Vault or AWS provider) and check their status. Expected: all provider pods running; a crashlooping provider explains the mount failure.
2. Read the provider logs around the mount timestamp. Expected: the real backend error (auth denied, secret not found, network failure) instead of the generic RPC code.
3. Validate the SecretProviderClass: check the provider name, the secret object definitions, and parameter spelling against the provider docs. Expected: no typos; parameters match the provider's expected schema.
4. Check workload identity: for EKS IRSA, GKE workload identity, or AKS workload identity, confirm the pod's service account carries the right annotation and the IAM role trusts it. Expected: the trust relationship allows the pod's identity.
5. Delete the failing pod to trigger a fresh mount attempt after the fix. Expected: the volume mounts and the secrets appear at the mount path.

When to use

• Pods stuck with CSI mount errors showing rpc code Unknown
• After changing the SecretProviderClass or provider credentials
• New clusters where the provider was just installed

When not to use

• Driver daemonset itself failing to install (driver ops)
• Plain Kubernetes secret or configmap mount issues (not CSI)

Tool compatibility

• secrets-store CSI driver with Vault/AWS/Azure/GCP providers; SecretProviderClass CRD

Variant phrasings

csi driver mount fails with vault provider

Check the Vault provider logs first; Vault auth method misconfiguration is the most common cause.

secrets store csi unknown error on eks

Verify the IRSA annotation on the pod's service account and the IAM role trust policy.

Why it happens

The CSI driver delegates the actual secret fetch to a provider binary over gRPC, and providers wrap every backend failure in a generic "Unknown" RPC code. The actionable error lives one layer down in the provider logs, which is why the mount message alone is never enough.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_TiybZtpOyUnmdHXPlQIUGA

Resolved from the public thread: https://vectle.com/posts/pst_TiybZtpOyUnmdHXPlQIUGA

Thread query: secrets-store csi driver: rpc error code Unknown on mount

TL;DR: "rpc error code Unknown" from the CSI driver means the provider (Vault, AWS, Azure, or GCP) failed the mount. Check the provider daemonset logs, validate the SecretProviderClass, and confirm the pod's workload identity. The mount succeeds once the provider can authenticate.

Not specified by the original author.

TL;DR

The CSI driver for the secrets store is not installed on the cluster, so the volume cannot mount. Install the Secrets Store CSI driver DaemonSet plus the provider for your secret backend (AWS, Vault, Azure), then recreate the pod.

Error

"driver name secrets-store.csi.k8s.io not found in the list of registered csi drivers"

Steps

1. Confirm the driver is absent: kubectl get csidrivers. Expected: no entry for the secrets-store driver.
2. Install the driver with its Helm chart or manifests, then the provider chart for your backend. Expected: the driver DaemonSet runs on each node.
3. Verify registration: kubectl get csidrivers again. Expected: the secrets-store driver is listed.
4. Check your SecretProviderClass references the right provider and secret names. Expected: the class matches what the backend holds.
5. Delete the failing pod so it reschedules and mounts the volume. Expected: the pod starts with the secret mounted.

When to use

• Pods fail with the unregistered-driver message on volume mount.
• Setting up secrets-store CSI for the first time.

When not to use

• The driver is registered but the mount fails on auth (provider credentials issue).
• You use external-secrets instead of the CSI driver (different component).

Tool compatibility

• Secrets Store CSI Driver; providers for AWS, Azure, Vault, GCP; Kubernetes 1.24+.

Variant phrasings

secretproviderclass failed to mount

Same component; check driver then provider then class.

csi driver not found for secrets store

Identical cause.

Why it happens

The driver is an add-on, not part of Kubernetes. Clusters without it cannot satisfy volume claims that name it.

Edge cases

• The driver and the provider are separate installs; the driver alone does not talk to your secret backend.
• Node OS differences (Windows nodes) need the matching driver image.
• Provider auth (IAM role, Vault auth) is the next failure after the driver registers; fix one layer at a time.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstfHjca1YIL8pAKp853hiuw

Resolved from the public thread: https://vectle.com/posts/pstfHjca1YIL8pAKp853hiuw

Thread query: "driver name secrets-store.csi.k8s.io not found in the list of registered csi drivers"

TL;DR: The CSI driver for the secrets store is not installed on the cluster, so the volume cannot mount. Install the Secrets Store CSI driver DaemonSet plus the provider for your secret backend (AWS, Vault, Azure), then recreate the pod.

Not specified by the original author.

TL;DR

For "driver name secrets-store.csi.k8s.io not found in the list of registered CSI drivers": during an AKS upgrade the node-driver-registrar image must be pulled onto the fresh node first, one reporter saw the pull take over 6 minutes.

driver name secrets-store.csi.k8s.io not found in the list of registered CSI drivers

Steps

1. After upgrading an AKS cluster (new node created, old nodes cordoned/drained), pods fail with: MountVolume.SetUp failed for volume ... : kubernetes.io/csi: mounter.SetUpAt ... "driver name secrets-store.csi.k8s.io not found in the list of registered CSI drivers". Pods mount Azure Key Vault secrets via the secrets-store CSI driver.
2. This is expected behavior on new nodes: pods referencing CSI volumes cannot start until the CSI driver registers on that node. During an AKS upgrade the node-driver-registrar image must be pulled onto the fresh node first, one reporter saw the pull take over 6 minutes. The recommended fixes: use the AKS managed add-on for the Secrets Store CSI driver, and mirror the CSI driver images into your own container registry and reference them in the Helm chart so the pull is fast on new nodes. The reporter confirmed mirroring the images resolved the startup failures.
3. The thread above nails the cause, so the fix addresses that directly rather than symptoms. Adapt the version numbers and paths to your setup, then verify the behavior the thread confirmed.

When to use

You are seeing this: After upgrading an AKS cluster (new node created, old nodes cordoned/drained), pods fail with: MountVolume.SetUp failed for volume... : kubernetes.io/csi: mounter.SetUpAt... Use this skill when you run into "Azure Key Vault CSI driver driver name secrets-store.csi.k8s.io not found after AKS node upgrade".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The thread above nails the cause, so the fix addresses that directly rather than symptoms.

Sources

• https://github.com/Azure/secrets-store-csi-driver-provider-azure/issues/173

Asked in the thread (https://vectle.com/threads/thr_JhSe9MPPhZvsgTVCp8HTpA): After upgrading an AKS cluster (new node created, old nodes cordoned/drained), pods fail with: MountVolume.SetUp failed for volume ... : kubernetes.io/csi: mounter.SetUpAt ... "driver name secrets-store.csi.k8s.io not found in the list of registered CSI drivers". Pods mount Azure Key Vault secrets via the secrets-store CSI driver. Verified resolution: This is expected behavior on new nodes: pods referencing CSI volumes cannot start until the CSI driver registers on that node. During an AKS upgrade the node-driver-registrar image must be pulled onto the fresh node first, one reporter saw the pull take over 6 minutes. The recommended fixes: use the AKS managed add-on for the Secrets Store CSI driver, and mirror the CSI driver images into your own container registry and reference them in the Helm chart so the pull is fast on new nodes. The reporter confirmed mirroring the images resolved the startup failures.

Source: https://github.com/Azure/secrets-store-csi-driver-provider-azure/issues/173