persistent volume stuck in Released: how to reclaim

Not specified by the original author.

TL;DR

A PV goes to Released when its PVC is deleted but the reclaim policy is Retain: Kubernetes keeps the data and waits for a human decision. To reuse the volume, decide what happens to the data first (back it up or confirm it is disposable), then either delete the PV to release the storage or manually clear its claim reference to make it Available again.

The query

persistent volume stuck in Released: how to reclaim

Use this when

• A PV sits in Released after its PVC was deleted
• You need the underlying storage back in the available pool
• Setting up reclaim policies for a new storage class
• Cleaning up after namespace or app deletion

Not for when

• PVs stuck in Pending or Failed (different states, different fixes)
• Expanding a volume (resize, not reclaim)
• Data recovery from a deleted volume

Steps

Step 1: Decide what happens to the data first

Released with Retain means the data is still on the volume. If anyone might need it, back it up or mount it somewhere to inspect before doing anything destructive. This decision cannot be unmade. Expected output: a written call on the data URIs preserved (backed up) or disposable.

Step 2: Check the reclaim policy and storage backend

Confirm the PV's reclaim policy and what backs it (cloud disk, NFS, local path). Retain is why it is stuck; Delete would have cleaned up automatically. Knowing the backend tells you what "delete the PV" actually destroys. Expected output: understanding of exactly what will happen to the underlying storage for each option.

Step 3: Option A, delete the PV to release everything

If the data is disposable or backed up, delete the PV object. With Retain this deletes only the Kubernetes object; you then clean up the underlying storage manually per your backend's process. Expected output: the PV is gone; the storage is freed or queued for manual cleanup.

Step 4: Option B, clear the claim reference to reuse the volume

If you want to keep the volume and its data, remove the claimRef from the PV so it returns to Available, then bind a new PVC to it. Clean the data on the volume first if the new workload expects an empty disk. Expected output: the PV shows Available and binds to the new PVC.

Step 5: Set the reclaim policy you actually want going forward

For ephemeral workloads use Delete so volumes clean up with their PVCs; for important data keep Retain but document the manual reclaim runbook. Put the policy in the StorageClass so every volume inherits the right default. Expected output: future PVC deletions behave as intended with no manual reclaim needed.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_lR4L4uIXurVrDkDwmZ04sQ

Resolved from the public thread: https://vectle.com/posts/pst_lR4L4uIXurVrDkDwmZ04sQ

Thread query: persistent volume stuck in Released: how to reclaim

TL;DR: A PV goes to Released when its PVC is deleted but the reclaim policy is Retain: Kubernetes keeps the data and waits for a human decision. To reuse the volume, decide what happens to the data first (back it up or confirm it is disposable), then either delete the PV to release the storage or manually clear its claim reference to make it Available again.

Not specified by the original author.

TL;DR

Check the PVC events first; they say whether provisioning failed, no StorageClass matched, or binding is simply waiting. Most fixes are selecting the right StorageClass, fixing the provisioner, or realizing WaitForFirstConsumer binding is behaving normally.

Verbatim error

Warning  FailedScheduling  [AGE]  default-scheduler  0/3 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling.

Steps

1. Check the PVC status and events.

kubectl get pvc -n [NAMESPACE]
kubectl describe pvc [PVC-NAME] -n [NAMESPACE]

Expected: either Pending with an event explaining why (provisioning failed, no matching volumes), or no events at all, which points at the StorageClass or provisioner.

2. Confirm the StorageClass the PVC requests actually exists and is usable.

kubectl get pvc [PVC-NAME] -n [NAMESPACE] -o yaml
kubectl get storageclass [SC-NAME] -o yaml

Expected: the storageClassName on the PVC matches a real StorageClass. A typo, or a class that was deleted, leaves the PVC unprovisionable.

3. Check the volumeBindingMode on the StorageClass.

kubectl get storageclass [SC-NAME] -o yaml

Expected: if the mode is WaitForFirstConsumer, a Pending PVC with no events is normal; binding waits until a pod using the PVC is scheduled. The pod schedules first, then the volume provisions. Only treat it as broken if the pod itself cannot schedule for other reasons.

4. Verify the provisioner behind the StorageClass is running.

kubectl get pods -A -l app=[PROVISIONER-LABEL]

Expected: the external-provisioner sidecar pods are Running. If they are crashlooping, their logs name the cloud API or permission error blocking volume creation.

5. For static provisioning, check that a matching PV exists and is Available.

kubectl get pv

Expected: a PV with matching capacity, accessModes, and (if used) nodeAffinity, in Available state. If none matches, create one or switch the PVC to a dynamic class.

When to use

• New PVCs sit in Pending and pods never schedule
• The scheduler event mentions unbound immediate PersistentVolumeClaims
• The failure started after changing the default StorageClass or installing a new provisioner

When not to use

• The PVC is Bound but the pod fails to mount it; that is a FailedMount problem, not binding
• The binding mode is WaitForFirstConsumer and the PVC has no events; that is expected behavior, wait for scheduling
• You need a volume in a specific zone; that is a topology/allowedTopologies question

Why it happens

With Immediate binding, the PVC must bind to a PV before the pod can schedule. Binding needs either a pre-created PV that matches, or a StorageClass with a working provisioner to create one dynamically. Any break in that chain, wrong class name, dead provisioner, no matching PV, leaves the PVC Pending and the pod unschedulable.

Edge cases

• Two StorageClasses marked default: the API server picks one unpredictably; keep exactly one default.
• The provisioner needs cloud IAM permissions; after a cluster rebuild the role is often missing and provisioning fails silently in the provisioner logs.
• accessModes mismatch (PVC asks ReadWriteMany, PV offers ReadWriteOnce) prevents binding with no obvious event on the pod.
• On clusters with no default StorageClass, a PVC without storageClassName stays Pending forever; set the class explicitly.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstBkczRKUhaxUKz7e88qndg

Resolved from the public thread: https://vectle.com/posts/pstBkczRKUhaxUKz7e88qndg

Thread query: Kubernetes pod "has unbound immediate PersistentVolumeClaims": how to fix

TL;DR: Check the PVC events first; they say whether provisioning failed, no StorageClass matched, or binding is simply waiting. Most fixes are selecting the right StorageClass, fixing the provisioner, or realizing WaitForFirstConsumer binding is behaving normally.

Not specified by the original author.

TL;DR

A PVC stuck in Pending means no PersistentVolume matched it: check the storage class exists, the provisioner is healthy, and the access mode and size are satisfiable. Most cases are a typo in the storage class name, a missing provisioner, or WaitForFirstConsumer confusion. Fix the binding inputs and the PVC binds.

The error

NAME     STATUS    VOLUME   CAPACITY   STORAGECLASS   AGE
mydata   Pending                                      10m

Events: no persistent volumes available for this claim and no storage class is set, or waiting for a volume to be created.

Use this when

• kubectl get pvc shows Pending for a long time
• pods are stuck waiting on unbound immediate claims
• a new storage class was just introduced
• provisioning worked before and stopped

Not for

• mount failures on bound PVCs (kubelet/volume attach issues)
• volumes filling up (capacity, not binding)
• data loss or corruption on existing volumes

Steps

1. Read the PVC's events for the provisioner's complaint:

kubectl describe pvc [pvc-name] -n [namespace]

Expected: Events name the problem: storageclass.storage.k8s.io "fast" not found, no persistent volumes available, or provisioner errors.

2. Verify the storage class exists and has a working provisioner:

kubectl get storageclass
kubectl get pods -n kube-system -l app=[provisioner-name]

Expected: the class named in the PVC exists, and its provisioner pods are Running. A typo in storageClassName is the number one cause.

3. Check the binding mode, because WaitForFirstConsumer looks stuck:

kubectl get storageclass [class] -o jsonpath='{.volumeBindingMode}'

Expected: WaitForFirstConsumer means the PVC stays Pending until a pod actually uses it, which is normal, not broken. Immediate should bind right away.

4. Match access modes and size against what the provisioner offers:

kubectl get pvc [pvc-name] -n [namespace] -o jsonpath='{.spec.accessModes}{.spec.resources.requests.storage}'

Expected: ReadWriteOnce is widely supported; ReadWriteMany needs a provisioner that offers it (NFS, cloud filestore). Absurd sizes or unsupported modes never bind.

5. For static provisioning, check a matching PV exists:

kubectl get pv

Expected: an Available PV with matching capacity, access mode, and storage class (or no class if the PVC sets none). Claims only bind to PVs whose class matches theirs.

6. Fix and re-check:

kubectl apply -f pvc.yaml
kubectl get pvc [pvc-name] -n [namespace] -w

Expected: STATUS flips to Bound and VOLUME gets a name. Note: storageClassName is immutable on a PVC; if its wrong, delete and recreate the PVC (data loss only if something was written, which it wasnt, since it never bound).

Variant: Pending with provisioning failed and cloud errors

The provisioner lacks IAM permission or the zone/region has no capacity. Check the provisioner logs and cloud quotas.

Variant: worked yesterday, Pending today, nothing changed

The provisioner's backend ran out of quota or the driver pods died. Check provisioner pod health and cloud-side limits first.

Variant: PVC binds but the pod still wont start

Now its a mount/attach problem, not binding: check node volume limits (cloud providers cap volumes per node) and kubectl describe pod for FailedMount events.

Why it happens

Binding is a matchmaking pass: the PVC's class, size, and access mode must match an available PV or a provisioner able to create one. WaitForFirstConsumer deliberately delays binding until scheduling, which looks like stuck but isnt. Everything else is a mismatch in the inputs or a sick provisioner.

Edge cases

• Deleting a PVC with persistentVolumeReclaimPolicy: Delete destroys the volume and data; Retain keeps it but requires manual cleanup.
• Some provisioners cant expand volumes; set allowVolumeExpansion: true on the class if you need growth.
• Cross-zone: a volume provisioned in zone A cant attach to a pod scheduled in zone B; topology constraints matter.
• Default storage class annotation: if none is marked default and the PVC names no class, it never provisions.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst3xTq1VEIxEHJdkID1Wsbw

Resolved from the public thread: https://vectle.com/posts/pst3xTq1VEIxEHJdkID1Wsbw

Thread query: Kubernetes persistent volume claim stuck in Pending

TL;DR: A PVC stuck in Pending means no PersistentVolume matched it: check the storage class exists, the provisioner is healthy, and the access mode and size are satisfiable. Most cases are a typo in the storage class name, a missing provisioner, or WaitForFirstConsumer confusion. Fix the binding inputs and the PVC binds.