Kubernetes deployment rollout stuck: kubectl rollout debugging

TL;DR

A stuck rollout means the new ReplicaSet cant get healthy pods: check kubectl rollout status output, then kubectl describe the new ReplicaSet and its pods for the real blocker. Usually its failing readiness probes, image pull errors, or resource constraints. Fix the pod problem and the rollout resumes on its own.

The error

$ kubectl rollout status deployment/myapp
Waiting for deployment "myapp" rollout to finish: 1 old replicas are pending termination, 1 new replicas are unavailable...

Or: ProgressDeadlineExceeded, rollout hanging for 10+ minutes.

Use this when

• kubectl rollout status hangs or times out
• new pods never reach Ready during a deploy
• events show ProgressDeadlineExceeded
• a deploy is half old pods, half new pods, stuck

Not for

• pods crashing after the rollout finished (debug the app, not the rollout)
• kubectl apply rejected by admission webhooks (fix the manifest)
• Helm releases stuck (check helm status first)

Steps

1. See where the rollout stands:

kubectl rollout status deployment/[name] -n [namespace]
kubectl get replicasets -n [namespace] -l app=[app]

Expected: status names whats stuck (new replicas unavailable, old pending termination). The ReplicaSet list shows the new RS with fewer ready pods than desired.

1. Inspect the new ReplicaSet and its pods:

kubectl describe replicaset [new-rs-name] -n [namespace]
kubectl get pods -n [namespace] -l app=[app] --sort-by=.metadata.creationTimestamp | tail -5

Expected: the newest pods reveal the blocker: ImagePullBackOff, CrashLoopBackOff, Pending, or readiness probe failures.

1. Read the failing pod's events:

kubectl describe pod [new-pod] -n [namespace] | tail -25

Expected: the exact cause. Fix that (image, probe, resources, config), not the deployment object itself.

1. Check rollout strategy settings that can look like a hang:

kubectl get deployment [name] -n [namespace] -o jsonpath='{.spec.strategy}{.spec.progressDeadlineSeconds}'

Expected: maxUnavailable: 0 with failing new pods means old pods stay until new ones are ready, which looks stuck but is working as designed. progressDeadlineSeconds (default 600) is when Kubernetes gives up and marks it failed.

1. If the new pods are fine but old ones wont terminate, check for stuck finalizers or PDBs:

kubectl get pod [old-pod] -n [namespace] -o jsonpath='{.metadata.finalizers}'
kubectl get pdb -n [namespace]

Expected: a PodDisruptionBudget with maxUnavailable: 0 blocks all evictions, including rollout-driven ones. Loosen it or the rollout waits forever.

1. Once fixed, either let it resume or restart it cleanly:

kubectl rollout restart deployment/[name] -n [namespace]
kubectl rollout status deployment/[name] -n [namespace]

Expected: rollout completes, all pods on the new revision. Verify with kubectl rollout history deployment/[name].

Variant: ProgressDeadlineExceeded already fired

The deployment is marked failed but pods may still be trying. Fix the pod issue, then kubectl rollout restart to kick a fresh attempt.

Variant: rollout stuck because the new image tag didnt change

image: myapp:latest with imagePullPolicy IfNotPresent never pulls the new build. Use immutable tags (git SHA) so every deploy is actually new.

Variant: canary or blue-green stuck halfway

Check the traffic-splitting config (service selectors, ingress weights) separately from the rollout; the pods may be fine while the traffic shift is what hung.

Why it happens

Rolling updates only proceed when new pods become Ready. Anything that keeps a new pod unready (bad image, failing probe, no resources, crashing app) pauses the whole rollout by design, so a bad deploy cant take down the working old pods. The rollout status is just the messenger; the pod is the problem.

Edge cases

• kubectl rollout undo rolls back to the previous revision, but only helps if the previous revision was healthy.
• Paused deployments (kubectl rollout pause) look stuck; check .spec.paused.
• Readiness gates from external controllers can hold pods unready indefinitely; check for custom readiness gates.
• Very large minReadySeconds plus slow probes makes every rollout look slow; tune it to what the app actually needs.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_kQUhVa2n8XMunSye1FcBQg