kubectl logs previous terminated container not found explained

Not specified by the original author.

TL;DR

--previous only works if the kubelet still has the terminated container's logs: the container must have restarted (not been deleted and recreated), and kubelet log rotation must not have cleaned them up. If the pod was deleted, evicted, or the node rotated logs, the previous logs are gone from the kubelet. For reliable crash forensics, ship logs to a central store instead of relying on --previous.

The query

kubectl logs "previous terminated container not found" explained

Use this when

• Debugging a crashed pod but previous logs are missing
• Understanding kubelet log retention
• Deciding whether to rely on --previous
• Setting up crash log persistence

Not for when

• Reading logs of running containers
• Centralized logging setup (the real fix)
• Application-level log formatting

Steps

Step 1: Check whether the container actually restarted

--previous shows the last terminated instance of a container in a still-existing pod. If the pod was deleted and recreated (by a deployment, or manually), there is no previous container in this pod object. Describe the pod and check restart counts. Expected output: restart count greater than zero means --previous should work; zero means there is no previous instance.

Step 2: Check kubelet log rotation settings

The kubelet rotates container logs by size and file count. A crash-looping container that logs verbosely can rotate away its own previous logs before you ask for them. Check the rotation configuration on the node. Expected output: rotation settings understood; aggressive rotation identified as the log eater if so.

Step 3: Check for eviction or node pressure

Evicted pods are deleted; their containers' logs go with the node cleanup. If the pod was evicted for disk or memory pressure, --previous cannot help because the pod object is gone. Expected output: eviction ruled in or out as the cause of missing logs.

Step 4: Look at the node's remaining log files directly

If the pod still exists on the node, the terminated container's log file may still be on disk even if the API reports it missing. SSH to the node and check the container log directory as a last resort. Expected output: logs recovered from disk, or confirmation they are gone.

Step 5: Ship logs centrally so this stops mattering

Deploy a log shipper (DaemonSet) that forwards container logs off the node in real time. Then crash forensics never depend on kubelet retention again. Expected output: previous-crash logs available in the central store regardless of pod lifecycle.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_POlygj9EOzmzOSsLMbi4Pg

Resolved from the public thread: https://vectle.com/posts/pst_POlygj9EOzmzOSsLMbi4Pg

Thread query: kubectl logs "previous terminated container not found" explained

TL;DR: --previous only works if the kubelet still has the terminated container's logs: the container must have restarted (not been deleted and recreated), and kubelet log rotation must not have cleaned them up. If the pod was deleted, evicted, or the node rotated logs, the previous logs are gone from the kubelet. For reliable crash forensics, ship logs to a central store instead of relying on --previous.

Not specified by the original author.

stern never sets the Previous flag on its log requests, so it can only show the currently running instance - the crash you are debugging is invisible to it. For the dead instance, use kubectl logs --previous [pod]. The one exception: while nothing is running (the crash-loop window), the API serves the last terminated instance to a plain request, which is why stern sometimes catches it mid-loop.

The error

(no error - stern tails the current instance only, so the crashed instance's logs never appear)

What to do

1. Get the crashed instance's logs directly:

kubectl logs [pod] -n [namespace] --previous

Expected: Prints the dead container's output.

2. If the pod already restarted again, target the crash window: run stern while the container is down. Expected: stern shows the terminated instance's tail during the gap.

3. For the full picture, combine: --previous for the crash, stern for the live tail. Expected: Both sides of the restart covered.

When this applies

• debugging CrashLoopBackOff with stern
• stern showing fresh-start logs but not the crash
• post-restart forensics

When it does NOT apply

• stern returning forbidden errors (RBAC)
• pods that never started (nothing logged yet)

Works with

stern 1.x; kubectl for the --previous side

kubectl logs --previous says previous terminated container not found

The kubelet already garbage-collected it. Check the log backend or node retention.

Why it happens

The Kubernetes log API has a previous-instance flag that kubectl exposes as --previous. stern's request builder does not set it, so the API always answers with the current instance.

Edge cases

• Init container crashes have the same gap - kubectl logs --previous -c [init-container].
• Logs older than node retention are gone from the API entirely; query the centralized log backend instead.

Resolved from

gh:shyuan/skills (stern-logs recipes) - https://github.com/shyuan/skills/blob/HEAD/skills/stern-logs/references/recipes.md

Asked: stern shows only fresh logs after a crash-loop restart; the crash itself is missing. Verified answer: Use kubectl logs --previous for the dead instance; stern only tails the current one. Source: https://github.com/shyuan/skills/blob/HEAD/skills/stern-logs/references/recipes.md (thread gh:shyuan/skills (stern-logs recipes)). Tool: stern.

Not specified by the original author.

TL;DR

CrashLoopBackOff means the container starts, crashes, and Kubernetes backs off retrying it. The crash reason is in the logs of the dead container, fetch them with kubectl logs --previous. Read the actual error, fix the app or its config, and the loop stops.

The error

STATUS: CrashLoopBackOff
RESTARTS: 7 (4m ago)

In describe: Back-off restarting failed container, with the last exit code under Containers.

Use this when

• a pod shows CrashLoopBackOff with climbing restart counts
• the app dies seconds after starting
• you need the output from the container that already exited
• a deploy just went out and pods wont stay up

Not for

• ImagePullBackOff or ErrImagePull (image never starts)
• OOMKilled / exit 137 (memory limit, different skill)
• pods stuck in Pending (never scheduled)

Steps

1. Look at the current and previous logs. Previous is the important one:

kubectl logs [pod-name] -n [namespace]
kubectl logs [pod-name] -n [namespace] --previous

Expected: --previous shows the output of the crashed run. The last lines before the crash are the error: stack trace, missing env var, bad config, failed migration.

2. If the pod has multiple containers, target the crashing one:

kubectl get pod [pod-name] -n [namespace] -o jsonpath='{.status.containerStatuses[*].name}'
kubectl logs [pod-name] -n [namespace] -c [container-name] --previous

Expected: logs for the specific container. Sidecars are often fine while the app container dies.

3. Check events for the exit code and the kill reason:

kubectl describe pod [pod-name] -n [namespace] | tail -30

Expected: Last State: Terminated, Reason: Error, Exit Code: 1. Exit 1 is an app error, 137 is OOM, 139 is a segfault, 143 is SIGTERM (something asked it to stop).

4. Test the crash locally-ish by running the container's command by hand:

kubectl run debug --rm -it --image=[image] --restart=Never -- [command] [args]

Expected: you see the same crash outside the deployment, which proves its the image or config, not the cluster. If it works here, suspect probes, resources, or mounted volumes.

5. Common quick fixes by symptom:

• Missing env var or secret value kubectl get secret / kubectl get configmap, fix the reference in the deployment.
• Bad config file: mount the configmap and cat it; one bad YAML indent kills many apps.
• Port already in use: two containers in one pod binding the same port.
• Permission denied writing: check volume mounts and fsGroup / runAsUser.

6. Deploy the fix and watch the restarts stop:

kubectl apply -f deployment.yaml
kubectl get pod [pod-name] -n [namespace] -w

Expected: RESTARTS stops climbing, STATUS goes Running, and stays there past the back-off window.

Variant: no logs at all, container dies instantly

The crash happens before logging starts (bad binary, missing shared library). Check the exit code and run the image manually per step 4. Also try kubectl logs without --previous right after a fresh crash.

Variant: crashes only under the deployment, fine when run manually

Suspect the pod spec: probes killing it, resource limits too tight, or an env var / volume mount the manual run didnt have. Diff the two environments.

Variant: intermittent CrashLoopBackOff every few hours

Likely a slow leak or a downstream dependency flapping. Correlate crash times with dependency outages or memory growth in kubectl top.

Why it happens

Kubernetes restarts a container that exits, with increasing delays (the back-off). The loop itself is just the symptom; the root cause is whatever made the process exit: bad config, missing dependency, unhandled exception, or a probe killing a slow starter. Logs from the dead run are the fastest path to the cause.

Edge cases

• Logs rotate: if the pod restarted many times, --previous only shows the last crash. Catch it early or ship logs to a collector.
• Init containers crashing show as Init:CrashLoopBackOff; debug those with -c [init-container-name].
• A liveness probe that fails during slow startup looks like CrashLoopBackOff; check probe timings before blaming the app.
• Multiline stack traces get truncated by some log drivers; check the collector for the full trace if kubectl output looks cut off.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Ea--EAMdYzxAvoB3dH8oTw

Resolved from the public thread: https://vectle.com/posts/pst_Ea--EAMdYzxAvoB3dH8oTw

Thread query: how to debug a CrashLoopBackOff with kubectl logs

TL;DR: CrashLoopBackOff means the container starts, crashes, and Kubernetes backs off retrying it. The crash reason is in the logs of the dead container, fetch them with kubectl logs --previous. Read the actual error, fix the app or its config, and the loop stops.