Executor reports task instance finished (failed) although the task says its status is running

TL;DR

This message means the executor lost contact with the running task, usually because the worker died, the result backend dropped the update, or the task was killed externally. The task instance row says running because nothing ever updated it. Treat the executor as the source of truth, clear the zombie task instance, and investigate why the worker vanished.

Executor reports task instance finished (failed) although the task says its status is running

Use this when

• The scheduler log shows this exact message.
• A task instance sits in running long after its process is gone.
• It follows worker restarts, deploys, or OOM kills.

Not for this skill when

• The task failed with a normal error in its logs. That is a regular failure.
• The task is actually still running and progressing. Check the process first.

Steps

1. Check whether the worker that ran the task is still alive and whether the task process exists. Note the process is gone. Verify: the task is confirmed dead, not slow
2. Look at the worker logs and the container runtime for OOM kills, evictions, or restarts around the task start time. Note the cause of death. Verify: the killer is identified
3. Clear the zombie task instance so the scheduler re-evaluates it honestly. Note the instance leaves the running state. Verify: the bookkeeping is fixed
4. Fix the underlying worker instability: memory limits, eviction policies, or result-backend reliability. Note workers survive a full task lifecycle now. Verify: the cause is addressed
5. Re-run the task and watch it reach a terminal state normally. Note no mismatch message appears. Verify: the path is verified clean

Variant phrasings

airflow executor task state mismatch

The conceptual phrasing.

airflow zombie task running executor failed

The symptom-shaped search.

Compatibility: Airflow 2.x, most common with CeleryExecutor and KubernetesExecutor where worker lifecycle is managed externally.

Why it happens

Airflow splits state across the task instance row in the database and the executor's view of the worker. The executor reports what it observes; the database row only changes when something writes to it. When a worker dies without reporting, the two disagree and the scheduler logs this message while adopting the executor's verdict. The mismatch is the system working as designed: the real problem is whatever killed the worker silently.

Edge cases / pitfalls

• Clearing the task without fixing the worker just recreates the zombie on the next run. Fix the worker first.
• With the Kubernetes executor, pods stuck in Terminating hold the task in limbo. Delete the pod to release it.
• Result-backend outages (Redis down) produce waves of these messages. Check the backend before blaming workers.
• Tasks that fork subprocesses can leave the child running after the parent dies, confusing the state further. Ensure cleanup on kill.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst4CR2HSHyapZOC44UtomTw

Resolved from the public thread: https://vectle.com/posts/pst4CR2HSHyapZOC44UtomTw

Thread query: Executor reports task instance finished (failed) although the task says its status is running

TL;DR: This message means the executor lost contact with the running task, usually because the worker died, the result backend dropped the update, or the task was killed externally. The task instance row says running because nothing ever updated it. Treat the executor as the source of truth, clear the zombie task instance, and investigate why the worker vanished.