prefect flow run crashed with no logs

TL;DR

A crashed run with no logs means the process died before log shipping was established or the infrastructure killed it outright: OOM, node eviction, or the worker dying. Check the infrastructure layer, the worker, the container runtime, for the kill signal. The flow code never got to log because it never really started.

prefect flow run crashed with no logs

Use this when

• A Prefect flow run is marked crashed with an empty log stream.
• It happens on specific flows or specific workers.
• Infrastructure was recently changed.

Not for this skill when

• Logs exist and show an exception. Read the exception.
• The run is marked failed, not crashed. Failed means the flow raised.

Steps

1. Check the worker that ran the flow: is it alive, and what do its logs say around the crash time? Note the worker state. Verify: the worker is confirmed alive or dead
2. Look at the infrastructure: container runtime events, node pressure, OOM kills. Note any kill signal. Verify: the kill is found or ruled out
3. Verify log shipping config: where logs go and whether the crashed run ever opened the stream. Note the log path. Verify: the log plumbing is checked
4. Fix the underlying cause: memory limits, eviction policies, worker stability. Note the cause is addressed. Verify: the root cause is fixed
5. Re-run the flow and confirm logs stream normally. Note logs appear. Verify: logging is verified

Variant phrasings

prefect flow run crashed no logs

The compact search.

prefect crashed vs failed

The state-meaning question.

Compatibility: Prefect 2.x. Crashed state semantics and log shipping config are stable across 2.x.

Why it happens

Prefect distinguishes crashed (the process died unexpectedly) from failed (the flow raised an exception), and crashed with no logs points below the flow: the worker, the container, the node. Log shipping is set up by the worker process, so a worker that dies during startup never opens the stream. The empty log page is not missing data; it is the signature of an infrastructure kill. Chasing the flow code for this symptom wastes the investigation.

Edge cases / pitfalls

• Flows that fork subprocesses can orphan them when the parent is killed, leaving the run crashed while work secretly continues. Ensure cleanup.
• Log retention or shipping delays can make a crashed run look logless for minutes. Wait before concluding the logs are truly empty.
• Spot or preemptible nodes produce exactly this pattern at scale. If crashes correlate with node churn, that is the cause.
• A worker with a full disk cannot write logs or state. Check disk before assuming memory.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_46OkErmKFY1FhhH3njdqiQ