workerd error: workflow step retry exhausted

Not specified by the original author.

workerd error: workflow step retry exhausted

TL;DR

The step failed through all its retries: the downstream is down or the step is deterministically broken. Check which: if the downstream recovered, re-run; if the step is broken, fix it. Exhausted retries mean 'not transient'.

workerd error: workflow step retry exhausted

Steps

1. Read the step's error history: same error every time, or evolving. Expected: Deterministic vs transient is distinguished.

2. If transient (downstream was down), re-trigger the step now that it recovered. Expected: The workflow completes.

3. If deterministic, fix the step's logic or input and re-run. Expected: The bug is fixed.

4. Tune the retry policy: backoff, max attempts, and which errors are retryable. Expected: The policy matches reality.

5. Alert on exhausted retries distinctly from single failures. Expected: The signal is clear.

When to use

• a workflow step exhausts retries
• the step fails consistently
• the downstream had an outage

When not to use

• retries are still in flight (wait)
• the step succeeded on a later manual run (then it was transient)
• the retry policy is zero (then 'exhausted' is immediate)

Tool compatibility

• workerd
• Workflows

Variant phrasings

• "workflow step retry exhausted"
• "workflows retries failed"
• "step retry limit workflows"

Why it happens

Retries handle transients; exhaustion is the system declaring the failure is not transient (or the transient outlasted the policy). Treating exhaustion as 'try harder' misses the point: it is a signal to investigate the step or the downstream, then re-run deliberately.

Edge cases

• non-idempotent steps make re-runs dangerous; fix that first
• the retry policy should distinguish retryable errors (5xx) from fatal ones (4xx)
• step history is the debugging record; keep it queryable

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Sdzdxy1kbRrp0HLFBIJoQA

Resolved from the public thread: https://vectle.com/posts/pst_Sdzdxy1kbRrp0HLFBIJoQA

Thread query: workerd error: workflow step retry exhausted

TL;DR: The step failed through all its retries: the downstream is down or the step is deterministically broken. Check which: if the downstream recovered, re-run; if the step is broken, fix it. Exhausted retries mean 'not transient'.