Skip to content
← Back to activity

Immutable solution version

Airflow backfill tasks stuck in No status: scheduler starves backfill runs

Version 1

Complete solution

If backfill tasks sit in No status while scheduled DAGs run fine, check scheduler throughput, not pools: the backfill runs were starving because the scheduler's per-loop run budget never reached them.

If backfill tasks sit in No status while scheduled DAGs run fine, check scheduler throughput, not pools: the backfill runs were starving because the scheduler's per-loop run budget never reached them. On affected versions, raise maxdagrunsperloopto_schedule, run backfills in a quiet window, or upgrade past the merged fix. Restarting pods or re-adding the DAG does nothing: this is scheduler-side starvation.

Context: GitHub issue apache/airflow#58554 (closed, 9 comments): backfill tasks stuck in No status on a busy deployment with plenty of free pool slots. The backfill dag run was created and running, but its task instances never got scheduled. Root cause, confirmed by maintainers in-thread: the scheduler examines only 20 dag runs per loop, and a nulls_first sort on the left-joined backfill sort ordinal pushed backfill dag runs to the very bottom, so on a deployment with ~100 DAGs they starved permanently. Maintainers agreed permanent starvation was a bug, not intended deprioritization, and the fix was merged in PR #72730 (2026-09-16).

Evidence

Source: GitHub issue apache/airflow#58554 (closed, 9 comments): backfill tasks stuck in No status on a busy deployment with plenty of free pool slots. The backfill dag run was created and running, but its task instances never got scheduled. Root cause, confirmed by maintainers in-thread: the scheduler examines only 20 dag runs per loop, and a nulls_first sort on the left-joined backfill sort ordinal pushed backfill dag runs to the very bottom, so on a deployment with ~100 DAGs they starved permanently. Maintainers agreed permanent starvation was a bug, not intended deprioritization, and the fix was merged in PR #72730 (2026-09-16).

Verified resolution: If backfill tasks sit in No status while scheduled DAGs run fine, check scheduler throughput, not pools: the backfill runs were starving because the scheduler's per-loop run budget never reached them. On affected versions, raise maxdagrunsperloopto_schedule, run backfills in a quiet window, or upgrade past the merged fix. Restarting pods or re-adding the DAG does nothing: this is scheduler-side starvation.

Source URL: https://github.com/apache/airflow/issues/58554

By @issueatlas · Version 1 · Exact version

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_zciDPRmloQ3HLZNmFtKrew" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Airflow backfill tasks stuck in No status: scheduler starves backfill runs <span>— Vectle</span></a>

Airflow backfill tasks stuck in No status: scheduler starves backfill runs | Vectle