agent's airflow backfill command timed out before any dag runs were created

TL;DR

An agent that shells out to the Airflow backfill command inherits the command's worst property: it parses every DAG, plans the whole range, and can run for a long time before the first run appears, so a tool timeout kills it with nothing created and no record of progress. Shorten the range into chunks the agent runs one at a time, raise or remove the tool timeout for the planning step, or trigger runs through the Airflow REST API per interval so each run creation is a small, checkable call.

agent's airflow backfill command timed out before any dag runs were created

Use this when

• An automated backfill times out and the DAG runs list for the range is empty.
• The same backfill works by hand in a terminal with no timeout.
• The agent reports a timeout but Airflow logs show parsing still in progress when it was killed.

Not for this skill when

• Runs were created and then tasks failed. That is a task problem inside a working backfill.
• The DAG has catchup or a timetable that intentionally creates no runs for that range. Check the schedule semantics first.

Steps

1. Confirm nothing was created: list DAG runs for the target range in Airflow and note the count is zero, and check whether a backfill job record exists. Note the baseline before retrying. Verify: you know the retry starts from a clean slate, not a partial one
2. Measure the planning time by running the backfill command with a dry run or a tiny one day range in a shell and timing parse plus planning. Note how long one day takes. Verify: the per interval cost is measured, so chunk size can be chosen on evidence
3. Split the requested range into chunks sized to finish planning and creation well inside the agent timeout, for example a week at a time, and record progress per chunk outside the agent's memory. Note the chunk list. Verify: each attempt is small enough to complete and resumable
4. Alternatively, create runs through the REST API one interval at a time, checking each response before the next. Note the run id returned for each interval. Verify: run creation becomes many small verified calls instead of one long one
5. Run the chunks in order, verifying after each that its runs exist in Airflow before starting the next, and stop on the first chunk that creates nothing. Note the final run count for the whole range. Verify: the full range is covered with proof per chunk

Variant phrasings

airflow backfill cli hangs parsing dags agent timeout

The CLI planning hang phrasing.

airflow create backfill runs via rest api instead of cli

The API alternative searched directly.

Compatibility: Airflow 2.x backfill command and DAG run creation through the REST API; applies to any agent or script driving Airflow with an outer execution timeout. Backfill behaviour changes in Airflow 3, where backfills are a first class API resource.

Why it happens

The backfill command was built for a human at a terminal who can wait, not for a caller with a hard timeout. It front loads parsing and planning for the entire range, so wall clock time before the first visible run grows with both DAG folder size and range length. An agent watching for output or a result sees silence, hits its limit, and kills the process during the silent phase. From the agent's side it looks like Airflow ignored the command; from Airflow's side the command never got to speak.

Edge cases / pitfalls

• Killing a backfill midway in a chunk can leave some runs created in that chunk. Always re-list runs for the chunk before retrying it.
• Creating runs via the API bypasses some backfill bookkeeping. Record the run ids your automation created so a later cleanup can find them.
• A very large DAG folder makes every CLI invocation slow, including status checks. Factor parse time into any agent timeout around Airflow commands.
• Do not parallelise chunks against the same DAG until you know maxactiveruns will not just queue them into confusion.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstV6tPgytMLah3vWLRM-dcQ