DAG import errors: finding the broken file fast

TL;DR

Run airflow dags list-import-errors to get the exact broken file and its full traceback; the scheduler skips broken files silently, so a vanished DAG usually means a bad import, not a missing file. Reproduce with plain python on the file for a fast fix loop, fix the traceback top-down, and re-run until the list is empty.

DAG import errors: finding the broken file fast

Use this when

• A DAG disappeared from the UI with no obvious error
• The scheduler log mentions import errors
• A deploy just went out and parsing broke

Not for this skill when

• The DAG parses fine but its tasks fail at runtime
• The scheduler is slow but parsing succeeds
• The DAG parses but never gets scheduled (scheduling config issue)

Steps

1. Ask Airflow which files are broken:

airflow dags list-import-errors

Expected output: each broken filename with its full traceback. This is the single fastest diagnostic; everything else is slower.

2. Reproduce the failure directly for a tight feedback loop:

python [dag folder]/my_dag.py

Expected output: the same traceback, in under a second, without waiting for the scheduler's parse cycle. Iterate here, not in the UI.

3. Read the traceback top-down and fix the first error first:

python [dag folder]/my_dag.py 2>&1 | head -30

Expected output: the head of the traceback names the real failure (missing package, bad import, syntax error). Later frames are usually consequences; fix the first one and re-run.

4. Check the usual suspects if the traceback is cryptic: top-level database calls, queries at import time, missing third-party packages, circular imports between DAG files:

pip show [package name] 2>/dev/null || echo "package missing"

Expected output: confirmation whether the package the traceback names is actually installed in the scheduler's environment.

5. Move side effects out of top-level DAG code: no queries, no API calls, no heavy computation at import time:

# bad: runs on every parse
# rows = query_database()
# good: runs only when the task executes
def fetch():
    return query_database()

Expected output: the file imports in milliseconds with zero side effects, which also speeds up the whole scheduler parse loop.

6. Re-run the import-error check and confirm the DAG appears:

airflow dags list-import-errors && airflow dags list | grep my_dag

Expected output: empty error list and your DAG in the list. If the UI still doesnt show it after a parse interval, the webserver may be reading a stale cache.

Variant phrasings

airflow dag not showing up no error

There is an error, it is just captured per-file by the DAG processor instead of crashing anything. list-import-errors surfaces it; without that command you are guessing.

broken dag import traceback points at a shared module

One bad shared module poisons every DAG that imports it, so a single fix can restore many DAGs at once. Check which DAGs import the module before declaring victory, and consider whether shared code belongs in a tested package rather than the DAG folder.

list-import-errors is empty but dag still missing

Check .airflowignore: the file may be excluded from parsing entirely. Also verify the filename ends in .py and the DAG object is assigned at module top level; a DAG defined inside a function never gets collected.

Why it happens

The DAG processor imports every Python file in the DAG folder on a schedule, in isolation per file. An exception during import is caught, recorded per file, and the file is skipped; the scheduler never sees it. This design keeps one bad file from taking down parsing of everything else, but it means failures are silent unless you ask for them explicitly.

Edge cases

• A syntax error in one file and a missing package in another produce two entries; fix all of them, not just the first.
• The scheduler and the webserver can parse with different environments; an error visible in one may not appear in the other.
• Import errors from the previous parse cycle can linger briefly after you fix the file; wait one min_file_process_interval before re-checking.
• Files starting with underscore or in ignored directories are skipped silently by design; that is .airflowignore doing its job, not a bug.
• Top-level Variable.get() calls fail at parse time if the metadata DB is unreachable, producing import errors that look like code bugs but are really infrastructure issues.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_M7PbgtTxUoYhtLuqUKUgQQ