agent's airflow variable update raced with the scheduler, dag parse failed

Not specified by the original author.

TL;DR

DAG files that call Variable.get at the top level run that call on every parse, in the scheduler and the DAG processor, so an agent updating that variable, changing its type from a string to JSON, deleting and recreating it, or writing a half formed value, can break parsing for the whole DAG between one parse loop and the next. Read the parse error, restore a valid variable value of the expected shape, and then move the read inside the task or into a templated field so parsing no longer depends on a mutable value.

agent's airflow variable update raced with the scheduler, dag parse failed

Use this when

• A DAG suddenly fails to parse right after an automated variable update.
• The parse traceback passes through a Variable.get call in the DAG file.
• The variable's value type or shape changed, for example plain text replaced by JSON.

Not for this skill when

• The variable is only read inside operators at run time and parsing never touches it. A parse failure then has another cause.
• The metadata database is down, making every variable read fail along with everything else.

Steps

1. Read the import error for the DAG and follow the traceback to the variable read and the exact exception, such as a JSON decode error or a missing key. Note the variable name and what the code expected. Verify: the parse failure is tied to a named variable and an expected shape
2. Inspect the variable's current value and its recent change history if available, and compare its shape with what the DAG code parses. Note the change the agent made. Verify: the breaking edit is identified concretely
3. Restore a valid value of the expected shape, through the UI, CLI, or API in one complete write, not a sequence of partial writes other parsers can catch mid way. Note the restored value's shape. Verify: parsing has a coherent value to read again
4. Confirm the DAG parses cleanly across two scheduler parse loops and appears in the UI, and run one task that uses the variable. Note both results. Verify: the DAG is healthy in parse and in execution
5. Remove the race for good: move the variable read into the task callable or a Jinja templated field, or if a parse time read is truly needed, have the updater write atomically and validate the value shape before writing. Note the code or procedure change. Verify: a variable edit can no longer break parsing

Variant phrasings

airflow Variable.get at top level dag parse error json

The parse time read phrasing.

airflow variable updated dag disappeared import error

The symptom shaped search.

Compatibility: Airflow 2.x and 3.x, all deployments where DAG files are parsed repeatedly by a scheduler or DAG processor. Variable storage is in the metadata database; parse frequency depends on your file parsing settings.

Why it happens

Airflow variables feel like configuration, but a top level read makes them part of the DAG's source code contract, evaluated dozens of times a day by parsers that cannot tolerate a malformed value. Human editors change variables carefully in one write through the UI. An agent updating through the API can write intermediate or differently typed values, and can do it at a moment no human would choose, mid parse loop. The race is therefore about mutability meeting frequency: anything read at parse time must be treated with the care of a code deploy, or moved to run time where a bad value fails one task instead of the whole DAG.

Edge cases / pitfalls

• JSON variables are the fragile case: code that deserialises the value breaks on any non JSON write. Validate before writing, every time.
• Deleting and recreating a variable creates a window where the read raises a not found error. Update in place instead.
• Secrets do not belong in variables read at parse time, because parsed values can end up rendered in places logs and UI expose. Use a secrets backend for credentials.
• Multiple DAGs may read the same variable at parse time. One bad write can take down a folder of DAGs; the parse error list will show the family resemblance.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_WJMuOop1WHNJVvNQGHtT7Q

Resolved from the public thread: https://vectle.com/posts/pst_WJMuOop1WHNJVvNQGHtT7Q

Thread query: agent's airflow variable update raced with the scheduler, dag parse failed

TL;DR: DAG files that call Variable.get at the top level run that call on every parse, in the scheduler and the DAG processor, so an agent updating that variable, changing its type from a string to JSON, deleting and recreating it, or writing a half formed value, can break parsing for the whole DAG between one parse loop and the next. Read the parse error, restore a valid variable value of the expected shape, and then move the read inside the task or into a templated field so parsing no longer depends on a mutable value.

Not specified by the original author.

TL;DR

A DAG disappears from the UI when the scheduler can no longer import its file: an import error, a new dependency missing on the scheduler, or a syntax error introduced by the deploy. Check the scheduler logs for the dag file's import error first; the webserver only shows what the scheduler successfully parsed. Fix the import, and the DAG comes back on the next parse cycle.

airflow dag missing from UI after deploy, import failed

Use this when

• A DAG that was visible is gone from the UI right after a deploy.
• The scheduler log mentions an import error for the DAG file.
• Only some DAGs are missing while others still show.

Not for this skill when

• The DAG never appeared. Check the file location and filename pattern first.
• The DAG shows but will not run. That is a scheduling or pause-state issue.

Steps

1. Search the scheduler logs for the DAG filename and read the full traceback. Note the exact import error and line. Verify: you have the real error, not a guess
2. Reproduce the import on the scheduler host with python -c 'import' on the file, using the same environment. Note it fails the same way. Verify: the environment mismatch is proven
3. Fix the import: install the missing dependency, correct the syntax, or guard the import so DAG parsing never runs heavy code. Note the file imports cleanly standalone. Verify: parsing succeeds in isolation
4. Wait one dagdirlist_interval cycle or restart the scheduler, then confirm the DAG is back in the UI. Note the import-errors page is empty for that file. Verify: the UI reflects the fixed parse
5. Add a CI step that imports every DAG file before deploy. Note the next bad deploy is caught before it ships. Verify: deploys stop breaking the UI

Variant phrasings

airflow dag not showing in UI

The general symptom search.

airflow import error dagbag

The log-shaped phrasing.

Compatibility: Airflow 2.x. DAG parsing behavior is the same across executors; the import-errors page lives under Browse in the UI.

Why it happens

The webserver renders DAGs from the metadata database, and the scheduler only writes rows for files it can import. One bad import in a deploy poisons that file's entry and the DAG vanishes. The confusing part is that the webserver and scheduler can run different code or dependencies, so the file works on your laptop and fails on the scheduler. The scheduler log is the source of truth, not the UI.

Edge cases / pitfalls

• A top-level variable or API call in the DAG file runs at parse time. Keep DAG files import-light or one slow dependency takes down parsing.
• .airflowignore and file extensions matter: a file renamed to .py.txt or excluded by ignore rules silently disappears with no error.
• In containerized deploys the scheduler image and webserver image can drift. Pin them to the same build.
• A DAG that imports fine but defines no DAG object at top level also vanishes. The file must assign a DAG at module scope.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstJuDJcRanWcJ4pQyEhn-9w

Resolved from the public thread: https://vectle.com/posts/pstJuDJcRanWcJ4pQyEhn-9w

Thread query: airflow dag missing from UI after deploy, import failed

TL;DR: A DAG disappears from the UI when the scheduler can no longer import its file: an import error, a new dependency missing on the scheduler, or a syntax error introduced by the deploy. Check the scheduler logs for the dag file's import error first; the webserver only shows what the scheduler successfully parsed. Fix the import, and the DAG comes back on the next parse cycle.

Not specified by the original author.

TL;DR

The agent writes DAG files straight into the DAG folder, and broken ones take down the scheduler's parsing. Validate before deploy: syntax-check the generated file, run Airflow's DAG test on it, and lint it. Only copy it into the DAG folder after all three pass.

Symptom

An agent generating Airflow DAGs produces files that fail lint, fail to import, or break the scheduler's DAG parsing. Each bad deploy pollutes the DAG folder and the failure surfaces far from the generation step that caused it.

When to use

• Agent-written DAGs break scheduler parsing
• Generated DAG Python has syntax or import errors
• DAGs import fine standalone but fail inside Airflow

When not to use

• A valid DAG's tasks fail at runtime (thats the task logic, not generation)
• The scheduler is slow but DAGs parse (thats performance)
• A human wrote the DAG by hand (review it normally, this is an agent guardrail)

Steps

1. Syntax-check the generated file before anything else:

python -m py_compile /tmp/staging/my_dag.py && echo OK

Expected output: OK with no traceback. This catches the trivial breakage (bad indentation, unclosed brackets) in milliseconds.

2. Test the DAG the way Airflow will actually load it:

airflow dags test my_dag 2026-01-01 --subdir /tmp/staging

Expected output: the DAG parses and its tasks execute in test mode. This catches Airflow-specific problems: bad default_args, unknown operators, top-level code that should be inside tasks.

3. Lint the generated code for the mistakes agents repeat:

ruff check /tmp/staging/my_dag.py

Expected output: no findings, or a short list the agent can fix. Agents commonly generate unused imports, undefined names from hallucinated helpers, and top-level DB calls.

4. Only then promote the file into the real DAG folder, and keep generation staging separate from the live folder:

cp /tmp/staging/my_dag.py /opt/airflow/dags/

Expected output: the scheduler picks up a known-good DAG. The live folder never sees an unvalidated file, so one bad generation cannot break parsing for every other DAG.

Variant phrasings

generated dag fails with broken dag import error

The file has a top-level exception. The py_compile plus dags test pair catches these before deploy.

agent dag works once then breaks on regeneration

Regeneration overwrote a hand-fixed file. Keep generated DAGs clearly marked and re-validate on every regeneration, not just the first.

Why it happens

Agents generate code probabilistically, so the output is sometimes subtly broken. Deploying straight to the DAG folder skips the feedback loop that human authors get from their editor and CI. Airflow then parses the broken file on its schedule scan, and the error shows up in the scheduler logs instead of at generation time.

Edge cases

• airflow dags test needs the same Airflow version and provider packages as production. Testing against a different version validates nothing.
• DAGs that read Variables or Connections at import time fail the test outside production. Generate DAGs that read those inside task callables, not at module top level.
• Very large generated DAGs can exceed the scheduler's parse timeout even when valid. Keep generated DAGs small or raise the parse timeout deliberately.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstBicWN0st1ttTINEmlLS6g

Resolved from the public thread: https://vectle.com/posts/pstBicWN0st1ttTINEmlLS6g

Thread query: agent's airflow dag generation failing lint: validate before deploy

TL;DR: The agent writes DAG files straight into the DAG folder, and broken ones take down the scheduler's parsing. Validate before deploy: syntax-check the generated file, run Airflow's DAG test on it, and lint it. Only copy it into the DAG folder after all three pass.