airflow Variable.get returns None, task failed

TL;DR

Variable.get returns None when the variable does not exist in the environment the task runs in, when the stored value failed to deserialize, or when you relied on a default that masks the miss. Check the variable in the UI or CLI from the worker's environment first. Store JSON values as valid JSON and parse them explicitly instead of trusting implicit deserialization.

airflow Variable.get returns None, task failed

Use this when

• Variable.get returns None for a variable that exists in the UI.
• It works on the scheduler but not on workers.
• A JSON variable comes back as None instead of a dict.

Not for this skill when

• You need connection credentials. Variables and connections are different stores.
• The variable exists but the value is wrong. That is a content problem, not a lookup problem.

Steps

1. Fetch the variable with airflow variables get from the worker's environment, not just the UI. Note whether it exists there. Verify: environment mismatch is confirmed or ruled out
2. Check the stored value's format: valid JSON for JSON variables, plain string otherwise. Note any malformed JSON. Verify: serialization problems are found
3. Search the task code for default_var usage that silently returns None on a miss. Note every silent default. Verify: hidden misses are exposed
4. If you use a secrets backend for variables, verify the task reads the same backend you wrote to. Note the read and write paths match. Verify: backend mismatch is ruled out
5. Fix the value or the lookup, then re-run the single task and confirm it gets the expected value. Note the task proceeds. Verify: the lookup works

Variant phrasings

airflow variable.get none

The compact phrasing.

airflow variable not found worker

The environment-mismatch search.

Compatibility: Airflow 2.x. Variable storage moved to the secrets-backend framework; JSON variables need valid JSON in all versions.

Why it happens

Variables are resolved at task runtime in the worker's environment, and deserialization happens then too. A value that looks fine in the UI can fail JSON parsing on the worker, and get with a default silently converts a missing variable into a None that blows up three steps later. The UI shows you the stored string; the worker shows you what deserialization produced. Those are different things.

Edge cases / pitfalls

• default_var=None is the default behavior, which means a typo in the variable name surfaces as a NoneType error far from the lookup. Prefer raising explicitly.
• Variables over a few KB get slow to fetch on every task. Cache them or move big config to files.
• The UI test-render uses the webserver environment. A variable visible there can still be missing for workers.
• Secrets backends cache variable values. After changing a variable, allow for cache TTL or restart workers.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_mvb-AswjQFQjjh15yl0Bbg