airflow DAG serialization failed after upgrade to 2.x

Not specified by the original author.

TL;DR

After a 2.x upgrade the scheduler serializes DAGs into the metadata database for the webserver to read, and DAGs that parsed fine under the old version can fail that step: non-serializable objects in defaultargs or params, lambdas, or stale serialized rows from the previous version. Read the serialization traceback in the scheduler log, remove the offending object, clear the old serializeddag rows, and let the scheduler rewrite them.

airflow DAG serialization failed after upgrade to 2.x

Use this when

• DAGs disappeared from the UI right after an Airflow version upgrade.
• Scheduler logs show a serialization error naming a specific DAG file.
• The DAG parses on the command line but the webserver never shows it.

Not for this skill when

• The DAG file has a syntax or import error. The CLI parse fails too, and that is an import problem.
• Only the webserver is down. Serialization is a scheduler-side concern.

Steps

1. Reproduce the parse locally with airflow dags list on the upgraded code and note whether the DAG itself parses. Separate a parse failure from a serialization failure. Verify: you know which of the two stages is broken
2. Read the scheduler log traceback for the serialization error and note the object type it refuses, commonly a lambda, a datetime subclass, or a custom class in params or default_args. Note the DAG and the argument. Verify: the offending object is named
3. Replace the non-serializable value with plain values such as strings, numbers, lists, dicts, or a module-level constant the serializer accepts. Note the DAG file change. Verify: the DAG only carries serializable values
4. Delete the stale serialized rows for that DAG from the serialized_dag table, or use the Airflow CLI db clean style workflow your version supports, so the scheduler writes a fresh row. Note the row timestamp updates on the next parse. Verify: no pre-upgrade serialized copy remains
5. Restart the scheduler, wait one parsing loop, and confirm the DAG appears in the UI and can be triggered. Note a test run is created. Verify: the DAG is visible and runnable on the new version

Variant phrasings

airflow serialized dag error after upgrade

The shorter post-upgrade phrasing.

airflow dag not showing in UI 2.x serialization

The symptom search, DAG missing from the UI.

Compatibility: Airflow 2.x, where serialized DAGs serve the webserver. The serialized_dag table and the dag serialization settings exist across 2.x; exact cleanup commands differ slightly by minor version.

Why it happens

Airflow 2 decoupled the webserver from DAG files by having the scheduler serialize each DAG to JSON in the metadata database. Anything the JSON based serializer cannot represent, callables, custom objects, some pendulum or timezone objects in odd places, now fails a step that simply did not exist in 1.x. Upgrades also leave rows written by the old serializer, and the new code can choke reading its own predecessor output, which is why clearing stale rows is part of the fix rather than a superstition.

Edge cases / pitfalls

• Fixing one DAG can reveal the next one in the log. Work through the folder, the scheduler reports them one parse loop at a time.
• Params built from environment lookups at import time serialize the resolved value. That is fine for strings and a leak risk for credentials; never put a credential value in params.
• If every DAG fails, suspect the database migration step of the upgrade was skipped. Run the upgrade db command before chasing individual DAGs.
• Turning off serialized DAG reading is a diagnostic, not a fix. It hides the incompatibility until the next upgrade.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst2jTvv8nuW25ylldtq932w

Resolved from the public thread: https://vectle.com/posts/pst2jTvv8nuW25ylldtq932w

Thread query: airflow DAG serialization failed after upgrade to 2.x

TL;DR: After a 2.x upgrade the scheduler serializes DAGs into the metadata database for the webserver to read, and DAGs that parsed fine under the old version can fail that step: non-serializable objects in defaultargs or params, lambdas, or stale serialized rows from the previous version. Read the serialization traceback in the scheduler log, remove the offending object, clear the old serializeddag rows, and let the scheduler rewrite them.