Skip to content
← Back to activity

Public problem

Meltano with Airflow fails: sqlite database file doesn't exist, do you need to create the database with...

Posted by @issueatlas · Outcome: unknown

Originally posted by issueatlas · Updated

After upgrading to Meltano 2.0, running Airflow failed with an error saying the sqlite database file didn't exist and asking whether to run airflow db init. The reporter had Airflow 1.10 configured and then tried 2.1, but the metadata DB setup kept breaking. A maintainer walked through the migration: Airflow 2.x replaced the old AIRFLOW__CORE__SQL_ALCHEMY_CONN with AIRFLOW__DATABASE__SQL_ALCHEMY_CONN, and the postgres connection scheme had to be postgresql:// (with the l), not postgres://. The working setup was Airflow 2.3.2 with a fresh metadata database, the new env var names, and the postgresql:// scheme. The reporter confirmed this combination worked and thanked the maintainers.

Finding relevant solutions

This problem is public. Solutions will appear here as matching completes. This page checks automatically while it’s open.

History & evidence

Original discussion

Preserved replies and documents. Ordinary replies are not outcome reports.

issueatlas · · Reply permalink

Meltano 2.0 changed how Airflow config is passed through, and Airflow 2.x renamed the database env vars, so the old settings silently stopped pointing at a real metadata DB. Fix: use Airflow 2.3.2, set AIRFLOW__DATABASE__SQL_ALCHEMY_CONN (the 2.x name) with a fresh metadata database, and use the postgresql:// connection scheme with the l, since postgres:// is no longer accepted. Delete or re-init the stale sqlite metadata file so the new connection is what Airflow actually uses. The reporter used this exact combination and confirmed Airflow started cleanly. Source: https://github.com/meltano/meltano/issues/6209

Source: https://github.com/meltano/meltano/issues/6209