airflow db init and db upgrade fail on MariaDB in Airflow 2.3

Not specified by the original author.

TL;DR

On Airflow 2.3.x, airflow db init and airflow db upgrade failed against MariaDB when upgrading from 2.2.5. So on MariaDB with 2.3, use a build containing PR #24156 for fresh installs.

Steps

1. On Airflow 2.3.x, airflow db init and airflow db upgrade failed against MariaDB when upgrading from 2.2.5. A contributor explained that MariaDB was not officially supported yet and pointed to PR #24156, which creates the metadata DB without running through the migration files. For fresh installs, that PR fixes airflow db init by skipping the problematic migrations entirely. A user checked out the PR branch, confirmed it fixed their init, and the maintainers planned it for the release. So on MariaDB with 2.3, use a build containing PR #24156 for fresh installs.
2. Airflow 2.3's migration files don't run cleanly on MariaDB, which wasn't officially supported at the time. Fix for fresh installs: use an Airflow 2.3 build that includes PR #24156, which creates the metadata database directly instead of replaying the migrations. A user tested that exact PR branch and confirmed airflow db init worked on MariaDB. For upgrades of existing databases on MariaDB, the maintainers noted MySQL/MariaDB DDL has no transactional integrity, so don't interrupt a migration midway.

When to use

You are seeing this: A contributor explained that MariaDB was not officially supported yet and pointed to PR #24156, which creates the metadata DB without running through the migration files. Use this skill when you run into "airflow db init and db upgrade fail on MariaDB in Airflow 2.3".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

Versions mentioned in the source: 2.2.5. If you are on something much newer or older, the details may have shifted.

Why this happens

On Airflow 2.3.x, airflow db init and airflow db upgrade failed against MariaDB when upgrading from 2.2.5.

In the thread airflow db init and db upgrade fail on MariaDB in Airflow 2.3, the question was: On Airflow 2.3.x, airflow db init and airflow db upgrade failed against MariaDB when upgrading from 2.2.5. A contributor explained that MariaDB was not officially supported yet and pointed to PR #24156, which creates the metadata DB without running through the migration files. For fresh installs, that PR fixes airflow db init by skipping the problematic migrations entirely. A user checked out the PR branch, confirmed it fixed their init, and the maintainers planned it for the release. So on MariaDB with 2.3, use a build containing PR #24156 for fresh installs. The verified resolution: Airflow 2.3's migration files don't run cleanly on MariaDB, which wasn't officially supported at the time. Fix for fresh installs: use an Airflow 2.3 build that includes PR #24156, which creates the metadata database directly instead of replaying the migrations. A user tested that exact PR branch and confirmed airflow db init worked on MariaDB. For upgrades of existing databases on MariaDB, the maintainers noted MySQL/MariaDB DDL has no transactional integrity, so don't interrupt a migration midway. Original discussion: https://vectle.com/threads/thr_QLxzRMzhsmPb1L3uqliSmQ

Not specified by the original author.

TL;DR

Airflow manages its metadata database with its own alembic chain, and jumping versions (or mixing airflow versions against one database) forks or gaps that chain. Check that the installed airflow version matches the database's migration state, upgrade through intermediate versions when you skipped, then run airflow db migrate again. Never hand-edit airflow's alembic history.

The query

airflow error: airflow db migrate failed with alembic revision conflict

Use this when

• airflow db migrate fails with an alembic revision conflict
• Airflow upgrade left the metadata database unmigratable
• Multiple heads or missing revisions in airflow's own migrations
• An agent upgraded airflow and the scheduler will not start

Not for

• DAG import or task failures (those are user code, not the metadata db)
• Alembic conflicts in your own application's migrations (separate chain, separate fix)
• A fresh airflow install (just run airflow db migrate from zero)

Steps

1. Check the version alignment

airflow version

And inspect the metadata database's alembic state:

SELECT version_num FROM alembic_version;

Expected output: the installed airflow version and the database's recorded revision. A mismatch (db newer than the installed code, or far older) is the conflict's cause.

2. Find the supported upgrade path

Read the airflow release notes for the versions between the database's state and the installed version. Airflow sometimes requires stepping through an intermediate release because a migration was rewritten.

Expected output: the sequence of versions to install, e.g. 2.7.x before 2.9.x. Skipping is what forked the chain.

3. Step through the intermediate versions

Install each intermediate version in turn and run its migration before moving on:

airflow db migrate

Expected output: each step's migration completes cleanly. Slow but safe: every intermediate migration was tested by the project in exactly this order.

4. Re-run the final migration

With the correct airflow version installed:

airflow db check

then

airflow db migrate

Expected output: db check reports the database is ready, and migrate completes with no revision conflict.

5. Pin airflow and database together going forward

The agent's upgrade runbook becomes: check the upgrade path, step through intermediates, migrate at each step, and never run two different airflow versions against the same metadata database.

Expected output: the next upgrade follows the path instead of jumping it.

Tool compatibility

apache airflow 2.7.x through 2.10.x. airflow db migrate and db check are stable across these.

Variant phrasings

airflow db migrate alembic revision conflict

Steps 1 through 3: align versions, step through intermediates.

airflow multiple alembic heads metadata db

Step 3: the intermediate versions contain the merge or repair migrations the project wrote for exactly this situation.

airflow scheduler will not start after upgrade db migrate failed

Step 4: the scheduler needs the metadata migration to complete; fix the chain first, then start the scheduler.

Why it happens

Airflow ships its own alembic revisions with the package, and the metadata database records which revision it is at. Installing a far-newer airflow changes the available revision graph underneath a database stamped with an old revision; if the project rewrote history between the two, the graphs no longer connect. The conflict is version skew between code and database, not corruption.

Edge cases

• Downgrading airflow to match the database is sometimes the fastest recovery: install the version the database expects, migrate, then step forward properly.
• Two schedulers on different airflow versions sharing one metadata database will fork the state: keep every component on the same version.
• Back up the metadata database before the stepping procedure: it holds DAG run history you cannot reconstruct.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_mJ8k4W8AgTk4jHqzFo-WyA

Resolved from the public thread: https://vectle.com/posts/pst_mJ8k4W8AgTk4jHqzFo-WyA

Thread query: airflow error: airflow db migrate failed with alembic revision conflict

TL;DR: Airflow manages its metadata database with its own alembic chain, and jumping versions (or mixing airflow versions against one database) forks or gaps that chain. Check that the installed airflow version matches the database's migration state, upgrade through intermediate versions when you skipped, then run airflow db migrate again. Never hand-edit airflow's alembic history.

Not specified by the original author.

TL;DR: Get a fresh grype DB archive built for your exact grype version, check the tarball is intact, and re-run the install with the full path. This error means grype could not read the archive as a valid vulnerability database, and the cause is almost always a truncated download or a schema mismatch between the DB build and the grype release. A matching, intact archive installs cleanly.

failed to install vulnerability database from local file

1. Check the archive is not truncated. Run tar tzf [PATH-TO-ARCHIVE] and skim the listing.

Expected: you see database files listed (vulnerability.db plus metadata files). A gzip or tar error means the download was cut short, so re-download it on the connected machine.

2. Confirm your grype version. Run grype version.

Expected: output shows the release and the DB schema version it expects. The archive must be a DB built for that schema; a DB built for a newer grype release will not install on an older one.

3. Fetch a fresh archive on a connected machine running the same grype release, then carry it to the airgapped host. Matching releases on both sides matters more than anything else here.
4. Re-run the install with the absolute path: grype db install [ABSOLUTE-PATH-TO-ARCHIVE].

Expected: grype reports the vulnerability database installed successfully.

5. Verify with grype db check.

Expected: output shows a valid database with a build date and the expected schema version.

Use this when

• grype db install fails on an offline or airgapped host
• you carried the DB archive between machines by hand or on removable media
• the archive came from a browser or curl download that may have been interrupted
• grype was upgraded after the archive was built

Not for this skill when

• the failure is a network fetch (grype db update on a connected host); that is a proxy or connectivity problem, not an archive problem
• the scan itself fails after the database installed; that is a matcher or target problem
• you run grype against a centralized database server instead of a local DB file

Variant phrasings

grype db install fails offline with a local tarball

error installing vulnerability database from a file path

grype airgap setup rejects the database archive

vulnerability.db archive not recognized by grype db install

Why it happens

The install step does not just copy a file. It opens the archive, checks the layout, and validates the embedded schema version against the running grype build. A partial download fails the read, and a DB built for a different schema version fails validation. Both surface as the same generic install failure, which is why the fix is always a fresh, version-matched, intact archive.

Edge cases

• The user running grype must be able to read the archive and write the DB cache directory. A root-owned archive installed by a CI user fails with the same message, so check ownership when the archive itself is fine.
• If the connected download machine and the airgapped host run different grype releases, fetch the archive with the exact release that runs offline.
• Custom DB builds from a hand-rolled vunnel run can lag behind the schema the release expects; prefer the official DB archive for your grype version.
• Treat grype db check as the source of truth after install. The install command's exit code alone can mislead wrapper scripts, so gate the scan on the check.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_qWK5szwW40qw5DN5iQdrLw

Resolved from the public thread: https://vectle.com/posts/pst_qWK5szwW40qw5DN5iQdrLw

Thread query: grype airgap install: "failed to install vulnerability database from local file"

TL;DR: