rails migration hit a timeout acquiring the advisory lock because another deploy was running db:migrate at the same...

TL;DR: Do not fight the other deploy - let it finish. Rails serializes migrations with a Postgres advisory lock, so a second db:migrate waits and then times out. Confirm the competing migration in pgstatactivity, wait for it to complete, then make sure only one release task runs migrations so the race stops happening.

rails migration hit a timeout acquiring the advisory lock because another deploy was running db:migrate at the same time - the agent needs a lock strategy

1. Confirm the competing migration is real and identify it.

SELECT pid, usename, query, now() - query_start AS running_for
FROM pg_stat_activity WHERE query ILIKE '%migrate%' AND pid != pg_backend_pid();

Expected: one row showing the other deploy's migration query and how long it has been running.

2. Wait for it to finish instead of killing it. Poll the state table until it stops changing.

SELECT version FROM schema_migrations ORDER BY version DESC LIMIT 3;

Expected: the versions stop changing and the competing backend disappears from pgstatactivity, meaning its migration completed.

3. Re-run your migration after the other deploy is done.

Expected: the advisory lock is acquired immediately and the migration completes without a timeout.

4. Fix the root cause: run migrations from exactly one place. Move db:migrate out of web dyno boot or multi-instance release scripts and into a single release-phase task that runs once per deploy.

Expected: you can describe the one place migrations run, and concurrent deploys no longer start competing migrates.

5. Tune the advisory lock timeout to your longest legitimate migration so slow-but-healthy migrations do not get killed, and add deploy queueing if you ship many times per hour.

Expected: lock timeouts only fire on genuine contention, not on slow migrations.

Use this when

• A Rails migration fails or hangs acquiring the advisory lock during deploy
• Two deploys or two dynos are running db:migrate at the same time
• You see a lock timeout mentioning the migration advisory lock
• You need a strategy so concurrent deploys stop racing on migrations

Not for this skill when

• The lock is held by a crashed process that will never release it - advisory locks release when the session disconnects, so check for a stale session and terminate it
• The migration itself is slow (rewriting a big table) rather than contended - that is a migration-design problem
• You are not on Postgres - the advisory lock strategy is Postgres-specific; MySQL Rails migrations serialize differently
• A single deploy is timing out with no competitor - look at locktimeout and statementtimeout settings instead

Variant phrasings

• rails ConcurrentMigrationError another migration process running
• db migrate advisory lock timeout during deploy
• two deploys running db migrate at once
• rails migration lock strategy concurrent deploys

Why it happens

ActiveRecord wraps migrations in a Postgres advisory lock so two migrates cannot interleave DDL. The lock is session-scoped: whoever holds it migrates, everyone else waits. When two deploys each run db:migrate - common with multi-dyno release scripts or overlapping CI deploys - the second waits on the advisory lock until its timeout fires. The timeout is doing its job; the bug is that two writers were started.

Edge cases

• Advisory locks are released when the holding session disconnects. A deploy killed mid-migration frees the lock, but its migration may be half applied - verify schema_migrations before re-running.
• Very long migrations can exceed the lock timeout on their own. Distinguish contention (a competitor exists) from slowness (no competitor, still timing out).
• Running migrations in web boot means every dyno restart races. The release-phase task is the fix, not a longer timeout.
• If deploys overlap constantly, serialize at the deploy layer with a queue rather than relying on the database lock to sort it out.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_StbQZ2pvmQncXP1mKo7F3g