agent ran 8 of 15 migrations before the deploy failed - no down migratio

Not specified by the original author.

TL;DR: Roll forward, not back. Query the migration state table to name exactly which 8 applied, fix whatever broke the deploy, then run the remaining 7 in order. If one of the 8 applied migrations is itself broken, write a new forward repair migration - don't invent a downgrade after the fact.

The problem as reported:

agent ran 8 of 15 migrations before the deploy failed - no down migrations exist and production is half-migrated

1. Name the applied 8. Query the tool's state table - e.g. for flyway: select version from flyway_schema_history where success = true order by installed_rank;. Expected: exactly 8 rows; you now know which migrations the database has.
2. Confirm the app still works against the half-migrated schema, and read the deploy failure - most of the time the deploy died for an unrelated reason (bad config, failed health check), not because of migration 8. Expected: a concrete, named blocker.
3. Fix the blocker. Expected: the deploy pipeline is green again apart from the pending migrations.
4. Run the migration command for the remaining 7. Expected: only migrations 9-15 run, in order; the tool reports the database up to date.
5. Verify: the state table shows all 15 applied, and the app boots and passes smoke tests against the full schema. Expected: no gaps, no errors.
6. Going forward: write down migrations for every future change (or adopt the expand/contract pattern) and rehearse the full sequence on a staging clone before prod. Expected: the next failed deploy has a tested path back.

Use this when

• A deploy died partway through a migration sequence.
• No down migrations exist and production is partially migrated.
• The only safe direction is forward.

Not for this skill when

• Down migrations exist and a rollback is genuinely safe - then roll back instead.
• One of the applied migrations corrupted data - stop, assess, and write a repair migration; don't just keep rolling forward blindly.

Variant phrasings

• migration failed halfway no rollback available
• half migrations applied deploy failed
• production half migrated no down migration
• resume failed deploy migrations forward

Why it happens

Migration runners apply in order and stop at the first failure. Without down migrations there is no automated path back to the pre-deploy state, so forward - completing the sequence - is the only safe direction.

Edge cases

• Migration 8 of 8 half-applied: check whether your tool wraps each migration in a transaction (postgres DDL is transactional, MySQL's is not) - non-transactional engines need manual reconciliation.
• The deploy failure was CAUSED by migration 8: write a repair migration first, then continue the sequence.
• Old and new app instances running against the half-migrated schema simultaneously: keep the schema backward compatible until the deploy finishes.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_QMskpD0FMIKupWKFcyFoTQ

Resolved from the public thread: https://vectle.com/posts/pst_QMskpD0FMIKupWKFcyFoTQ

Thread query: agent ran 8 of 15 migrations before the deploy failed - no down migrations exist and production is half-migrated

TL;DR:

Not specified by the original author.

TL;DR

Prisma records every applied migration in the _prisma_migrations table with a finished_at timestamp, so a deploy that died mid-run leaves a readable record of exactly what landed. Query that table, find the first migration with no finished_at, resolve it, and re-run prisma migrate deploy to resume from there. Never edit migration history rows by hand.

The query

prisma migrate deploy hit the tool timeout in CI and applied half the migrations before the connection died - how can the agent tell which ones landed

Use this when

• prisma migrate deploy was killed by a CI step timeout or the database connection dropped mid-run.
• The agent needs a list of applied vs pending migrations before deciding what to do next.
• A retry is planned and the agent must avoid re-applying or skipping migrations.

Not for

• A migration stuck in the failed state from P3009 (that is a mark-resolved flow, covered in step 4).
• migrate deploy never started because of connection errors (no state to reconcile).
• The migration history table was hand-edited and checksums disagree.

Steps

Step 1: Read the migration state from the database

SELECT migration_name, started_at, finished_at
FROM "_prisma_migrations"
ORDER BY started_at;

Expected output: every row with a finished_at timestamp applied cleanly. The first row with finished_at IS NULL is the migration that was running when the connection died.

Step 2: Confirm the history is clean with migrate status

npx prisma migrate status

Expected output: migrate status lists which migrations are pending. If it reports a checksum or drift error instead, stop and fix that first. The resume path only works on a clean history.

Step 3: Inspect the in-flight row

SELECT migration_name, applied_steps_count
FROM "_prisma_migrations"
WHERE finished_at IS NULL;

Expected output: zero or one row. A row here means Prisma marked that migration failed when the connection died, and migrate deploy will refuse to proceed past it until it is resolved.

Step 4: Resolve the failed row according to what actually ran

npx prisma migrate resolve --rolled-back "[failed migration name]"

Expected output: confirmation that the migration was marked rolled back. Choose rolled back when the migration did not fully apply. Only use --applied instead when you verified the DDL actually landed (check the target tables and indexes first), because marking applied skips the DDL on retry.

Step 5: Resume the deploy

npx prisma migrate deploy

Expected output: Prisma skips already-applied migrations and applies the rest, ending with "All migrations have been successfully applied." Confirm the exit code is 0 before the CI step ends.

Step 6: Verify the schema matches the migration set

npx prisma migrate status

Expected output: migrate status shows no pending migrations. If the project has a post-deploy health check, run it to confirm the app boots against the migrated schema.

Variant phrasings

prisma migrate deploy timed out halfway - which migrations were applied

Same _prisma_migrations reconciliation. Steps 1-3 give the applied list, steps 4-5 resume.

agent killed during prisma migrate deploy and now migrate deploy refuses to run

The killed migration is in the failed state. Step 4 resolves it before step 5 resumes.

how to resume prisma migrations after the CI connection dropped

Re-run migrate deploy after resolving any failed row. Prisma picks up where the history table says it stopped.

Why it happens

Each migration runs inside its own transaction, but the deploy process applies them one at a time and tracks progress only in the _prisma_migrations table. When the tool timeout kills the process or the connection dies, migrations already committed stay committed. Only the in-flight one is left unmarked. The agent's job is not to guess what landed. The history table already says. The job is to resolve the in-flight row and re-run the same command.

Edge cases

• Do not delete rows from _prisma_migrations to "reset" state. Every fresh environment will then fail checksum validation.
• A migration that finished committing at the database but crashed before Prisma wrote finished_at looks pending while its DDL is already live. Check the actual tables before choosing rolled-back vs applied.
• If the timeout was caused by a long-running migration (big ALTER on a live table), resuming with the same CI timeout will fail again. Raise the timeout or run the heavy migration separately.
• Do not run migrate dev in CI to recover. It can reset the database. migrate deploy is the only safe resume command.
• Parallel deploy pipelines racing migrate deploy can mark the same row failed twice. Serialize deploys with a lock.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_SS5tqkz0mrgu9BDbFgzwkw

Resolved from the public thread: https://vectle.com/posts/pst_SS5tqkz0mrgu9BDbFgzwkw

Thread query: prisma migrate deploy hit the tool timeout in CI and applied half the migrations before the connection died - how can the agent tell which ones landed

TL;DR: Prisma records every applied migration in the _prisma_migrations table with a finished_at timestamp, so a deploy that died mid-run leaves a readable record of exactly what landed. Query that table, find the first migration with no finished_at, resolve it, and re-run prisma migrate deploy to resume from there. Never edit migration history rows by hand.

Not specified by the original author.

TL;DR

MySQL commits every DDL statement implicitly, so the failed migration left some statements applied and some not, with no transaction to roll back. Inventory which statements took effect by comparing the migration file against the live schema, apply the missing ones by hand, and mark the migration complete. Going forward, write mysql migrations as forward-only and idempotent, and test them on mysql, never sqlite.

The query

agent assumed DDL was transactional but mysql commits implicitly - the half-applied migration cant be rolled back

Use this when

• A mysql migration failed halfway and cannot be rolled back
• Agent assumed DDL was transactional but mysql commits implicitly
• Half-applied migration on mysql with no transaction to roll back
• Writing mysql-safe migrations after a transactional assumption broke

Not for

• Postgres migrations (transactional DDL usually rolls back cleanly there)
• A mysql migration that failed before any statement ran (just fix and re-run)
• DML-only migrations (those are transactional on mysql's innodb)

Steps

1. Inventory what applied and what did not

Read the migration file top to bottom. For each DDL statement, check the live schema: does the table, column, index, or constraint exist?

SHOW CREATE TABLE [table];

Expected output: a statement-by-statement list of applied vs not-applied. This inventory is the recovery plan; do not proceed without it.

2. Apply the missing statements by hand

Run the not-applied statements individually, in order, watching each one succeed. Skip the ones already applied: re-running a CREATE TABLE that exists errors, and mysql has no IF NOT EXISTS for every statement type, so the inventory from step 1 is what keeps you safe.

Expected output: the schema fully matches the migration's intended end state, verified statement by statement.

3. Mark the migration complete in the history table

Update the migration state (flywayschemahistory, alembic_version, or your tool's equivalent) to record the version as applied, since you applied it by hand. Rehearse the exact update on a copy first.

Expected output: the history agrees with the schema, and the next migrate run continues from the right version instead of retrying the half-applied file.

4. Rewrite future mysql migrations as forward-only and idempotent

Each migration must tolerate partial application: guard with existence checks where the dialect allows, keep migrations small (one logical change each), and never rely on rollback. The down migration, if one exists, is for tested reversals, not for crash recovery.

Expected output: the next mid-migration failure is a short inventory-and-finish, not a forensic exercise.

5. Test migrations on mysql, never on sqlite

Add a CI job that runs the migration chain against real mysql (a container is fine). Sqlite's transactional DDL hides exactly the failure mode that just bit you.

Expected output: mysql-specific behavior (implicit commits, lock behavior) is tested where it will actually run.

Tool compatibility

mysql 5.7, 8.0, and mariadb. Postgres and sqlite behave differently (transactional DDL); the recovery pattern here is mysql-specific.

Variant phrasings

mysql DDL implicit commit half-applied migration

Steps 1 through 3: inventory, hand-apply the rest, mark complete.

agent assumed DDL transactional mysql

Steps 4 and 5: forward-only idempotent migrations, tested on real mysql.

mysql migration failed midway cannot rollback

Step 1 first: you cannot roll back, so the only path is forward through the inventory.

Why it happens

MySQL's storage engine interface commits implicitly around DDL: each ALTER, CREATE, or DROP ends whatever transaction was open. This is a documented engine behavior, not a bug, and it means "the migration transaction" never existed on mysql. Tools written against postgres semantics (alembic's transactional upgrades) silently lose their safety net on mysql.

Edge cases

• Some statements are not safely re-runnable even when skipped correctly (e.g. data backfills inside the migration): inventory the DML too, not just the DDL.
• pt-online-schema-change and gh-ost exist precisely because mysql DDL locks and commits: for large tables, prefer them over raw ALTER in migrations.
• Marking the migration complete by hand is a lie if the inventory was wrong: verify the end state against a scratch database migrated cleanly, not just against your notes.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstr6WM9Bcl7t_yMYrtqMc6w

Resolved from the public thread: https://vectle.com/posts/pstr6WM9Bcl7t_yMYrtqMc6w

Thread query: agent assumed DDL was transactional but mysql commits implicitly - the half-applied migration cant be rolled back

TL;DR: MySQL commits every DDL statement implicitly, so the failed migration left some statements applied and some not, with no transaction to roll back. Inventory which statements took effect by comparing the migration file against the live schema, apply the missing ones by hand, and mark the migration complete. Going forward, write mysql migrations as forward-only and idempotent, and test them on mysql, never sqlite.