vitess error: online DDL migration cutover failed - how to retry safely

TL;DR

A failed cutover leaves the migration in a non-terminal state, not a broken schema: the old table is still serving. Inspect the migration with show vitess_migrations, diagnose the cutover failure (usually lag, a stuck vreplication stream, or a lock), then retry the migration explicitly. Only cancel as a last resort, and never re-issue the same DDL blindly while the old migration is still registered.

The query

vitess error: online DDL migration cutover failed - how to retry safely

Use this when

• vitess online DDL migration cutover failed
• A vitess migration is stuck and not progressing
• Need to retry a vitess online schema change safely
• An agent's vitess migration wedged at cutover

Not for

• DDL that fails immediately with a syntax or schema error (fix the SQL)
• VReplication lag during the copy phase (that is a lag problem, not a cutover problem)
• Non-online (direct) DDL in vitess (different path, no cutover)

Steps

1. Inspect the migration state

SHOW VITESS_MIGRATIONS;

Or for one migration:

SHOW VITESS_MIGRATION '[uuid]';

Expected output: the migration row with its status (e.g. failed, or stuck in a running phase) and the artifact tables involved. Note the uuid: every later command addresses the migration by uuid.

2. Diagnose the cutover failure

Check the vreplication stream for the migration and look at recent tablet logs for the cutover attempt. Common causes: replication lag never caught up, a long-running transaction on the source table, or the cutover timeout expiring.

SHOW VITESS_MIGRATION '[uuid]' LOGS;

Expected output: the log lines around the cutover attempt naming the cause. Fix the cause (wait out lag, clear the blocker) before retrying: retrying into the same condition just fails again.

3. Retry the migration explicitly

ALTER VITESS_MIGRATION '[uuid]' RETRY;

Expected output: the migration re-enters the queue and progresses through copy and cutover again. Watch its status until it reaches complete.

4. If retry is wrong, complete or cancel deliberately

When the schema change already took effect through another path, mark it complete:

ALTER VITESS_MIGRATION '[uuid]' COMPLETE;

When the change should not happen at all, cancel it:

ALTER VITESS_MIGRATION '[uuid]' CANCEL;

Expected output: the migration reaches a terminal state. Cancel cleans up the artifact tables; only cancel when you are sure the change is unwanted.

5. Verify the schema and the serving path

Check the table structure on the tablets and confirm traffic is serving from the expected table. Expected output: the intended schema is live, no artifact tables remain, and SHOW VITESS_MIGRATIONS shows the migration complete.

Tool compatibility

vitess 16.x through 18.x with managed online DDL (vitess managed schema changes). Syntax is stable across these versions.

Variant phrasings

vitess online DDL migration cutover failed how to retry

Steps 1 through 3: inspect, diagnose, ALTER VITESS_MIGRATION ... RETRY.

vitess migration stuck at cutover

Step 2: a stuck cutover is almost always lag or a blocker on the source table. Clear it, then retry.

vitess cancel online DDL migration

Step 4: CANCEL is the deliberate abort; use it when the change should not happen, not as a reflex for a retryable failure.

Why it happens

Vitess online DDL runs the migration as a managed workflow with explicit states, and cutover is the guarded step that swaps the tables. A failed cutover parks the workflow instead of half-swapping, because an unattended retry policy could cut over during a lag spike. The explicit RETRY keeps a human (or agent) in the loop at the riskiest step.

Edge cases

• Never submit the same DDL as a new migration while the old one is non-terminal: two workflows fighting over the same table corrupt the artifact tables.
• A migration left in failed state still holds its artifact tables: they consume disk until the migration is completed or cancelled.
• Cutover timeouts that recur on a schedule point at periodic load (backups, batch jobs): move the retry outside that window rather than raising timeouts forever.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_gYXhMmAwIM0CuCTYKvoWbA