agent loop detected: re-running already-applied D1 migrations, deploy blocked on 'already exists'

TL;DR

The agent treats 'migration failed' as 'migrations not applied' and reruns the full set each pass. The migrations ARE applied, so every run dies on 'already exists'. The deploy pipeline should apply migrations once and separately from the code deploy, with the agent checking the migration journal before running anything.

The query

agent loop detected: re-running already-applied D1 migrations, deploy blocked on 'already exists'

Use this when

• Every run executes the full migration set.
• The failure is always 'already exists' or journal conflicts.
• No schema change is actually pending.

Not for

• Genuine pending migrations that need applying (normal flow).
• First-time database setup (expected to run once).
• Migration SQL errors (fix the SQL).

Steps

Step 1: Stop the loop and inspect the journal

npx wrangler d1 migrations list [db] --remote

Expected result: the journal shows the migrations already applied. There is nothing to do; the agent was wrong.

Step 2: Separate migrations from deploys

Run migrations as their own pipeline step, gated on 'are there new migration files since the last apply', not as part of every deploy attempt.

Expected result: deploys no longer run migrations unconditionally. The loop's fuel is gone.

Step 3: Teach the agent the check-first pattern

Before applying, the agent lists the journal and diffs against the migrations directory. Only genuinely new files get applied.

Expected result: the agent's migration step is a no-op when nothing is pending, which is the correct behavior most of the time.

Step 4: Fix any journal drift the loop caused

Repeated partial applies can leave the journal and the schema disagreeing. Compare the journal to the actual schema and reconcile.

Expected result: journal and schema agree. Future applies are clean.

Step 5: Verify a full clean run

Expected result: the agent checks, finds nothing pending, skips migrations, and the deploy proceeds. The 'already exists' error never appears because apply is never wrongly invoked.

Why it happens

The agent conflates 'deploy failed' with 'database not ready'. Migration tooling is designed to be run by humans who know when schema changed; the agent runs it as a ritual. Idempotency in migration tooling is limited (journals, not true idempotence), so the ritual fails loudly instead of no-op-ing.

Edge cases

• Local vs remote journals drift independently. The agent must check the right one for the target environment.
• A migration file added but never applied IS pending; the check-first pattern handles this correctly.
• Never delete journal entries to 'fix' the loop. The journal is the source of truth for what schema the data matches.
• If the loop ran against production, audit for partial applies before declaring the schema healthy.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_TGACNWOU4-gEeu9raTeLFg