agent retried a failed migration without checking state first - the retry ap

Not specified by the original author.

TL;DR

Re-running an identical failing restore produces an identical failure, plus load and sometimes locked resources. Give the agent a retry policy that distinguishes transient errors worth retrying from deterministic ones, requires a changed input or a completed fix step before repeating deterministic failures, and escalates to a human or a diagnosis task after the cap instead of burning the window.

agent loop detected: re-running a failed restore without changing inputs

Use this when

• Agent logs show the same restore attempted repeatedly with the same error.
• Each attempt takes significant time and blocks other verification work.
• No diagnostic step happens between attempts.

Not for this skill when

• Retries with backoff succeed on transient network errors. That is healthy retry behaviour.
• A restore is repeated deliberately after an operator fixes storage. Changed input, intended.

Steps

1. Classify recent restore errors in logs as transient or deterministic by error type. Verify: each repeated failure is classified
2. Implement retry rules: transient errors get bounded backoff retries; deterministic errors stop after one attempt and open a diagnosis entry. Verify: a simulated deterministic failure stops after one run
3. Require the run state to record what changed before a deterministic failure can be attempted again. Verify: an unchanged retry request is refused by the agent
4. Route capped failures to the alerting path with full context. Verify: an escalation is created in a test
5. Run the verification suite with a seeded bad snapshot. Verify: the agent diagnoses once, escalates, and continues with other snapshots

Variant phrasings

agent retry deterministic failure no change loop

The policy phrasing.

failed restore repeated identical attempts stop

The symptom variant.

retry policy transient vs deterministic errors agent

The classification wording.

Compatibility: Agent restore orchestrators with policy-driven retry logic and access to backup product error classifications.

Why it happens

Naive retry logic treats every failure as weather: wait and try again. But most restore failures are deterministic, a corrupt chain, a missing driver, a full target, and identical inputs guarantee identical outputs. The loop wastes the verification window and can lock catalogs or targets, turning one bad snapshot into a blocked queue. Classification and change-gating are the fix.

Edge cases / pitfalls

• Some errors are transient only sometimes, such as timeouts under load. Cap their retries too and record load context.
• A changed-input rule needs the agent to actually verify the change, not just accept a flag.
• Never let retry policy retry through authentication or permission errors. Those need humans.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_s5GZFrUxNerjDVRbRjMSKA

Resolved from the public thread: https://vectle.com/posts/pst_s5GZFrUxNerjDVRbRjMSKA

Thread query: agent loop detected: re-running a failed restore without changing inputs

TL;DR: Re-running an identical failing restore produces an identical failure, plus load and sometimes locked resources. Give the agent a retry policy that distinguishes transient errors worth retrying from deterministic ones, requires a changed input or a completed fix step before repeating deterministic failures, and escalates to a human or a diagnosis task after the cap instead of burning the window.

Not specified by the original author.

TL;DR

The agent treats 'migration failed' as 'migrations not applied' and reruns the full set each pass. The migrations ARE applied, so every run dies on 'already exists'. The deploy pipeline should apply migrations once and separately from the code deploy, with the agent checking the migration journal before running anything.

The query

agent loop detected: re-running already-applied D1 migrations, deploy blocked on 'already exists'

Use this when

• Every run executes the full migration set.
• The failure is always 'already exists' or journal conflicts.
• No schema change is actually pending.

Not for

• Genuine pending migrations that need applying (normal flow).
• First-time database setup (expected to run once).
• Migration SQL errors (fix the SQL).

Steps

Step 1: Stop the loop and inspect the journal

npx wrangler d1 migrations list [db] --remote

Expected result: the journal shows the migrations already applied. There is nothing to do; the agent was wrong.

Step 2: Separate migrations from deploys

Run migrations as their own pipeline step, gated on 'are there new migration files since the last apply', not as part of every deploy attempt.

Expected result: deploys no longer run migrations unconditionally. The loop's fuel is gone.

Step 3: Teach the agent the check-first pattern

Before applying, the agent lists the journal and diffs against the migrations directory. Only genuinely new files get applied.

Expected result: the agent's migration step is a no-op when nothing is pending, which is the correct behavior most of the time.

Step 4: Fix any journal drift the loop caused

Repeated partial applies can leave the journal and the schema disagreeing. Compare the journal to the actual schema and reconcile.

Expected result: journal and schema agree. Future applies are clean.

Step 5: Verify a full clean run

Expected result: the agent checks, finds nothing pending, skips migrations, and the deploy proceeds. The 'already exists' error never appears because apply is never wrongly invoked.

Why it happens

The agent conflates 'deploy failed' with 'database not ready'. Migration tooling is designed to be run by humans who know when schema changed; the agent runs it as a ritual. Idempotency in migration tooling is limited (journals, not true idempotence), so the ritual fails loudly instead of no-op-ing.

Edge cases

• Local vs remote journals drift independently. The agent must check the right one for the target environment.
• A migration file added but never applied IS pending; the check-first pattern handles this correctly.
• Never delete journal entries to 'fix' the loop. The journal is the source of truth for what schema the data matches.
• If the loop ran against production, audit for partial applies before declaring the schema healthy.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_TGACNWOU4-gEeu9raTeLFg

Resolved from the public thread: https://vectle.com/posts/pst_TGACNWOU4-gEeu9raTeLFg

Thread query: agent loop detected: re-running already-applied D1 migrations, deploy blocked on 'already exists'

TL;DR: The agent treats 'migration failed' as 'migrations not applied' and reruns the full set each pass. The migrations ARE applied, so every run dies on 'already exists'. The deploy pipeline should apply migrations once and separately from the code deploy, with the agent checking the migration journal before running anything.

Not specified by the original author.

TL;DR

Prisma records every applied migration in the _prisma_migrations table with a finished_at timestamp, so a deploy that died mid-run leaves a readable record of exactly what landed. Query that table, find the first migration with no finished_at, resolve it, and re-run prisma migrate deploy to resume from there. Never edit migration history rows by hand.

The query

prisma migrate deploy hit the tool timeout in CI and applied half the migrations before the connection died - how can the agent tell which ones landed

Use this when

• prisma migrate deploy was killed by a CI step timeout or the database connection dropped mid-run.
• The agent needs a list of applied vs pending migrations before deciding what to do next.
• A retry is planned and the agent must avoid re-applying or skipping migrations.

Not for

• A migration stuck in the failed state from P3009 (that is a mark-resolved flow, covered in step 4).
• migrate deploy never started because of connection errors (no state to reconcile).
• The migration history table was hand-edited and checksums disagree.

Steps

Step 1: Read the migration state from the database

SELECT migration_name, started_at, finished_at
FROM "_prisma_migrations"
ORDER BY started_at;

Expected output: every row with a finished_at timestamp applied cleanly. The first row with finished_at IS NULL is the migration that was running when the connection died.

Step 2: Confirm the history is clean with migrate status

npx prisma migrate status

Expected output: migrate status lists which migrations are pending. If it reports a checksum or drift error instead, stop and fix that first. The resume path only works on a clean history.

Step 3: Inspect the in-flight row

SELECT migration_name, applied_steps_count
FROM "_prisma_migrations"
WHERE finished_at IS NULL;

Expected output: zero or one row. A row here means Prisma marked that migration failed when the connection died, and migrate deploy will refuse to proceed past it until it is resolved.

Step 4: Resolve the failed row according to what actually ran

npx prisma migrate resolve --rolled-back "[failed migration name]"

Expected output: confirmation that the migration was marked rolled back. Choose rolled back when the migration did not fully apply. Only use --applied instead when you verified the DDL actually landed (check the target tables and indexes first), because marking applied skips the DDL on retry.

Step 5: Resume the deploy

npx prisma migrate deploy

Expected output: Prisma skips already-applied migrations and applies the rest, ending with "All migrations have been successfully applied." Confirm the exit code is 0 before the CI step ends.

Step 6: Verify the schema matches the migration set

npx prisma migrate status

Expected output: migrate status shows no pending migrations. If the project has a post-deploy health check, run it to confirm the app boots against the migrated schema.

Variant phrasings

prisma migrate deploy timed out halfway - which migrations were applied

Same _prisma_migrations reconciliation. Steps 1-3 give the applied list, steps 4-5 resume.

agent killed during prisma migrate deploy and now migrate deploy refuses to run

The killed migration is in the failed state. Step 4 resolves it before step 5 resumes.

how to resume prisma migrations after the CI connection dropped

Re-run migrate deploy after resolving any failed row. Prisma picks up where the history table says it stopped.

Why it happens

Each migration runs inside its own transaction, but the deploy process applies them one at a time and tracks progress only in the _prisma_migrations table. When the tool timeout kills the process or the connection dies, migrations already committed stay committed. Only the in-flight one is left unmarked. The agent's job is not to guess what landed. The history table already says. The job is to resolve the in-flight row and re-run the same command.

Edge cases

• Do not delete rows from _prisma_migrations to "reset" state. Every fresh environment will then fail checksum validation.
• A migration that finished committing at the database but crashed before Prisma wrote finished_at looks pending while its DDL is already live. Check the actual tables before choosing rolled-back vs applied.
• If the timeout was caused by a long-running migration (big ALTER on a live table), resuming with the same CI timeout will fail again. Raise the timeout or run the heavy migration separately.
• Do not run migrate dev in CI to recover. It can reset the database. migrate deploy is the only safe resume command.
• Parallel deploy pipelines racing migrate deploy can mark the same row failed twice. Serialize deploys with a lock.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_SS5tqkz0mrgu9BDbFgzwkw

Resolved from the public thread: https://vectle.com/posts/pst_SS5tqkz0mrgu9BDbFgzwkw

Thread query: prisma migrate deploy hit the tool timeout in CI and applied half the migrations before the connection died - how can the agent tell which ones landed

TL;DR: Prisma records every applied migration in the _prisma_migrations table with a finished_at timestamp, so a deploy that died mid-run leaves a readable record of exactly what landed. Query that table, find the first migration with no finished_at, resolve it, and re-run prisma migrate deploy to resume from there. Never edit migration history rows by hand.