migration agent's tool call returned success but the migration actually failed silently - the wrapper script...

TL;DR

Fix the wrapper so the tool's real exit code reaches the harness. In shell wrappers add set -euo pipefail at the top and never pipe the tool through a filter without pipefail; in Python wrappers pass check equals True to subprocess.run. The migration failed but the wrapper exited 0, so the agent recorded success and moved on while nothing changed.

migration agent's tool call returned success but the migration actually failed silently - the wrapper script swallowed the exit code

Steps

1. Prove the swallow. Run the wrapper against a deliberately failing migration, then print the exit code right after. Expected: the exit code is 0 even though the tool's own output shows a failure. That confirms the wrapper is lying.

2. Find the swallowing line in the wrapper. Common culprits: migrate_command | tee log.txt (a pipeline returns the last command's status), migrate_command || echo done, migrate_command && true, Python subprocess.run(cmd) without checking the return code. Expected: you can point at the exact line that discards the failure.

3. Fix it. Bash: put set -euo pipefail at the top of the script so pipelines report failures. Python: use subprocess.run(cmd, check=True) so a non-zero exit raises. Node: await the exec call and let the rejection propagate instead of catching and ignoring it. Expected: the wrapper now exits non-zero whenever the tool fails.

4. Re-run the failing migration through the fixed wrapper. Expected: the wrapper exits non-zero and the harness marks the step failed instead of green.

5. Now fix the actual migration failure, re-run, and confirm the history or version table shows the new revision applied. Expected: green step AND a changed schema. Both must be true.

Use this when

• The tool reports success but the schema is unchanged.
• Logs show an error while the harness step is green.
• The wrapper pipes tool output through tee, grep, or a log filter.
• An agent keeps "completing" migrations that never land.

Not for this skill when

• The migration genuinely succeeded (verify against the history table first).
• The tool exits non-zero but the harness is configured to ignore failures (that is a harness config issue).
• The migration SQL itself fails silently inside a transaction that then commits (different problem).

Variant phrasings

• wrapper script swallowed the exit code
• migration failed but CI passed
• bash pipe hides failing exit code
• subprocess.run did not raise on migration failure

Why it happens

In bash, a pipeline's exit status is the last command's status, so alembic upgrade head | tee log.txt returns tee's 0 even when alembic fails. || true idioms and unchecked subprocess calls do the same thing. The harness trusts the wrapper's exit code, so the failure becomes invisible and the agent proceeds as if the migration landed.

Edge cases

• pipefail makes grep-in-pipeline return 1 when nothing matches. That is correct behavior, but handle it deliberately if your wrapper greps output.
• Some tools exit 0 with warnings that still matter. Parse the output for error strings as a second check, not just the exit code.
• Fixing detection surfaces old failures. The first run after the fix may turn several "green" steps red. That is the fix working.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_ohhX3tpoVVuh57ISHbc3cQ