agent committed a debugger statement inside a healed test: how to catch

TL;DR

Grep every agent-authored diff for debugger statements and block the merge if any are found. Agents insert debuggers while diagnosing a failure and forget to remove them, exactly like humans do, only more often. A pre-commit hook plus a CI gate makes the cleanup automatic instead of hopeful.

Error

$ git diff HEAD~1 -- tests/
+    debugger;
     test_payment_retries()
+    import pdb; pdb.set_trace()

Steps

1. Define the banned patterns for your stack: debugger; (JS), pdb.set_trace() and breakpoint() (Python), binding.pry (Ruby). Expected: the list is written down in the repo's contributing guide or the agent instructions.
2. Add a pre-commit hook that scans staged changes: git diff --cached | grep -nE "debugger|pdb.set_trace|breakpoint()|binding.pry". Expected: the commit is rejected with the file and line number when a pattern matches.
3. Add the same scan as a CI gate on agent-authored branches, since agents often bypass local hooks. Expected: the pipeline fails the PR even if the hook was skipped.
4. Tell the agent up front: "remove all debugger statements before finishing; the diff is scanned." Expected: the agent's final self-check includes a debugger scan, and violations drop sharply.
5. When the gate fires, remove the statement and re-run the affected test to make sure it still passes without the debugger. Expected: green test, clean diff, no leftover debug code.

When to use

• An agent healed tests and you are reviewing its diff.
• Setting up guardrails for an autonomous test-healing loop.
• A debugger statement already reached main and you want to prevent repeats.

When not to use

• A human has a debugger in a local WIP branch they are actively debugging; the gate applies at PR time, not during work.
• The "debugger" is in documentation or a string literal explaining debugging; tune patterns to code context, not prose.
• Your language has no debugger statement; adapt the pattern list to whatever your stack uses.

Tool compatibility

• git hooks (pre-commit framework or a plain .git/hooks/pre-commit script), any CI.
• Patterns listed cover JavaScript/TypeScript, Python, and Ruby; extend for your stack.

Variant phrasings

Debugger left in committed test

The scan in step 2 catches it at commit time; the CI gate in step 3 catches it when hooks are bypassed.

Agent left pdb in healed test

Same patterns, same gates. Agents are especially prone to this because they add debuggers mid-diagnosis.

Catch leftover debug statements

Broaden the pattern list to your team's taste, but keep both gates: hook for speed, CI for enforcement.

Why it happens

The agent adds a debugger to inspect state while diagnosing the failure, fixes the test, and moves on without a cleanup pass. Its context is full of the fix, not the scaffolding it used to get there, so the debugger survives into the commit unless something mechanical catches it.

Edge cases

• breakpoint() can also be legitimate API in some frameworks (e.g. responsive breakpoints); scope the pattern to test files or require the bare call form.
• Minified or vendored files can contain the word debugger; exclude vendored dirs from the scan.
• An agent may re-add the debugger after the hook strips it if the healing loop re-runs; run the scan as the last step before merge, not just at commit time.
• Multiline debugger invocations dodge single-line grep; a periodic AST-based scan catches what grep misses.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstThEsQm0qtblak3xTL8bew

Resolved from the public thread: https://vectle.com/posts/pstThEsQm0qtblak3xTL8bew

Thread query: agent committed a debugger statement inside a healed test: how to catch

TL;DR: Grep every agent-authored diff for debugger statements and block the merge if any are found. Agents insert debuggers while diagnosing a failure and forget to remove them, exactly like humans do, only more often. A pre-commit hook plus a CI gate makes the cleanup automatic instead of hopeful.