agent skipped e2e on a dependency bump - the checkout flow broke in production
Restores e2e coverage for dependency bumps on critical paths. Use it when a bump skipped e2e and broke a user-facing flow in production. Key trigger: the broken flow touches code paths the dependency serves, and e2e was not in the PR checks. Not for: a bump with no user-facing surface, where unit tests suffice.
TL;DR
Unit tests passed, e2e never ran, and the checkout flow broke in production. Make e2e a required check for bumps touching critical paths, run it on the bump PR, and hotfix or roll back the breakage now.
Error
agent skipped e2e on a dependency bump - the checkout flow broke in productionSteps
Triage production: confirm the breakage, assess impact, and decide between hotfix and rollback. Expected output: Production is stable again.
Reproduce with e2e: run the checkout flow tests against the bumped version in staging. Expected output: The e2e suite catches the breakage the unit tests missed.
Mark the dependency (and the paths it serves) as e2e-gated: any PR touching it must run the e2e suite. Expected output: The next bump on this dependency cannot skip e2e.
Add the e2e requirement to the agent's bump checklist, keyed by critical-path labels. Expected output: The agent knows which bumps need e2e before it opens the PR.
Review what the unit tests missed and add a targeted regression test at the unit level too. Expected output: Two layers of coverage instead of one.
When to use
- A dependency bump broke a user-facing flow in production
- e2e was skipped or not required on the bump PR
- You are defining which bumps need e2e
When not to use
- Bumps with no user-facing surface; unit and integration tests suffice
- e2e failures that are flaky or environment-related; fix the suite
- Breakage from a code change bundled with the bump; separate the concerns
Variant phrasings
dependency bump broke checkout in production
require e2e tests for dependency upgrades
skipped end to end tests on a version bump
Why it happens
Unit tests mock the dependency or exercise it in isolation, so a behavioral change in the real integration only shows up end to end. The agent optimized for fast green CI, and nothing in the checklist forced the slow suite that would have caught it.
Edge cases
- Full e2e on every bump is expensive; gate on critical-path labels, not on all bumps.
- e2e suites that take hours need a fast smoke subset for PRs and the full suite nightly.
- If the bump is a security fix, e2e-gating must not delay it past your SLA; run e2e in parallel with the merge process.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_1Eh4WROqdbUenaGoQzn3YA