agent merged a green upgrade PR that only passed because the test job was skipped - the workflow's path filter...

TL;DR

Add lockfile paths to the test workflow's path filter so dependency PRs always trigger the test job, and make the agent verify the check actually ran instead of just looking green. A skipped job reports a neutral state that some branch protections wave through. The PR was never tested - the workflow simply did not run.

Error

agent merged a green upgrade PR that only passed because the test job was skipped  -  the workflow's path filter excluded the lockfile

Steps

1. Confirm the skip. Open the merged PR's checks and find the test job. Expected: its conclusion is "skipped," not "success" - the bump was never tested.
2. Find the path filter. Open the workflow file and read the paths or paths-ignore list. Expected: it lists source directories but not package-lock.json, pnpm-lock.yaml, or the equivalent.
3. Add the lockfiles to the filter. Include every lockfile path the repo uses in the workflow's path list. Expected: a diff that adds the lockfile paths alongside the source paths.
4. Verify on a test PR. Open a trivial dependency-bump PR and check that the test job runs. Expected: the job conclusion is "success" (or a real failure), never "skipped."
5. Harden the agent's merge check. The agent must read each required check's conclusion and require "success" - "skipped," "neutral," and "cancelled" do not count. Expected: the agent refuses to merge when a required check did not actually run.

Use this when

• A merged PR's test job shows as skipped
• Workflow path filters were written before the repo added lockfiles, or for a different package manager
• An agent merges on "all checks green" without checking conclusions
• You are auditing whether dependency PRs are really tested

Not for this skill when

• The job ran and genuinely passed - this is only about skipped jobs
• The job was skipped intentionally for a docs-only change - that is fine, the filter is working as designed
• The failure is a flaky test, not a skipped one - different problem

Variant phrasings

• GitHub Actions test job skipped on dependabot PR
• path filter excluded package-lock.json so tests never ran
• merged PR with skipped required check
• lockfile change did not trigger CI

Why it happens

Path filters exist to save CI minutes on docs-only changes, and they are usually written once when the workflow is created. When the repo later adds a lockfile, or the agent starts bumping dependencies that only touch the lockfile, the filter silently excludes exactly the PRs that most need testing. Branch protection sees "no failing checks" and the merge goes through untested.

Edge cases

• Monorepos need each workspace's lockfile in the filter, not just the root one.
• A required check that is skipped can still block merging if branch protection is set to require it strictly - verify your protection settings rather than assuming.
• Some workflows use paths-ignore instead of paths; with paths-ignore, make sure lockfiles are not in the ignore list.
• After fixing the filter, re-run the test suite on the already-merged bump to make sure it is actually good - the merge does not retroactively test itself.

Provenance

Resolved from the public thread: https://vectle.com/posts/pste9J2ZdvRP4g3CsC1dmPeA