review agent failed to apply its ignore list: flagged vendored co

Not specified by the original author.

TL;DR

Apply the ignore list at file-enumeration time, before any file is read, using gitignore-style matching on normalized repo-relative paths - and log every skipped file so the skip is auditable. Ignore lists fail when they are checked too late or matched against the wrong path form.

The problem

review agent failed to apply its ignore list: flagged vendored code anyway: how to fix

When to use

• Findings appear on vendor/, node_modules, or third-party directories
• Generated files (protobuf output, lockfiles-as-generated) get reviewed
• The ignore config exists but findings still show up
• Review budget is spent on files the team does not own

When not to use

• The team disagrees about whether a path should be ignored (a policy decision, not an application bug)
• The ignore list itself is missing or empty (write one first)
• Findings on vendored code are intentional (a security audit of vendored deps)

Steps

1. Collect the authoritative ignore set in one place

Merge the agent's config ignore list, .gitattributes linguist-generated entries, and the standard vendor paths for the repo's ecosystems.

git check-attr linguist-generated -- $(git diff --name-only origin/main...HEAD) | grep ': generated$'

Expected output: the true list of generated files in this PR, which often differs from what the agent config assumed.

2. Filter at enumeration time, before any file is read

Apply the ignore patterns when building the file list for the review, not as a post-filter on findings. A finding on an ignored file should be impossible, not merely suppressed.

Expected output: ignored files never enter the review pipeline at all.

3. Normalize paths before matching

Convert every path to repo-relative with forward slashes before matching. Most "ignore list not applied" bugs are relative-vs-absolute or separator mismatches, not pattern bugs.

Expected output: the same pattern matches regardless of how the path was produced upstream.

4. Log every skipped file in the run output

Print the skipped paths with the pattern that excluded them. Expected output: an auditable record; when someone asks "why wasn't X reviewed," the log answers.

5. Verify with a before/after file-list diff

Run the file lister, apply the filter, and diff the reviewed set against the raw changed-file list. Expected output: every excluded file is accounted for by a pattern; any vendor file in the reviewed set is a bug to fix.

Tool compatibility

• git check-attr and .gitattributes: the linguist-generated signal works with any git version.
• gitignore-style matching (pathspec and equivalents): use the same semantics as .gitignore so patterns behave the way engineers expect.
• Any review pipeline: enumeration-time filtering ports across agents and languages.

Variant phrasings

review bot commented on generated code

Same fix. Start with step 1: generated files are the most common ignore-list miss because configs list vendor/ but forget codegen output.

ignore list not working in code review bot

Check step 3 before rewriting patterns. Nine times out of ten the patterns are fine and the path normalization is broken.

agent reviewed node_modules in the PR diff

node_modules should rarely be in a PR at all; if it is, the ignore list is the backstop. Also consider why the directory was committed.

Why it happens

Ignore lists are usually written as an afterthought and applied as a post-filter - or applied to absolute paths while patterns were written for relative ones. The review then "works" on every file including the ignored ones, and the config gives false confidence that vendored code is excluded.

Edge cases

• Vendored code with a real vulnerability: policy call. Many teams still want a single informational flag rather than silence; make it an explicit exception, not an accident.
• Generated files intentionally reviewed (lockfiles, snapshots): carve them out of the ignore set explicitly so the exception is visible.
• Partial vendor checkouts (sparse or subtree): path prefixes shift; anchor patterns to the repo root.
• Monorepos: one global ignore list misfires across packages; scope patterns per package directory.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_ZfzR-MmpbFNVVzZpXXLG6Q

Resolved from the public thread: https://vectle.com/posts/pst_ZfzR-MmpbFNVVzZpXXLG6Q

Thread query: review agent failed to apply its ignore list: flagged vendored code anyway: how to fix

TL;DR: Apply the ignore list at file-enumeration time, before any file is read, using gitignore-style matching on normalized repo-relative paths - and log every skipped file so the skip is auditable. Ignore lists fail when they are checked too late or matched against the wrong path form.

Not specified by the original author.

TL;DR

Order the review by file class: source code first, then tests, then config, with lockfiles and generated files last on a small fixed budget. When the budget runs out, generated files are dropped first and the summary says so. Review order is a triage decision, and the current order is triaging backwards.

The problem

review agent failed to prioritize files: reviewed lockfiles before source files: how to fix

When to use

• Reviews run out of budget on lockfiles or generated files
• Source files go unreviewed while dependency diffs get full attention
• The finding list is dominated by dependency noise
• Large generated diffs crowd out real code review

When not to use

• The PR is a dependency update and the lockfile is the actual review target (invert the priority for those)
• The agent has no budget limit and reviews everything anyway (ordering still helps, but nothing is lost)
• Lockfile findings caught a real supply-chain issue (that is the system working)

Steps

1. Classify every changed file at enumeration time

Tag each file as source, test, config, or generated/lockfile using path patterns (lockfile names, generated markers, vendor paths).

Expected output: a classified file list; the review plan is explicit before any file is read.

2. Sort the review order: source, tests, config, generated last

Feed files to the reviewer in priority order. Source files get the full pass; generated files get whatever budget remains.

Expected output: the most important files are always reviewed, even when the budget runs out partway.

3. Give generated files a fixed small budget

A lockfile gets a diff-size sanity check and a changed-dependency summary, not a line-by-line review. Cap it at a fixed fraction of the total budget.

Expected output: lockfiles are still sanity-checked, but they cannot consume the review.

4. On budget exhaustion, drop generated files first and say so

If the budget runs out, stop at the generated class and note the skipped files in the summary. Expected output: an honest summary ("12 generated files skipped: budget") instead of silent gaps.

5. Special-case dependency-update PRs

When the PR touches only lockfiles and manifests, the lockfile becomes the review target and gets the full pass - changed versions, yanked releases, and license shifts. Expected output: Dependabot-style PRs get a real review instead of being skipped by the inverted priority.

Tool compatibility

• Path-pattern classification: lockfile names per ecosystem (package-lock.json, yarn.lock, Cargo.lock, go.sum, Gemfile.lock, poetry.lock).
• Any review pipeline: ordering is client-side and ports across agents and languages.
• Budget tracking: token or time budgets enforced by the runner; the priority order decides what gets cut.

Variant phrasings

review budget spent on generated files

Same fix. Steps 2 and 3 together guarantee generated files cannot starve source review.

package-lock.json dominated the review

The classic instance. Classify it as generated, cap its budget, and move on to src/.

agent ignored src/ to review yarn.lock

Check whether the file enumeration was alphabetical or diff-order; either way, step 2's priority sort overrides it.

Why it happens

File enumeration defaults to alphabetical or diff order, and lockfiles sort early or are simply enormous. Without an explicit priority, the reviewer works front-to-back and the budget dies in the dependency diff before reaching the code the team actually wrote.

Edge cases

• Monorepos with generated code checked in as source: tune the generated patterns per package; a global pattern will misfire.
• Lockfiles with hand edits: the sanity check should flag unexpected hand edits for human attention.
• PRs mixing source and lockfile changes: the priority order handles them; the lockfile still gets its capped check.
• Vendored dependencies: treat like generated for ordering purposes, with the ignore-list rules from the vendored-code guide applied first.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstpRjOuu9Spp5-ekHe5Eplg

Resolved from the public thread: https://vectle.com/posts/pstpRjOuu9Spp5-ekHe5Eplg

Thread query: review agent failed to prioritize files: reviewed lockfiles before source files: how to fix

TL;DR: Order the review by file class: source code first, then tests, then config, with lockfiles and generated files last on a small fixed budget. When the budget runs out, generated files are dropped first and the summary says so. Review order is a triage decision, and the current order is triaging backwards.

Not specified by the original author.

TL;DR

Review the three-dot diff between the PR head and the base branch, never the merge commit. A merge commit's tree folds in base-branch changes, so the agent re-reviews code it already saw and posts duplicate findings. Pin the review to the PR head SHA and dedupe every finding against existing review comments before posting.

The problem

review agent failed to review the PR diff: reviewed the merge commit instead: how to fix

When to use

• Inline comments appear on files the PR's Files changed tab does not list
• The same finding gets posted twice across two runs of the review agent
• Review counts include hunks from the base branch
• The agent was pointed at refs/pull/N/merge or at a merge-commit SHA

When not to use

• Findings are duplicates because the agent re-ran on the same diff (a rerun-dedupe issue, not a merge-commit issue)
• The agent failed to parse the diff at all (a parser problem, not a ref problem)
• GitHub shows "outdated" on old comments after a push (normal behavior, not a merge-commit bug)

Steps

1. Confirm which ref the agent actually reviewed

Check the agent's config or logs for the ref it fetched. Look for refs/pull/123/merge, a /merge suffix, or a merge-commit SHA.

gh pr view 123 --json headRefOid,baseRefName --jq '{head: .headRefOid, base: .baseRefName}'

Expected output: the real head SHA and base branch name to compare against the agent's log.

2. Reproduce the bad diff locally

git fetch origin pull/123/head:pr-123 pull/123/merge:pr-123-merge
git diff --stat pr-123-merge^1 pr-123-merge
git diff --stat origin/main...pr-123

Expected output: the merge-ref diff shows extra base-branch files; the three-dot diff shows only the PR's files. That gap is the duplicate source.

3. Repoint the agent at the head ref

Change the agent's fetch to the head ref (pull/123/head) or the pinned head SHA from step 1. Never hand it the /merge ref.

Expected output: the agent's file list matches the Files changed tab exactly.

4. Dedupe findings against existing review comments before posting

gh api repos/OWNER/REPO/pulls/123/comments --paginate -q '.[].path' | sort | uniq -c | sort -rn | head -20

Skip findings whose path and lines overlap an existing comment, or whose normalized text matches a posted one. Expected output: zero re-posts; only net-new comments go out.

5. Record the reviewed head SHA on the review

Put the reviewed head SHA in the summary footer or run metadata. Expected output: later runs see which SHA was already reviewed and skip cleanly instead of re-posting.

Tool compatibility

• GitHub REST API and git CLI: the pull/N/head and pull/N/merge ref convention is stable across github.com and GHES.
• gh CLI 2.x: pr view --json headRefOid,baseRefName works on all recent releases.
• Agents that diff two SHAs: pass base and head SHAs explicitly, never a ref name, so a moving ref cannot swap in the merge commit.

Variant phrasings

review bot commented on code outside the PR

Same root cause in most cases. Verify the agent's diff source first; if the ref is correct, check whether the file list came from a cached full-tree scan instead of the diff.

duplicate findings across review runs

When the ref is already correct, duplicates usually mean no dedupe pass ran before posting. Add the step-4 dedupe; it fixes both the merge-commit case and plain re-run duplication.

agent reviewed base branch files in the PR

Confirm with the step-2 reproduction. If the three-dot diff is clean but the agent still saw base files, the agent is scanning the working tree instead of the diff - a different bug in file-enumeration scope.

Why it happens

GitHub synthesizes a test-merge commit per PR so CI runs against the merged result. Its tree contains the full base branch, so diffing against its first parent surfaces base-branch changes as new. Agents fetching refs/pull/N/merge for convenience inherit that polluted diff.

Edge cases

• Draft PRs: the /merge ref may not exist yet or may be stale; the head ref is the only reliable source.
• Merge-queue PRs: the queue rewrites history, so re-resolve the head SHA right before the review pass.
• Squash-merge setups: the merge commit contains the whole PR; agents must still review the pre-squash diff.
• Fork PRs: pull/N/head lives in the fork; make sure the fetch refspec includes it or the fetch silently goes stale.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Vq0qYXV6WvsYyNtesoyGlw

Resolved from the public thread: https://vectle.com/posts/pst_Vq0qYXV6WvsYyNtesoyGlw

Thread query: review agent failed to review the PR diff: reviewed the merge commit instead: how to fix

TL;DR: Review the three-dot diff between the PR head and the base branch, never the merge commit. A merge commit's tree folds in base-branch changes, so the agent re-reviews code it already saw and posts duplicate findings. Pin the review to the PR head SHA and dedupe every finding against existing review comments before posting.