scraping the PR files page returned empty: agent's selector missed

Not specified by the original author.

TL;DR: GitHub's files list is virtualized - rows only exist in the DOM near the viewport, so a selector that runs too early (or against the wrong container) finds nothing. Wait for the first real row selector with an explicit timeout, scroll in increments to hydrate more rows, and collect rows as they appear. And keep the paginated files API as the fallback, because the DOM will always be flakier than the API.

scraping the PR files page returned empty: agent's selector missed the virtualized list

1. Confirm the virtualization. Open the files page, inspect the list container, and scroll: watch rows mount and unmount as the viewport moves. Expected: you see rows appearing and disappearing with scroll, proving the full list is never in the DOM at once.

2. Fix the wait. Replace any fixed sleep with an explicit wait for a concrete row selector (the per-file row element), with a timeout around 15 to 30 seconds. Expected: the wait succeeds when rows hydrate and fails fast with a clear timeout when they do not.

3. Check the selector itself against the live DOM. Virtualized lists often nest rows inside a scroll container with generated class names; verify your selector matches actual row elements in the current page markup. Expected: the selector matches at least the visible rows in a manual check.

4. Scroll to hydrate. Scroll the list container in viewport-sized increments, pausing briefly after each, and collect newly appeared rows each time until a full pass adds no new rows. Expected: the collected set grows with each scroll pass and then stabilizes.

5. Dedupe the collected rows by file path, since virtualization can re-mount the same row. Expected: the final list has one entry per file path.

6. Add the API fallback. If the row wait times out or the collected count looks wrong, fetch the paginated files API instead and log that the scrape was abandoned. Expected: the run still produces a complete file list even when the DOM path fails.

Use this when

• Scraping the PR files page returns empty but files are visible
• The selector worked last month and finds nothing now (markup changed)
• Only the first few rows are ever collected
• You need DOM rows but the list virtualizes

Not for this skill when

• The page never finishes loading at all (that is a page-load hang, different fix)
• GitHub serves a rate-limit interstitial instead of the page (rate-limit problem)
• You only need the file list, not rendered rows (use the files API directly and skip the browser)
• The browser automation times out mid-scroll rather than finding nothing (scroll-timeout problem)

Variant phrasings

• "GitHub files list virtualized, scraper finds no rows"
• "selector returns empty on PR files changed tab"
• "scrape virtualized diff list GitHub"
• "files page rows not in DOM for automation"

Why it happens

Virtualized lists render only the rows near the viewport to keep huge PRs fast. A scraper that queries the DOM once - especially before the list hydrates - sees an empty container. It is not that the selector is wrong in principle; it ran at a moment (or against a container state) where no rows existed yet.

Edge cases

• GitHub changes its markup without notice. Pin a periodic selector health check rather than assuming the selector stays valid.
• Rows for very large files may render a placeholder instead of a full row. Decide whether placeholders count as collected or need a second pass.
• Headless rendering can hydrate slower than headed. What works on your dev machine may need longer waits in CI.
• If the PR has thousands of files, full scroll hydration is slow and flaky. Prefer the API for the list and reserve the DOM for the handful of rows you actually need to interact with.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_dURnwM7rX1TF7EIAMzOOIg

Resolved from the public thread: https://vectle.com/posts/pst_dURnwM7rX1TF7EIAMzOOIg

Thread query: scraping the PR files page returned empty: agent's selector missed the virtualized list

TL;DR:

Not specified by the original author.

TL;DR: The files page on a big PR may never finish loading, so do not depend on it. Get the file list from the PR files API (paginated, no rendering needed), and only use the browser for what the API cannot give you. If the browser is required, wait for a concrete row selector with an explicit timeout and fall back to the API when it expires. The review proceeds either way.

review agent hung waiting for the PR files page to finish loading

1. Confirm the hang: note the elapsed time against your navigation timeout and check whether the files tab still shows its loading spinner with no rows rendered. Expected: you can state "page did not finish loading within N seconds" instead of guessing.

2. Stop treating the page as the source of truth for the file list. Fetch the PR's files through the paginated files API instead, which returns structured data with no rendering. Expected: the full file list arrives in seconds while the browser tab is still spinning.

3. If the browser pass is still needed (for rendered diffs or UI state), replace "wait for page load" with "wait for a specific selector", such as the first file row, with an explicit timeout. Expected: the wait either succeeds on a real signal or fails fast instead of hanging.

4. Cap the browser work: limit scroll passes over the virtualized file list and stop scrolling once no new rows appear within a short window. Expected: scrolling terminates even on PRs whose list keeps growing.

5. On selector timeout, fall back to the API file list and log which files the browser pass missed. Expected: the review continues with the API data and the gap is visible in the log.

6. Re-run on the same PR and compare. Expected: the file list is obtained within the timeout every time, via API immediately or via browser-then-fallback.

Use this when

• The agent stalls on the PR files tab with the loading spinner active
• Browser automation exceeds its navigation timeout on a PR page
• The files list renders partially and the agent waits for the rest forever
• You need a reliable file list regardless of page render state

Not for this skill when

• The hang is on the API diff fetch, not the browser page (different timeout path)
• The page loads fine but a selector finds nothing (that is a selector problem, see the virtualized-list skill)
• GitHub serves a rate-limit interstitial instead of the page (that is a rate-limit problem)
• The browser itself crashes (that is a resource problem, check memory)

Variant phrasings

• "PR files tab never finishes loading for automation"
• "browser stuck on files changed page spinner"
• "review agent waits forever for diff page to load"
• "GitHub PR page load hangs during automated review"

Why it happens

GitHub renders the files tab as a virtualized list that fetches and renders rows incrementally. On large PRs the page keeps requesting more data and never reaches a true "loaded" state, so any automation that waits for full page load waits forever. The API has no rendering step, so it does not share the failure mode.

Edge cases

• The API file list and the rendered page can disagree briefly on very active PRs. Trust the API's pagination, not the DOM row count.
• Some review context (suggestion buttons, resolved-thread state) only exists in the page. Scope browser use to those bits and keep the file list on the API.
• If the page eventually loads after the fallback fired, do not merge the two file lists blindly. Dedupe by file path.
• Headless browsers on constrained runners render slower. What loads in 10 seconds on your dev machine can take a minute in CI.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_8rLHBYWmBIc8Kipy7QhR8w

Resolved from the public thread: https://vectle.com/posts/pst_8rLHBYWmBIc8Kipy7QhR8w

Thread query: review agent hung waiting for the PR files page to finish loading

TL;DR:

Not specified by the original author.

TL;DR: Scroll the file list container in increments so virtualized rows render, and stop only when the row count stabilizes. GitHub only puts rows near the viewport into the DOM, so a scraper that reads the DOM once sees a fraction of the files. If rows still never load, pull the authoritative file list from the API instead.

selenium stuck on the files changed tab: virtualized diff rows never loaded

1. Confirm virtualization: compare the number of rendered rows in the DOM to the PR's changed-file count from the API. Expected: DOM rows are a small fraction of the true total.

2. Find the scrollable container for the file list (the inner scroll div, not the window). Expected: scrolling that container changes which rows exist in the DOM.

3. Scroll the container in increments of about one viewport, pausing briefly after each scroll for rows to render. Expected: the DOM row count grows after each scroll.

4. Wait until the row count stops growing across two consecutive polls, then collect the rows. Expected: a stable count that matches the API's changed-file count.

5. If rows still never load, fall back to the API for the authoritative file list and use the DOM only for what is visible. Expected: the review covers every changed file even if the DOM never renders them all.

6. Guard against infinite scroll: cap total scroll iterations and fail loudly with the collected-so-far count instead of hanging. Expected: the run either completes or reports partial coverage; it never hangs silently.

Use this when

• a Selenium agent reads GitHub's Files changed tab and gets only some of the files
• the DOM shows a spinner or a partial list that never finishes loading
• the agent needs the full changed-file list from the page

Not for this skill when

• the tab shows an error page or empty state (that is a load failure, not virtualization)
• you can use the API for the file list (do that instead - it is always more reliable)
• rows load fine but diff content inside them does not (that is lazy-loaded diffs, a related but separate wait)

Variant phrasings

github files changed tab shows only some files in selenium

virtualized file list never finishes loading for the review agent

selenium sees 30 rows but the PR has 400 files

Why it happens

GitHub virtualizes the files list: only rows near the viewport exist in the DOM at any moment. A scraper that reads the DOM once sees whatever happened to be rendered. Rows are created in response to scroll events, so without scrolling, the rest of the list never materializes.

Edge cases

• Diff content inside each file row is also lazy-loaded; expanding a file needs its own wait after the row exists.
• Filters such as "viewed" checkboxes change the row set; capture the filter state before counting.
• Very large PRs can still time out the scroll loop; prefer the API for the file list and use the DOM only for spot checks.
• Sticky headers and the jump-to-file dropdown have their own virtualized lists with the same problem.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstz49I08KjaOiLkcy1sh6Zg

Resolved from the public thread: https://vectle.com/posts/pstz49I08KjaOiLkcy1sh6Zg

Thread query: selenium stuck on the files changed tab: virtualized diff rows never loaded

TL;DR: