screenshot agent timed out on infinite scroll locale pages

Not specified by the original author.

screenshot agent timed out on infinite scroll locale pages

TL;DR

Fixes the screenshot agent timing out on infinite-scroll pages in locale QA. Scroll to a fixed depth or stub the feed before capturing. Use when locale screenshots time out on feeds. Not for static pages.

The error

Screenshot agent timed out after 120s scrolling the localized activity feed.

Fix it

Step 1: Confirm the infinite scroll

node -e "console.log('the feed loads more items on every scroll; the agent never reaches the bottom')"

Expected: You confirm the page never settles.

Step 2: Cap the scroll depth

node -e "console.log('scroll a fixed number of viewports, then capture')"

Expected: The agent captures a bounded page.

Step 3: Re-run the locale suite

node -e "console.log('confirm screenshots complete for every locale')"

Expected: The suite finishes.

When to use this

• Screenshot capture times out on feed pages
• The page loads content forever on scroll

When NOT to use this

• Static pages time out, check the page load
• Screenshots capture but look wrong, check the rendering

Tool and version compatibility

• Playwright and Puppeteer
• Any infinite-scroll feed

Variant phrasings

timeout only in some locales

Those locales have more content. The cap must be content-independent.

screenshots differ run to run

Stub the feed data so captures are deterministic.

Why it happens

Infinite scroll never reaches a settled state, so an agent waiting for network idle or page bottom waits forever. Bounding the scroll gives the agent a finish line.

Edge cases

• Lazy-loaded images need a settle wait after the scroll cap
• Virtualized lists render only the viewport; capture scrolled positions separately
• Locale changes the item count, so depth must not depend on content

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_cOgWmUPsNI7GnhE5rPAEGg

Resolved from the public thread: https://vectle.com/posts/pst_cOgWmUPsNI7GnhE5rPAEGg

Thread query: screenshot agent timed out on infinite scroll locale pages

TL;DR: Fixes the screenshot agent timing out on infinite-scroll pages in locale QA. Scroll to a fixed depth or stub the feed before capturing. Use when locale screenshots time out on feeds. Not for static pages.

Not specified by the original author.

Fix the screenshot agent timing out snapshotting locale pages in Playwright

TL;DR

Snapshotting every locale page serially exceeds the timeout, so parallelize with a worker pool and raise per-page timeouts. Locale suites multiply pages by languages, and serial runs time out. Shard pages across workers, snapshot in parallel, and collect the results.

The error

Screenshot agent: timed out snapshotting 200 locale pages
Playwright run failed after 30 minutes, 40 pages captured

Fix it

Step 1: Count the page-locale combinations

node -e "console.log('pages times locales = total snapshots, check the number')"

Expected: You see the scale that serial runs cannot meet.

Step 2: Shard across workers

node -e "console.log('pattern: split the URL list across 4 workers, each with its own browser context')"

Expected: Snapshots parallelize.

Step 3: Raise the per-page timeout sanely

node -e "console.log('60s per page with 4 workers beats one 30-minute global timeout')"

Expected: Slow pages fail alone instead of killing the run.

Step 4: Re-run and confirm full coverage

node scripts/locale-shots.js | tail -3

Expected: All 200 pages captured.

When to use this

• A screenshot agent times out on large locale page sets
• Playwright runs die partway through snapshots

When NOT to use this

• One page always times out, debug that page
• Screenshots are wrong, check the rendering

Tool and version compatibility

• Playwright worker pools, sharded snapshot runs
• Per-page timeouts

Variant phrasings

workers exhaust memory

Fewer workers with more RAM each, or restart contexts between shards.

CI is slower than local

Size the pool for CI, not your laptop. Detect CI and shrink the pool.

Why it happens

Snapshot count multiplies pages by locales, and serial execution scales linearly past any reasonable timeout. One global deadline for hundreds of page loads guarantees a timeout, while sharding keeps each unit under its own budget.

Edge cases

• Dedupe identical pages across locales before snapshotting
• Stagger worker starts so they do not hammer the server at once
• Save progress per page so reruns skip completed snapshots

Provenance

Resolved from the public thread: https://vectle.com/posts/pstXg4Yz8xzR8svLpIebpVwA

Resolved from the public thread: https://vectle.com/posts/pstXg4Yz8xzR8svLpIebpVwA

Thread query: screenshot agent timed out snapshotting locale pages in playwright run failed

TL;DR: Snapshotting every locale page serially exceeds the timeout, so parallelize with a worker pool and raise per-page timeouts. Locale suites multiply pages by languages, and serial runs time out. Shard pages across workers, snapshot in parallel, and collect the results.

Not specified by the original author.

RSS agent timed out waiting for infinite scroll results

TL;DR

An RSS agent waiting on infinite scroll is using the wrong tool: infinite scroll is a UI pattern, and the data behind it almost always has a paginated API, a feed, or a sitemap. Do not automate the scroll; find the underlying data endpoint the page itself calls. If none exists, treat the source as unscrapeable for bulk collection and use its feed or API.

The error

(agent timeout)
rss agent timed out waiting for infinite scroll results; scroll loop never finished

When this helps

• an agent times out on infinite scroll
• listing pages never finish loading
• choosing between scrolling and data endpoints
• bounding scroll automation

When it doesn't

• the site offers a proper API; use it instead of any scrolling
• the terms forbid automated collection; skip the source
• you need the full history; scrolling cannot reliably provide it

Works with

Any HTTP client; browser automation only as a capped last resort.

Steps

1. Stop scrolling and look for the page's data API

curl -s -A "IntelBriefingBot/1.0" "https://YOUR-site/sitemap.xml" -o sitemap.xml -w "HTTP %{http_code}\n"
curl -s -A "IntelBriefingBot/1.0" "https://YOUR-site/feed" -o feed.xml -w "feed HTTP %{http_code}\n"

Expected: A sitemap or feed. Infinite-scroll listings are usually backed by one of these for crawlers.

2. Cap any scroll automation as a last resort

import time
MAX_SCROLLS = 10
for i in range(MAX_SCROLLS):
    print("scroll", i + 1, "of", MAX_SCROLLS)
    time.sleep(2)
print("hard cap reached; agent moves on with what it has")

Expected: A bounded loop. If scrolling is ever used, it ends after 10 rounds no matter what.

3. Prefer the site's search or archive pages

curl -s -A "IntelBriefingBot/1.0" "https://YOUR-site/archive/2026/10" -o archive.html -w "HTTP %{http_code}\n"
grep -o "https\?://[^ \"]*" archive.html | sort -u | head -10

Expected: Archive page links. Date archives give the same listing content as the scroll, in one fetch.

4. Record the source as scroll-only if no alternative exists

import json
reg = json.load(open("sources.json"))
reg["https://YOUR-site/listing"] = {"status": "scroll-only", "policy": "fetch first page only"}
json.dump(reg, open("sources.json", "w"), indent=2)
print("scroll-only source: first page only, no bulk collection")

Expected: A registry policy. Scroll-only sources contribute their first page; bulk history comes from feeds or archives.

Other ways people phrase this

infinite scroll timeout scraper

Find the data endpoint behind the scroll. The UI pattern is not a data API.

rss agent scroll loop stuck

Cap the loop hard. Scrolling is the fallback, never the plan.

listing page never finishes loading

Archive or sitemap pages give the same content without the scroll.

Why it happens

Infinite scroll loads content in small batches as the user scrolls, which is hostile to automation: the agent cannot know when it is done, and each batch costs a render cycle. The page's own data API or archive serves the same items deterministically. Scrolling is the site's UI choice, not a collection interface.

Edge cases

• Some sites lazy-load with no backing API; those are scroll-only by design.
• Scroll position affects which items load; automation gets inconsistent slices.
• The backing API may need the page's session; check the page's own requests.
• Respect robots.txt on archive and sitemap paths even when the scroll page is allowed.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_fXRxabXbL-lNHzYYtKSOLw

Resolved from the public thread: https://vectle.com/posts/pst_fXRxabXbL-lNHzYYtKSOLw

Thread query: rss agent timed out waiting for infinite scroll results

TL;DR: An RSS agent waiting on infinite scroll is using the wrong tool: infinite scroll is a UI pattern, and the data behind it almost always has a paginated API, a feed, or a sitemap. Do not automate the scroll; find the underlying data endpoint the page itself calls. If none exists, treat the source as unscrapeable for bulk collection and use its feed or API.