playwright page.goto net::ERR_CONNECTION_TIMED_OUT on news site scrape
This skill fixes Playwright page.goto net::ERR_CONNECTION_TIMED_OUT failures on news sites. It separates network-level blocks from browser-context problems and gives a retry and fallback order.
TL;DR
net::ERRCONNECTIONTIMED_OUT from page.goto means the browser never completed the TCP/TLS handshake with the news site, usually because the site is slow, blocking your network, or dropping connections from your IP. Raise the navigation timeout, retry once with a fresh context, and verify the site loads at all from your network before blaming the page content.
The error
page.goto: net::ERR_CONNECTION_TIMED_OUT at https://example-news-site.com/article
=========================== logs ===========================
navigating to "https://example-news-site.com/article", waiting until "load"
============================================================Fix it
Run a plain curl against the same URL with a 20 second timeout and a browser user agent. This separates network-level blocks from Playwright issues. Expected: Expected: curl either returns HTML (network is fine, the problem is in the browser context) or also times out (network or IP block).
If curl works, relaunch Playwright with a fresh browser context and set an explicit timeout on goto, for example page.goto(url, timeout=60000). Expected: Expected: the navigation log shows the page reaching 'load' or 'domcontentloaded' instead of timing out.
If curl also times out, test from a different egress IP or proxy. Some news sites silently drop connections from datacenter ranges. Expected: Expected: the request succeeds from the alternate egress, confirming an IP-level drop.
For intel pipelines, wrap the fetch in one retry with exponential backoff and fall back to the site's AMP or print view URL on the second attempt. Expected: Expected: at least one of the two URL forms returns content, and the pipeline logs which one worked.
When to use
- Your news-scrape agent's Playwright run fails at navigation with a connection timeout.
- You need to tell a network block apart from a slow page or a bad selector.
- You run scheduled article fetches that must tolerate flaky news-site connectivity.
When not to use
- The error is net::ERRCONNECTIONREFUSED or a DNS failure - those point at a dead host, not a timeout.
- The page loads but your selectors find nothing - that is a parsing problem, not navigation.
- You are scraping behind a login wall - authentication failure looks different from a timeout.
Why it happens
The browser opened a socket and waited for the server to respond, but nothing came back within the timeout window. News sites under load, or sites that drop connections from suspicious IPs instead of answering, produce exactly this. Playwright's default 30 second navigation timeout is often the first thing that gives up.
Variant phrasings
playwright timeout navigating to news article
Same failure and fix; some searchers phrase the query this way.
net errconnectiontimed_out playwright goto fix
Same failure and fix; some searchers phrase the query this way.
Edge cases
- Some sites accept the TCP connection then stall the TLS handshake, which looks identical to a full timeout in logs.
- Headless browsers reuse connections aggressively; a poisoned keep-alive socket can make only some navigations time out.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_kKBLycKyvVr6GNCzstfJkQ