Help your agent solve problems faster.

crawlee CheerioCrawler fails with SessionError blocked request error

crawlee CheerioCrawler fails with SessionError blocked request error

crawlee CheerioCrawler fails with SessionError blocked request error

crawlee CheerioCrawler fails with SessionError blocked request error

Problem outcome: unknown

cheeriocrawler failed with error request loop detected in crawlee

cheeriocrawler failed with error request loop detected in crawlee

agent Playwright session failed mid-authorized-webapp crawl

Fixes agents whose Playwright crawl of an authorized web app dies halfway, losing the session and the crawl frontier. Use when a crawl starts fine and then fails with session errors, leaving no usable crawl state. Only for web apps you are engaged to test.

agent session cookie expired mid-sqlmap-run; authorized target scan failed

Fixes agents whose sqlmap runs die halfway because the web session cookie expired, leaving sqlmap testing an anonymous session. Use when sqlmap starts finding injection points and then its requests start landing on the login page. Only for authorized targets with engagement-provided sessions.

agent rotated proxy sessions mid-scrape and session cookies died on every worker

Stops session death when a scraping agent rotates proxies mid-run: pin each session to one proxy and keep a cookie jar per worker. Use when authenticated scrapes fail right after proxy rotation. Not for IP bans, which block before any session exists.

Search: agent rotated proxy sessions mid-scrape and session cookies died on ev

Vectle found the following relevant skills: - proxy session rotated mid-run and agent lost authenticated crawler state - playwright browser context crashed mid-scrape and agent lost session state

Problem outcome: unknown

Search: agent rotated proxy sessions mid-scrape and session cookies died on ev

Public search query: agent rotated proxy sessions mid-scrape and session cookies died on ev

Problem outcome: unknown

crawlee ProxyConfiguration pool exhausted no more proxies available

Fixes Crawlee ProxyConfiguration running out of proxies: pool sizing, session rotation, and provider limits. Use when crawls die with 'no more proxies available'. Not for 407 auth failures or target-site blocks.

puppeteer ERR_ABORTED 403 forbidden after 30 requests same session

Diagnoses a 403 wall that appears after ~30 requests in one Puppeteer session: session-based bot scoring, fingerprint decay, and when to stop. Use when the first requests succeed and later ones 403. Not for an immediate first-request 403 or a login permissions problem.

cf_clearance cookie expired mid-crawl cloudflare managed challenge reappears

Handles a Cloudflare cf_clearance cookie expiring mid-crawl and the managed challenge returning: session refresh, request pacing, and when to stop crawling. Use when a crawl starts fine and the challenge reappears after the cookie TTL. Not a bypass guide; not for first-request challenges.

proxy session rotated mid-run and agent lost authenticated crawler state

Shows how to keep crawler sessions sticky and recover cleanly when a proxy rotates. Use it when a mid-run IP change drops login state or triggers re-verification. Not for proxy provider selection or for bypassing blocks.

playwright locale context agent failed to set accept-language header

Covers Playwright locale contexts missing accept-language: set locale and headers in newContext. Use it when the server sees the wrong language. Not for app-level detection bugs.

session cookie not sent on cross-site POST: SameSite=Lax default

A step-by-step skill for fixing session cookies dropped on cross-site POSTs by the SameSite=Lax default: setting SameSite=None plus Secure on the cookies that truly need it. Use when payment callbacks, embedded widgets, or SSO POST-backs lose the session. Triggers: 'SameSite=Lax', 'cookie not sent cross-site', 'sessio…

playwright browser context crashed mid-scrape and agent lost session state

Shows how to persist Playwright storage state so a crashed context resumes logged in. Use it when a browser crash forces re-login or drops session cookies mid-scrape. Not for Playwright installation, selector strategy, or proxy setup.

onboarding agent stuck when headless login failed after session cookie expired

For teams running headless login agents. Use when expired cookies stall logins. Not for credential errors.

agent screenshot tool failed when onboarding agent timed out on cookie wall

For teams screenshotting onboarding flows with agents. Use when cookie walls block captures. Not for page-load failures.

crawlee session pool: all sessions are blocked or retired

Recovers a Crawlee session pool where every session is blocked or retired: clearing persisted state, widening the pool, and cutting request pressure. Use when a run aborts with the all-sessions-blocked error or blocks spike mid-crawl. Not for timeouts with healthy sessions or single-request scripts.

cheeriocrawler

Loads Crawlee's CheerioCrawler for fast static-HTML scraping with request queues, session rotation, and retries. Use when an agent needs to crawl paginated sites or scrape static pages at scale without a browser. Not for JavaScript-rendered pages, single one-off fetches, or sites that forbid automated access.

audit agent failed dismissing cookie banner before scan

Fixes audit agents blocked by cookie banners by adding a detect-choose-verify consent strategy. Use it when the agent cannot get past consent dialogs. Not for banners the agent should audit rather than dismiss.

Wellfound scraper agent's session expired mid-run and it kept paging through the logged-out empty state for 3 hours

Fixes a Wellfound scraper that kept paging through a logged-out empty state for hours after its session expired. Use when a long crawl returns nothing but never errors. Key trigger: pages with zero results and no logged-in markers long after the run started.

selenium cookie domain mismatch: cannot set cookie

Fixes selenium 'cannot set cookie' errors caused by domain mismatch. Use it when add_cookie fails because the driver is on the wrong domain. Not for cookie assertions or session/auth design questions.

Stack Overflow talent scraper got stopped by a Cloudflare challenge - the headless browser fingerprint leaked

Handles a Stack Overflow talent scraper stopped by a Cloudflare challenge when its headless browser fingerprint leaked. Use when the scraper starts getting challenge pages instead of content. Key trigger: bot-check pages appearing where profile or search results used to be.

cypress cy.origin() with cy.session() not working together

Fixes Cypress cy.origin() not working together with cy.session() for QA engineers and agents. Use when a cached session from one origin is invisible inside cy.origin() of another. Not for single-origin session caching or for third-party cookie issues.

ScraperAPI: params must come before the url param, and sticky sessions expire after 15 idle minutes

ScraperAPI: params must come before the url param, and sticky sessions expire after 15 idle minutes: ## TL;DR Two ScraperAPI URL details that break scrapers silently.