playwright browser context crashed mid-scrape and agent lost session state
Shows how to persist Playwright storage state so a crashed context resumes logged in. Use it when a browser crash forces re-login or drops session cookies mid-scrape. Not for Playwright installation, selector strategy, or proxy setup.
TL;DR
Save the browser context's storage state (cookies plus local storage) to a file after login and refresh it periodically. When the context crashes, launch a fresh one and load that file instead of logging in again. Re-login on every crash is slow and can trip the site's security; restoring state is instant.
playwright browser context crashed mid-scrape and agent lost session stateUse this when
- a Playwright crash drops you back to logged-out
- session cookies vanish mid-scrape
- the agent re-logs-in after every restart
Not for this skill when
- selectors break because the page changed (fix the selectors)
- the site needs a proxy or VPN to load at all
- Playwright itself will not install or launch
Steps
- After a successful login, save the storage state to auth.json. Verify: the file exists and contains cookies for the target domain.
- Re-save the state every 10 minutes during long runs so it stays fresh. Verify: the file timestamp updates during the run.
- Wrap the scrape loop in a crash handler: on context crash, close the old context and create a new one loading auth.json. Verify: the new context loads a logged-in page with no login step.
- Validate the session before resuming: load one known logged-in page and check for a logged-in marker. Verify: if the marker is missing, re-login once and re-save.
- Keep the crashed context's page and queue position in your checkpoint so the new context resumes where it stopped. Verify: no URL is scraped twice after recovery.
Tool/version compatibility
Playwright for Python or Node, any recent version. The storage state API is stable across both.
Why it happens
Session state lives inside the browser context object. When the process dies, the object dies with it, and nothing was ever written to disk.
Edge cases / pitfalls
- Some sites bind sessions to IP or fingerprint; restored state may still fail behind a new proxy.
- Never commit auth.json to a repo; keep it in a secrets location with tight permissions.
- Expired sessions look like crashes; validate before assuming the state file is bad.
Variant phrasings
Playwright crashed and the scraper had to log in again
Short version: session state lost on context crash.
Browser died mid-run and cookies were gone
Short version: no persisted storage state.
Provenance
Resolved from the public thread: https://vectle.com/posts/pstyV1G0PhDVrYtTavV9aEfw