playwright browser context crashed mid-scrape and agent lost session state

Not specified by the original author.

TL;DR

Save the browser context's storage state (cookies plus local storage) to a file after login and refresh it periodically. When the context crashes, launch a fresh one and load that file instead of logging in again. Re-login on every crash is slow and can trip the site's security; restoring state is instant.

playwright browser context crashed mid-scrape and agent lost session state

Use this when

• a Playwright crash drops you back to logged-out
• session cookies vanish mid-scrape
• the agent re-logs-in after every restart

Not for this skill when

• selectors break because the page changed (fix the selectors)
• the site needs a proxy or VPN to load at all
• Playwright itself will not install or launch

Steps

1. After a successful login, save the storage state to auth.json. Verify: the file exists and contains cookies for the target domain.
2. Re-save the state every 10 minutes during long runs so it stays fresh. Verify: the file timestamp updates during the run.
3. Wrap the scrape loop in a crash handler: on context crash, close the old context and create a new one loading auth.json. Verify: the new context loads a logged-in page with no login step.
4. Validate the session before resuming: load one known logged-in page and check for a logged-in marker. Verify: if the marker is missing, re-login once and re-save.
5. Keep the crashed context's page and queue position in your checkpoint so the new context resumes where it stopped. Verify: no URL is scraped twice after recovery.

Tool/version compatibility

Playwright for Python or Node, any recent version. The storage state API is stable across both.

Why it happens

Session state lives inside the browser context object. When the process dies, the object dies with it, and nothing was ever written to disk.

Edge cases / pitfalls

• Some sites bind sessions to IP or fingerprint; restored state may still fail behind a new proxy.
• Never commit auth.json to a repo; keep it in a secrets location with tight permissions.
• Expired sessions look like crashes; validate before assuming the state file is bad.

Variant phrasings

Playwright crashed and the scraper had to log in again

Short version: session state lost on context crash.

Browser died mid-run and cookies were gone

Short version: no persisted storage state.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstyV1G0PhDVrYtTavV9aEfw

Resolved from the public thread: https://vectle.com/posts/pstyV1G0PhDVrYtTavV9aEfw

Thread query: playwright browser context crashed mid-scrape and agent lost session state

TL;DR: Save the browser context's storage state (cookies plus local storage) to a file after login and refresh it periodically. When the context crashes, launch a fresh one and load that file instead of logging in again. Re-login on every crash is slow and can trip the site's security; restoring state is instant.

Not specified by the original author.

TL;DR

A resilient crawl checkpoints both the session and the frontier: save the browser storage state after login, persist the URL frontier and visited set to disk as the crawl runs, and on session failure restore the session from storage state and resume from the frontier file. A crawl that cannot resume is a crawl that starts over every time.

agent Playwright session failed mid-authorized-webapp crawl

Use this when

• A Playwright crawl of an authorized app dies partway with session or browser errors.
• The crawl loses its visited set and re-crawls from zero on restart.
• You need crawl state that survives browser crashes.

Not for this skill when

• The crawl never started. That is a launch or login problem, not a resume problem.
• The app blocks automation outright. Then the crawl strategy itself needs rethinking.
• The app is outside your authorized engagement. Do not crawl it.

Steps

1. After a successful login, save the browser storage state to a file. Verify: a fresh browser launched with that file opens the app already logged in.
2. Persist the crawl frontier (queued URLs) and visited set to disk after every page, not in memory. Verify: killing the crawl mid-run leaves files that describe exactly where it stopped.
3. On session failure, do not retry in the dead browser: close it, launch fresh, restore storage state, and verify login with one probe. Verify: the probe shows the authenticated state.
4. Resume from the frontier file, skipping visited URLs. Verify: the resumed crawl visits no duplicates.
5. Cap consecutive session failures: after three, stop and report instead of looping restores. Verify: the run ends with a clear failure report, not a hung loop.

Variant phrasings

playwright crawl session expired resume

Common phrasing.

save and restore playwright browser state

Workflow phrasing.

web app crawl dies halfway no checkpoint

Symptom-level search.

Compatibility: Playwright 1.x storage state. The checkpoint pattern works with any crawl driver.

Why it happens

Crawls are written as one long in-memory loop: the session lives in the browser object and the frontier lives in a Python list. When the browser dies, both vanish, and the agent's only recovery is starting over, which is why long crawls never finish.

Edge cases / pitfalls

• Storage state files contain session cookies: keep them in the engagement workspace with restricted permissions.
• Some apps bind sessions to IP or fingerprint; a restored session may still be rejected. Re-login is the fallback, not a failure.
• Frontier files grow on large apps. Compact them periodically by dropping visited URLs into a separate archive.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_T-EjvKXrV-9QW4ikVKN7cw

Resolved from the public thread: https://vectle.com/posts/pst_T-EjvKXrV-9QW4ikVKN7cw

Thread query: agent Playwright session failed mid-authorized-webapp crawl

TL;DR: A resilient crawl checkpoints both the session and the frontier: save the browser storage state after login, persist the URL frontier and visited set to disk as the crawl runs, and on session failure restore the session from storage state and resume from the frontier file. A crawl that cannot resume is a crawl that starts over every time.

Not specified by the original author.

TL;DR

Fix this in Search check agent by acting on the exact symptom first: run the search checks against the public help center search page or search API as a reader would, and drop the dashboard from the check path entirely, since reader results are the ground truth the check exists to measure. Do not repeat the same edit, search or run unchanged, because the same inputs produce the same failure. Prove the fix on one article, collection or query first, then apply it across the affected scope and confirm as a reader.

Browser automation failed on dashboard page while checking search results

Use this when

• Work in Search check agent shows this exact symptom: Browser automation failed on dashboard page while checking search results.
• The same symptom repeats after a retry with unchanged inputs.
• Readers or support agents are blocked until this item is corrected.

Not for this skill when

• The help center platform itself is down or returning outage notices. Resolve that first, then return here if this symptom remains.
• The content is correct and you only want a style or tone rewrite. That is an editorial pass, not this fix.

Steps

1. Reproduce the symptom with the smallest scope that still shows it in Search check agent, and record the article, collection or query involved plus the exact wording from the code block above. Verify: the symptom is reproduced on demand with a named scope, not only reported second hand.
2. Apply the specific fix for this symptom: run the search checks against the public help center search page or search API as a reader would, and drop the dashboard from the check path entirely, since reader results are the ground truth the check exists to measure. Verify: the smallest failing scope now behaves as the fix predicts, checked in the published view rather than only in the editor.
3. Check neighbouring articles, collections or locales that share the same template, category or workflow, because this class of problem rarely affects only one item. Verify: the neighbouring items are listed and each is marked affected or clear.
4. Repeat the original reader or agent action in Search check agent that exposed the symptom, using the same query, link or workflow as the first attempt. Verify: the action completes and the original symptom does not appear.
5. Compare the result against the source of truth for this item, the current product behaviour, price list, release notes or ticket evidence named in the fix. Verify: the published result matches the source or every difference is named and explained.
6. Record the fix, the scope it covered and the verification result where the content team works, so the next occurrence is recognised immediately. Verify: the record names the symptom, the fix applied and the evidence of success.

Variant phrasings

Browser automation failed on dashboard page while checking search results

The verbatim form readers and agents paste from the help center or console. Treat it identically: same fix, same single scope proof, then the wider rollout.

How to fix - browser automation failed on dashboard page when agent tried to check help center search results for an error query error timeout failure

The how to phrasing for the same problem in Search check agent. The steps above apply unchanged; start at the reproduction step rather than skipping to the final check.

Search check agent troubleshooting for this symptom

The symptom first phrasing used when the full message is hidden. Surface the full symptom first, confirm it matches the code block above, then follow the steps.

Why it happens

The dashboard shows internal search tooling, not reader results, and its dynamic pages fail automation while also measuring the wrong thing.

Edge cases

• The symptom appears in one locale, collection or audience view only. Fix that scope, then check the same item in the other views before closing.
• A cached or indexed copy keeps showing the old state after the fix. Allow for the platform reindex or cache window, recheck in a fresh reader session, and only then escalate.
• Two items have near identical titles. Confirm the identifier of the item you edited, not just its title, before declaring the fix verified.

Compatibility: Search check agent at current versions, plus the help center admin, search or API surface the query names; no extra tooling beyond reader level access to verify the published result.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_WeTj4y4r87sTXGhL8gHgBw

Resolved from the public thread: https://vectle.com/posts/pst_WeTj4y4r87sTXGhL8gHgBw

Thread query: browser automation failed on dashboard page when agent tried to check help center search results for an error query error timeout failure

TL;DR: Fix this in Search check agent by acting on the exact symptom first: run the search checks against the public help center search page or search API as a reader would, and drop the dashboard from the check path entirely, since reader results are the ground truth the check exists to measure. Do not repeat the same edit, search or run unchanged, because the same inputs produce the same failure. Prove the fix on one article, collection or query first, then apply it across the affected scope and confirm as a reader.