library · inferred from evidence
Apify/Crawlee
A web scraping and automation library for Node.js, referred to as Crawlee in the context of Apify.
- Crawlee SQLite "disk I/O error" in container (Cloud Run)
A Crawlee/Apify crawler works locally on macOS but throws SQLite `disk I/O error` when creating the RequestQueue inside a container on Google Cloud Run. Filesystem writes themselves work fine in the container, so it is not a missing volume
- RangeError: Map maximum size exceeded when uploading large file to Apify key-value store
Uploading a zip file larger than about 9 MB to an Apify Actor's key-value store with `Actor.setValue()` (crawlee 3.1.3 / apify 3.1.1) throws `RangeError: Map maximum size exceeded`. Building the archive itself takes only ~5 seconds; the cra
- CheerioCrawler overwrites request cookies when persistCookiesPerSession is false
In Apify/Crawlee 2.x, a CheerioCrawler configured with a SessionPool (e.g. for proxy rotation) and `persistCookiesPerSession: false` overwrites manually set request cookies. In 0.2.x, setting `persistCookiesPerSession: false` let you manage
- How to pass data between handlers in Crawlee Python, from the listing-page handler to the detail-page handler
They were scraping half of each item's data in the listing-page handler and the other half in the detail-page handler, and needed to carry data (like a screening start time scraped on the listing page) down into the detail handler to combin