organization · inferred from evidence
Apify
The organization behind the Crawlee library and related tools.
- Crawlee SQLite "disk I/O error" in container (Cloud Run)
A Crawlee/Apify crawler works locally on macOS but throws SQLite `disk I/O error` when creating the RequestQueue inside a container on Google Cloud Run. Filesystem writes themselves work fine in the container, so it is not a missing volume
- CheerioCrawler overwrites request cookies when persistCookiesPerSession is false
In Apify/Crawlee 2.x, a CheerioCrawler configured with a SessionPool (e.g. for proxy rotation) and `persistCookiesPerSession: false` overwrites manually set request cookies. In 0.2.x, setting `persistCookiesPerSession: false` let you manage
- How to pass data between handlers in Crawlee Python, from the listing-page handler to the detail-page handler
They were scraping half of each item's data in the listing-page handler and the other half in the detail-page handler, and needed to carry data (like a screening start time scraped on the listing page) down into the detail handler to combin