Immutable solution version
Each and every update to crawlee breaks Dataset.pushData on Apify Platform
Complete solution
If Dataset.pushData misbehaves after a Crawlee update, check for duplicate Crawlee installs: multiple copies each create their own global storage config, which is what breaks pushData on the Apify platform. Use pushData from the crawling context, do not open the request queue explicitly when you do not need to, and pass config: Actor.config so everything shares one config.
If Dataset.pushData misbehaves after a Crawlee update, check for duplicate Crawlee installs: multiple copies each create their own global storage config, which is what breaks pushData on the Apify platform. Use pushData from the crawling context, do not open the request queue explicitly when you do not need to, and pass config: Actor.config so everything shares one config.
Source: gh:apify/crawlee#2117 (Crawlee)
Evidence
In gh:apify/crawlee#2117, the issue was: ### Which package is this bug report for? If unsure which one to select, leave blank @crawlee/core ### Issue description Whenever crawlee is bumped, all the Dataset.pushData functionalities get broken and without any error, they just don't proceed with any. The package.json is the following `` "dependencies": { "apify": "^3.1.11", "cheerio": "1.0.0-rc.12", "crawlee": "^3.5.7", "got-scraping": "3.2.15" }, ### Code sample ``TypeScript import { BasicCrawler, log, Dataset } from 'crawlee'; await Dataset.pushData({a: 1});. The verified answer: If Dataset.pushData misbehaves after a Crawlee update, check for duplicate Crawlee installs: multiple copies each create their own global storage config, which is what breaks pushData on the Apify platform. Use pushData from the crawling context, do not open the request queue explicitly when you do not need to, and pass config: Actor.config so everything shares one config.. Issue URL: https://github.com/apify/crawlee/issues/2117
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_Thx5UCTWAyl8XbA6-CcDBA" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Each and every update to crawlee breaks Dataset.pushData on Apify Platform <span>— Vectle</span></a>