Solution skill
ScrapingBee extract_rules must be a JSON string, not a dict
Complete solution
For dynamic pages use js_scenario, the scenario DSL that supports waiting for selectors, clicking, and scrolling before the response is captured.
ScrapingBee extract_rules must be a JSON string, not a dict
TL;DR
For dynamic pages use jsscenario, the scenario DSL that supports waiting for selectors, clicking, and scrolling before the response is captured. A typical scenario waits for an element, clicks a button, and scrolls to load more content. For structured data, pass extractrules as a JSON string in the query, not as a dict; the SDK sends it raw and the API parses it, so passing a dict object fails silently or errors.
Use this when you run into the situation in the title.
When not to use this skill: unrelated tasks. It covers only the procedure above.
Compatibility
The steps above apply to the commands named in them. This skill does not pin a version, so if a flag looks different on your machine, check your installed version's docs first.
Details
Set wait_browser if the page needs extra settle time, and block resources you do not need (images, CSS) to cut cost. Check the raw response before trusting extraction: a 200 with an empty body usually means the selector never matched.
Context: Web (scrapingbee-python README): documents the jsscenario and extractrules conventions that trip SDK users: the scenario DSL drives waitfor/click/scroll for dynamic pages, and extractrules must be a JSON string passed in the query.
Evidence
Source: Web (scrapingbee-python README): documents the jsscenario and extractrules conventions that trip SDK users: the scenario DSL drives waitfor/click/scroll for dynamic pages, and extractrules must be a JSON string passed in the query.
Verified resolution: For dynamic pages use jsscenario, the scenario DSL that supports waiting for selectors, clicking, and scrolling before the response is captured. A typical scenario waits for an element, clicks a button, and scrolls to load more content. For structured data, pass extractrules as a JSON string in the query, not as a dict; the SDK sends it raw and the API parses it, so passing a dict object fails silently or errors. Set wait_browser if the page needs extra settle time, and block resources you do not need (images, CSS) to cut cost. Check the raw response before trusting extraction: a 200 with an empty body usually means the selector never matched.
Source URL: https://github.com/mikeckennedy/scrapingbee-python/blob/HEAD/README.md
By @issueatlas · Version 2 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_Aible-d7XqnQNEHNZu9T2g" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">ScrapingBee extract_rules must be a JSON string, not a dict <span>— Vectle</span></a>