ScrapingBee transparent_status_code bills every request - and block_resources can silently break pages

ScrapingBee's transparentstatuscode gives you the target's real status and body instead of the API's - but it bills every request, so do not leave it on in production loops. Two more defaults to know: blockresources defaults to true and can silently break pages whose JS needs those assets, so disable it if a rendered page comes back wrong. And file downloads need renderjs=false - non-HTML downloads are capped at 2 MB, and rendering a PDF wastes credits for nothing.

Context: ScrapingBee docs (via de0ch/claude-env SKILL.md): transparentstatuscode=true returns the target's real HTTP status and body - but it bills every request, including the ones that would otherwise be free or retried. Related: sessionid reuses the same IP for 5 minutes (sticky sessions), blockresources defaults to true and can break pages that need their images/CSS/JS to render correctly, and for file downloads use render_js=false since non-HTML downloads are capped at 2 MB.

Source: ScrapingBee docs (via de0ch/claude-env SKILL.md): transparentstatuscode=true returns the target's real HTTP status and body - but it bills every request, including the ones that would otherwise be free or retried. Related: sessionid reuses the same IP for 5 minutes (sticky sessions), blockresources defaults to true and can break pages that need their images/CSS/JS to render correctly, and for file downloads use render_js=false since non-HTML downloads are capped at 2 MB.

Verified resolution: ScrapingBee's transparentstatuscode gives you the target's real status and body instead of the API's - but it bills every request, so do not leave it on in production loops. Two more defaults to know: blockresources defaults to true and can silently break pages whose JS needs those assets, so disable it if a rendered page comes back wrong. And file downloads need renderjs=false - non-HTML downloads are capped at 2 MB, and rendering a PDF wastes credits for nothing.

Source URL: https://github.com/de0ch/claude-env/blob/HEAD/.claude/skills/scrapingbee/SKILL.md