ScrapingBee transparent_status_code bills every request - and block_resources can silently break pages

TL;DR

ScrapingBee's transparentstatuscode gives you the target's real status and body instead of the API's - but it bills every request, so do not leave it on in production loops. Two more defaults to know: blockresources defaults to true and can silently break pages whose JS needs those assets, so disable it if a rendered page comes back wrong. And file downloads need renderjs=false - non-HTML downloads are capped at 2 MB, and rendering a PDF wastes credits for nothing.

Fix

1. ScrapingBee's transparentstatuscode gives you the target's real status and body instead of the API's - but it bills every request, so do not leave it on in production loops. Expected: You get the expected result; the problem is gone.

2. Two more defaults to know: block_resources defaults to true and can silently break pages whose JS needs those assets, so disable it if a rendered page comes back wrong. Expected: You get the expected result; the problem is gone.

When to use

• You are dealing with this situation.
• The symptom matches: ScrapingBee transparentstatuscode bills every request - and block_resources can silently break pages.

When NOT to use

• Unrelated issues with a different cause.
• You need general documentation for the tool; check the official docs instead.

Compatibility

Reported per the linked source.

Variant phrasings

ScrapingBee transparentstatuscode bills every request - and block_resources can silently break pages

Why it happens

ScrapingBee docs (via de0ch/claude-env SKILL.md): transparentstatuscode=true returns the target's real HTTP status and body - but it bills every request, including the ones that would otherwise be free or retried. Related: sessionid reuses the same IP for 5 minutes (sticky sessions), blockresources defaults to true and can break pages that need their images/CSS/JS to render correctly, and for file downloads use render_js=false since non-HTML downloads are capped at 2 MB.

Edge cases

• If your error message differs even slightly, this is probably a different issue; search the exact text.
• If the fix does not help, capture the full error output and check the source link for updates.

Source: ScrapingBee docs (via de0ch/claude-env SKILL.md): transparentstatuscode=true returns the target's real HTTP status and body - but it bills every request, including the ones that would otherwise be free or retried. Related: sessionid reuses the same IP for 5 minutes (sticky sessions), blockresources defaults to true and can break pages that need their images/CSS/JS to render correctly, and for file downloads use render_js=false since non-HTML downloads are capped at 2 MB.

Verified resolution: ScrapingBee's transparentstatuscode gives you the target's real status and body instead of the API's - but it bills every request, so do not leave it on in production loops. Two more defaults to know: blockresources defaults to true and can silently break pages whose JS needs those assets, so disable it if a rendered page comes back wrong. And file downloads need renderjs=false - non-HTML downloads are capped at 2 MB, and rendering a PDF wastes credits for nothing.

Source URL: https://github.com/de0ch/claude-env/blob/HEAD/.claude/skills/scrapingbee/SKILL.md