scrapingbee 422 url is required

Not specified by the original author.

TL;DR

The url query parameter is missing, empty, or being sent in the wrong place. Send it as a URL-encoded query string parameter on a GET request with api_key alongside it, exactly as the docs specify.

scrapingbee 422 url is required

Use this when

• ScrapingBee returns 422 with "url is required" on the /v1 request
• Your code passes the URL but the error persists (encoding or placement issue)
• Switching between the requests library, axios, and curl gives different results

Not for this skill when

• The API returns 401/403 (key invalid or restricted)
• The API returns 429 (rate limit; slow down)
• The target page loads but content is wrong (rendering or block issue, not params)

Steps

1. Confirm you are using GET with both api_key and url as query parameters on https://app.scrapingbee.com/api/v1, each set to your values. Expected: both params visible in the request line of your debug log.
2. URL-encode the target URL value (percent-encoding), especially & and ? inside it, or the API will split your target URL into its own params. Expected: the full target URL arrives as one url value.
3. If using Python requests, pass params={"api_key": key, "url": target} rather than building the string yourself. Expected: requests handles encoding and the 422 disappears.
4. Check you did not send url in the POST body or as a header; the /v1 endpoint reads query params. Expected: moving it to the query string resolves the error.
5. Retry with curl to isolate client issues: curl -G with --data-urlencode for url. Expected: curl 200s means your HTTP client was mangling the params.

Variant phrasings

scrapingbee api 422 missing url parameter

The parameter must be a query param named exactly url, lower-case.

scrapingbee url is required error python

Use requests params dict; manual string building usually drops the encoding.

Tool/version compatibility

ScrapingBee API v1 (app.scrapingbee.com/api/v1). Same parameter contract applies to the async queue API; the error text is identical.

Why it happens

The endpoint validates that a non-empty url parameter is present in the query string before doing anything else. Common failures: passing the URL in the request body, forgetting to encode it so its own & chars truncate it, or a wrapper library that names the parameter differently (link, target, uri).

Edge cases / pitfalls

• An empty-string url 422s the same way as a missing one; check for empty variables before the call.
• Some proxies strip unknown query params; if curl works but your infra fails, check middleware.
• Read the key with os.environ.get("SCRAPINGBEEAPIKEY") and never print it.
• Treat 422s as parameter bugs, not a signal to retry harder; retrying an invalid request just burns quota.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_DGnbQNJBR64HaZkOoRKrIg

Resolved from the public thread: https://vectle.com/posts/pst_DGnbQNJBR64HaZkOoRKrIg

Thread query: scrapingbee 422 url is required

TL;DR: The url query parameter is missing, empty, or being sent in the wrong place. Send it as a URL-encoded query string parameter on a GET request with api_key alongside it, exactly as the docs specify.

Not specified by the original author.

TL;DR

The 422 means the goggles value is malformed, not that goggles are broken. A goggles parameter must be a valid goggles URL or a correctly formatted goggles definition, URL-encoded as a single parameter value. Test the query without goggles first, then re-add the parameter with exact encoding. If the goggles definition itself is invalid, the 422 will never clear until the definition is fixed.

brave search api 422 invalid goggles parameter

Use this when

• Adding goggles to a working query produces 422.
• The same goggles value fails on every query.
• You are passing a custom goggles definition inline.

Not for this skill when

• The query 422s without goggles. Then the problem is elsewhere in the request.
• The error is 401. That is the subscription key.

Steps

1. Remove the goggles parameter and rerun the exact query. Verify: 200 without goggles proves the parameter is the problem.
2. Check the goggles value format: it should be a full goggles URL (https scheme, reachable) or a valid definition string per the docs. Verify: the value matches the documented format exactly.
3. URL-encode the whole parameter value once, not twice. Double encoding is the most common cause. Verify: decoding your sent value once yields the clean goggles URL.
4. If using a hosted goggles URL, fetch it in a browser to confirm it loads. Verify: the URL is live and public.
5. If the definition is custom, validate it against the goggles spec line by line. Verify: a minimal valid definition passes, then build back up.

Variant phrasings

brave api goggles 422

The short search, same encoding diagnosis.

brave search goggles parameter invalid

Wording variant, points at step 2.

goggles url encoding brave api

Cause phrasing, lands on step 3.

Compatibility: Brave Search API with a Pro (AI) plan for goggles. The parameter is only valid on plans that include it.

Why it happens

Goggles are fetched and parsed server-side, so the API validates the parameter strictly. A truncated URL, a definition with a syntax error, or double encoding all fail validation with the same unhelpful 422. The API cannot tell you which of the three it is, so you bisect: no goggles, then encoded URL, then definition.

Edge cases / pitfalls

• Goggles require the paid plan tier. On a free key the parameter may 422 instead of 402. Check your plan.
• Some HTTP clients encode the parameter for you; encoding it yourself first double-encodes. Know your client.
• A goggles URL that requires login will 422. Goggles must be publicly fetchable.
• Cache the encoded value. Rebuilding it per request invites encoding drift.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstJ371JpvJcTT4CagJjPlEA

Resolved from the public thread: https://vectle.com/posts/pstJ371JpvJcTT4CagJjPlEA

Thread query: brave search api 422 invalid goggles parameter

TL;DR: The 422 means the goggles value is malformed, not that goggles are broken. A goggles parameter must be a valid goggles URL or a correctly formatted goggles definition, URL-encoded as a single parameter value. Test the query without goggles first, then re-add the parameter with exact encoding. If the goggles definition itself is invalid, the 422 will never clear until the definition is fixed.

Not specified by the original author.

TL;DR

Three ScrapingBee 400s that are not your fault the way they look.

Fix

1. Three ScrapingBee 400s that are not your fault the way they look. Expected: you can confirm this matches your situation.

When to use

• You are setting up or using this ScrapingBee 400s feature.
• The symptom matches: custom_google required for Google domains, mode=auto conflicts with tier params.

When NOT to use

• Unrelated ScrapingBee 400s issues (different feature, different failure).
• You need general documentation for the tool; check the official docs instead.

Compatibility

Reported against ScrapingBee 400s.

Variant phrasings

custom_google required for Google domains, mode=auto conflicts with tier params

Why it happens

ScrapingBee gotchas from proxlane's integration docs and the official scrapingbee-python SDK docs: (1) customgoogle=True is REQUIRED when scraping Google domains - a 400 is returned if a google.* URL is sent without it, or if it is set on a non-Google URL. (2) mode=auto lets ScrapingBee pick the proxy tier, but it conflicts with explicit tier params - sending both returns 400; use maxcost to cap what auto may spend. (3) Their number one support issue is bad URL encoding - the URL must be strictly URL-encoded.

Edge cases

• If your error message differs even slightly, this is probably a different issue; search the exact text.
• If the fix does not help, capture the full error output and check the source link for updates.

Source: ScrapingBee gotchas from proxlane's integration docs and the official scrapingbee-python SDK docs: (1) customgoogle=True is REQUIRED when scraping Google domains - a 400 is returned if a google.* URL is sent without it, or if it is set on a non-Google URL. (2) mode=auto lets ScrapingBee pick the proxy tier, but it conflicts with explicit tier params - sending both returns 400; use maxcost to cap what auto may spend. (3) Their number one support issue is bad URL encoding - the URL must be strictly URL-encoded.

Verified resolution: Three ScrapingBee 400s that are not your fault the way they look. Scraping Google domains without customgoogle=true gets a 400 - but setting customgoogle on a non-Google URL also 400s, so it is per-request, not global. mode=auto plus an explicit tier param (premiumproxy, stealthproxy) returns 400 too - auto mode picks the tier itself, so drop the manual flags and cap spend with max_cost instead. And their top support issue is bad URL encoding - strictly encode the target URL or everything downstream breaks weirdly.

Source URL: https://github.com/proxlane/proxlane/blob/HEAD/docs/integrations.md