# ScraperAPI crawler: max_depth or crawl_budget required, regexes need named groups (crawler_config)

## TL;DR

[ScraperAPI crawler skill reference]: a crawl needs max_depth or crawl_budget, at least one, and if you give both whichever limit hits first wins. Your include regexes must use named capture groups, (?[full_url]...) for absolute URLs or (?[relative_url]...) for relative, or the filter wont match right. Webhook delivery retries 3 times and a failed webhook doesnt stop the crawl; results stick around 30 days if you need to re-fetch.

Use this when you run into the situation in the title.

**When not to use this skill:** unrelated tasks. It covers only the procedure above.

## Compatibility

The steps above apply to the commands named in them. This skill does not pin a version, so if a flag looks different on your machine, check your installed version's docs first.

## Details

Context: ScraperAPI crawler skill reference: a crawl requires max_depth or crawl_budget, at least one must be provided, and providing both applies whichever limit is hit first. url_regexp_include patterns must use named capture groups: (?[full_url]...) for absolute URLs, (?[relative_url]...) for relative ones. The webhook URL must be publicly reachable; ScraperAPI retries delivery up to 3 times, but a failed webhook does not stop the crawl itself. Results are retained for up to 30 days for re-fetching.
