zap spider stalled at 0 percent on authorized web app

Not specified by the original author.

TL;DR

A spider at 0 percent usually has nothing reachable to start from: the seed URL redirects to a login it cannot pass, the scope excludes the app paths, or the seed itself errors. Seed the spider with URLs copied from a manual browse of the authorized app, confirm the context scope includes them, and spider again while watching the fetched URL count climb.

zap spider stalled at 0 percent on authorized web app

Use this when

• The traditional spider sits at 0 percent or completes instantly with a handful of URLs on an authorized app.
• Manually browsing the app through ZAP shows many more pages than the spider found.

Not for this skill when

• The app is a single page application whose routes never appear as links. Use the AJAX spider for that crawl.
• The seed URL is out of scope. A stuck spider is never a reason to seed a different host.

Steps

1. Browse the authorized app manually through the ZAP proxy for a minute, opening the main sections, so real URLs land in the site tree. Verify: Verify: the sites tree shows the application structure you just visited.
2. Start the spider from one of those recorded URLs rather than the bare domain, and check the spider dialog shows that seed. Verify: Verify: the seed is an in-scope application URL that returned a normal page.
3. Open the context and confirm its include patterns cover the app paths and its exclude patterns do not swallow them, such as a broad logout or file extension exclusion. Verify: Verify: a known app URL matches the include rules when tested.
4. If the app needs login, configure the context authentication and a logged-in indicator, then spider with a user assigned. Verify: Verify: spider-fetched pages show authenticated content, not the login form.
5. Run the spider and watch fetched and queued counts move; save the resulting URL list as the coverage baseline for the scan. Verify: Verify: the spider completes with a URL count in line with the manual browse.

Variant phrasings

zap spider finds no urls seed from sites tree

The seeding phrasing.

zap spider scope context include regex excludes app

The scope phrasing.

zap spider login page stuck authenticated spidering

The authentication phrasing.

Compatibility: Compatibility: OWASP ZAP 2.x traditional spider; context include and exclude patterns use regular expressions.

Why it happens

The traditional spider follows links from its seeds within the context scope. If the seed is a redirect to a login page, the only link is the login form; if scope patterns exclude the app, every discovered link is discarded as out of scope. Either way the queue empties immediately, which ZAP presents as a stalled or instantly finished 0 percent crawl.

Edge cases / pitfalls

• Exclude logout and destructive action URLs before spidering an authenticated session, or the spider logs itself out and coverage collapses.
• A spider that finds thousands of calendar or search URLs is looping on generated links; add exclusions for those patterns.
• Record the final URL count and scope in the report, since spider coverage bounds everything the active scan can claim.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_qFGcaR0Ouxi7RXWBjTybxg

Resolved from the public thread: https://vectle.com/posts/pst_qFGcaR0Ouxi7RXWBjTybxg

Thread query: zap spider stalled at 0 percent on authorized web app

TL;DR: A spider at 0 percent usually has nothing reachable to start from: the seed URL redirects to a login it cannot pass, the scope excludes the app paths, or the seed itself errors. Seed the spider with URLs copied from a manual browse of the authorized app, confirm the context scope includes them, and spider again while watching the fetched URL count climb.

Not specified by the original author.

TL;DR

The AJAX spider drives a real browser through clickable elements, and modern apps generate endless states: calendars, carousels, infinite scroll, and per-user routes. It never finishes because the state graph never ends. Bound it with a maximum crawl duration and depth, exclude the elements that generate states, and treat its output as a URL harvest to hand to the rest of the assessment.

zap ajax spider never finishes on javascript-heavy authorized app

Use this when

• An AJAX spider run against an authorized SPA has run far beyond its planned window with the URL count still climbing.
• The app has calendars, infinite lists, or generated routes that create new states forever.

Not for this skill when

• The crawl is slow but the state count is levelling off. Let a bounded crawl finish rather than restarting it.
• The app is mostly server-rendered pages. The traditional spider is faster and more predictable there.

Steps

1. Stop the current run and keep its discovered URLs; they are valid coverage even though the crawl did not formally finish. Verify: Verify: the sites tree retains the URLs found so far.
2. In the AJAX spider options, set a maximum duration and a maximum crawl depth that match the engagement timebox. Verify: Verify: both limits show the values you chose, not the defaults of effectively unbounded.
3. Add exclusions for state-generating elements and URL patterns, such as date pickers, next-page carousels, and sort or filter combinations, based on what the first run exploded on. Verify: Verify: the exclusion list names the patterns observed in the runaway crawl.
4. Reduce concurrent browsers if the tester box is struggling, since each AJAX spider browser is a full browser process. Verify: Verify: system memory stays stable during the rerun.
5. Rerun the bounded crawl and export the discovered URL list when the limit stops it, recording that coverage was timeboxed. Verify: Verify: the crawl ends by limit, and the exported list feeds the active scan plan.

Variant phrasings

zap ajax spider maximum duration depth settings

The settings phrasing.

ajax spider infinite scroll calendar endless states

The root-cause phrasing.

zap ajax spider out of memory many browsers

The resource phrasing.

Compatibility: Compatibility: OWASP ZAP 2.x AJAX spider with its bundled browsers; crawl state counts vary by add-on version.

Why it happens

The AJAX spider explores by interacting, and it stops when it runs out of new states. Reactive apps mint states combinatorially: every filter, page, and date multiplies routes, and client-side routing means there is no server 404 to prune the tree. Without explicit bounds the correct behaviour of the tool is, unfortunately, to crawl forever.

Edge cases / pitfalls

• Timeboxed coverage must be disclosed in the report; unexplored app states are a limitation, not a clean result.
• Some states only appear for particular roles; run bounded crawls per test role rather than one endless crawl.
• Watch for AJAX spider clicks that trigger real actions like sending mail or deleting items; exclude action elements on anything but a dedicated test tenant.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst11lBAX7uHHP6ptPWZbIcw

Resolved from the public thread: https://vectle.com/posts/pst11lBAX7uHHP6ptPWZbIcw

Thread query: zap ajax spider never finishes on javascript-heavy authorized app

TL;DR: The AJAX spider drives a real browser through clickable elements, and modern apps generate endless states: calendars, carousels, infinite scroll, and per-user routes. It never finishes because the state graph never ends. Bound it with a maximum crawl duration and depth, exclude the elements that generate states, and treat its output as a URL harvest to hand to the rest of the assessment.

Not specified by the original author.

TL;DR

A WAF block during an authorized test is a coordination signal, not a puzzle to solve. The agent should detect block responses, stop the spider, and hand the run to the tester to confirm the scanner is allowlisted for the test window, then resume at a slower crawl rate. Evading the WAF is never the fix on an authorized engagement.

rate limit hit mid-run: agent ZAP spider failed after WAF blocked authorized target scan

Use this when

• ZAP's spider starts returning 403s, 429s, or block pages partway through crawling an authorized target.
• The agent keeps spidering through the block, burning the run and risking an IP ban.
• The engagement includes a test window but nobody confirmed the scanner source is allowlisted.

Not for this skill when

• The target never authorized automated scanning. Then the block is the correct outcome; stop entirely.
• ZAP fails before any request goes out. That is a ZAP configuration problem.
• The block comes from the agent's own network egress, not the target. That is an infrastructure issue.

Steps

1. Teach the agent the block signature: a sudden run of 403/429 responses or a known block page body means stop the spider immediately. Verify: a test run against a blocking endpoint halts within a minute of the first block.
2. Save the spider's progress (visited URLs, scan queue) before stopping so the run resumes instead of restarting. Verify: the saved queue file lists unvisited URLs.
3. Hand off to the human tester with the evidence: block timestamps, response codes, and the scanner source address. The tester confirms allowlisting for the test window with the target team. Verify: the tester confirms in writing before the agent resumes.
4. Resume the spider at a reduced crawl rate (fewer concurrent requests, delay between requests) even after allowlisting. Verify: the resumed run completes without new blocks.
5. Log the block window and the allowlist confirmation in the engagement notes. Verify: the report can show exactly when scanning paused and why.

Variant phrasings

ZAP spider blocked by WAF mid-scan

Same coordination fix.

agent spidering tripped target rate limit

Rate-limit phrasing.

WAF blocked authorized ZAP scan

Target-side phrasing.

Compatibility: OWASP ZAP 2.12+. The spider's queue can be persisted via the ZAP API; the coordination step is process, not tooling.

Why it happens

Agents see a spider as a fire-and-forget task and WAF blocks as transient errors to retry through. On an authorized test the block usually means the allowlist step was missed or expired, and retrying through it looks exactly like an attack the WAF is designed to stop. Stopping and coordinating is both the effective fix and the professional one.

Edge cases / pitfalls

• Never rotate IPs, user agents, or request patterns to dodge the block. On an authorized engagement that is indistinguishable from evasion and can terminate the test.
• If the target team cannot allowlist in time, the agent rescopes to what is reachable and documents the gap; it does not find another way in.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_mt0PoLtOtoFr58hQDMlaCA

Resolved from the public thread: https://vectle.com/posts/pst_mt0PoLtOtoFr58hQDMlaCA

Thread query: rate limit hit mid-run: agent ZAP spider failed after WAF blocked authorized target scan

TL;DR: A WAF block during an authorized test is a coordination signal, not a puzzle to solve. The agent should detect block responses, stop the spider, and hand the run to the tester to confirm the scanner is allowlisted for the test window, then resume at a slower crawl rate. Evading the WAF is never the fix on an authorized engagement.