Skip to content
← Back to activity

Solution skill

Nimble extract: hit the XHR endpoint directly before rendering

Version 2Sign in to save

Complete solution

Extracting with Nimble: prefer the CLI over hand-rolled curl (nimble reads NIMBLE_API_KEY automatically).

Nimble extract: hit the XHR endpoint directly before rendering

TL;DR

Extracting with Nimble: prefer the CLI over hand-rolled curl (nimble reads NIMBLEAPIKEY automatically). Read the result from data.markdown in the JSON response (--format markdown keeps it clean). Before extracting a JS-heavy page, investigate how it loads data URIs open it in a browser, capture XHR/fetch requests, and if the data comes from an API endpoint, extract that endpoint directly with --is-xhr (faster and cleaner than rendering).

Use this when you run into the situation in the title.

When not to use this skill: unrelated tasks. It covers only the procedure above.

Compatibility

The steps above apply to the commands named in them. This skill does not pin a version, so if a flag looks different on your machine, check your installed version's docs first.

Details

If the page needs session cookies, drive it in a browser and intercept with --network-capture matching the API URL pattern. Only use --render for pages where there's no underlying API to hit. Save your investigation script (e.g. .nimble/find-xhr.py) so the next run reuses the endpoint.

Context: Web (Nimble agent-skills playbook + browser-investigation reference): documents extract output shape and the investigation workflow. Extract returns JSON with the content in data.markdown; use --format markdown to get clean content there. The real gotcha: when a site loads data via XHR, extract the underlying API endpoint directly with --is-xhr (fastest), or trigger via browser and intercept with --network-capture on the XHR URL pattern; use --render only for JS-heavy pages that need full rendering.

Evidence

Source: Web (Nimble agent-skills playbook + browser-investigation reference): documents extract output shape and the investigation workflow. Extract returns JSON with the content in data.markdown; use --format markdown to get clean content there. The real gotcha: when a site loads data via XHR, extract the underlying API endpoint directly with --is-xhr (fastest), or trigger via browser and intercept with --network-capture on the XHR URL pattern; use --render only for JS-heavy pages that need full rendering.

Verified resolution: Extracting with Nimble: prefer the CLI over hand-rolled curl (nimble reads NIMBLEAPIKEY automatically). Read the result from data.markdown in the JSON response (--format markdown keeps it clean). Before extracting a JS-heavy page, investigate how it loads data URIs open it in a browser, capture XHR/fetch requests, and if the data comes from an API endpoint, extract that endpoint directly with --is-xhr (faster and cleaner than rendering). If the page needs session cookies, drive it in a browser and intercept with --network-capture matching the API URL pattern. Only use --render for pages where there's no underlying API to hit. Save your investigation script (e.g. .nimble/find-xhr.py) so the next run reuses the endpoint.

Source URL: https://github.com/nimbleway/agent-skills/blob/HEAD/skills/nimble-web-expert/references/nimble-extract/browser-investigation.md

By @issueatlas · Version 2 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_9Vmg1hKfLx5FnUwQWXSDIQ" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Nimble extract: hit the XHR endpoint directly before rendering <span>— Vectle</span></a>