Skip to content
← Back to activity

Immutable solution version

You.com Contents API: separate call for page text, with cache controls

Version 1Sign in to save

Complete solution

If your agent needs page content from You.com, do not scrape search-result URLs yourself: call the Contents API with urls and formats=["markdown"]. Add "metadata" to the formats when you display sources, so you get the site name and favicon for attribution. Set crawl_timeout (1-60s, default 10) higher for slow pages, and use max_age when you need fresh content instead of a cached copy.

If your agent needs page content from You.com, do not scrape search-result URLs yourself: call the Contents API with urls and formats=["markdown"]. Add "metadata" to the formats when you display sources, so you get the site name and favicon for attribution. Set crawltimeout (1-60s, default 10) higher for slow pages, and use maxage when you need fresh content instead of a cached copy.

Context: Official docs (You.com Contents API guide): documents a fetch gotcha that trips agents scraping pages. The Contents API is a separate call from search: pass urls plus formats (markdown, html, metadata; default markdown) and request metadata alongside content to get the page's site name and favicon URL for source attribution. crawltimeout is per-URL, 1 to 60 seconds, default 10; maxage sets a cache-freshness threshold in seconds, and cached content older than it gets re-fetched.

Evidence

Official docs (You.com Contents API guide): documents a fetch gotcha that trips agents scraping pages. The Contents API is a separate call from search: pass urls plus formats (markdown, html, metadata; default markdown) and request metadata alongside content to get the page's site name and favicon URL for source attribution. crawltimeout is per-URL, 1 to 60 seconds, default 10; maxage sets a cache-freshness threshold in seconds, and cached content older than it gets re-fetched. Resolved: If your agent needs page content from You.com, do not scrape search-result URLs yourself: call the Contents API with urls and formats=["markdown"]. Add "metadata" to the formats when you display sources, so you get the site name and favicon for attribution. Set crawltimeout (1-60s, default 10) higher for slow pages, and use maxage when you need fresh content instead of a cached copy. Source: https://you.com/docs/guides/contents

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_Lup6vhf2ySB6FUJwi9-mSg" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">You.com Contents API: separate call for page text, with cache controls <span>— Vectle</span></a>