Immutable solution version
Use priority queue by number of url finds
Complete solution
The problem
Reported in Crawlee: Use priority queue by number of url finds.
The fix
Wontfix, by design. Crawlee will not get a priority queue that orders URLs by how many other pages link to them: it would have to be implemented across the Apify platform and API, not just in Crawlee, and URL frequency is not stored anywhere (crawlee cannot know a site's link structure upfront). If you need rough prioritization, see the workaround discussed in apify/crawlee#1309.
Evidence
In gh:apify/crawlee#1776 (https://github.com/apify/crawlee/issues/1776), the reporter ran into this: Reported in Crawlee: Use priority queue by number of url finds. The verified resolution: Wontfix, by design. Crawlee will not get a priority queue that orders URLs by how many other pages link to them: it would have to be implemented across the Apify platform and API, not just in Crawlee, and URL frequency is not stored anywhere (crawlee cannot know a site's link structure upfront). If you need rough prioritization, see the workaround discussed in apify/crawlee#1309.
By @issueatlas · Version 1 · Exact version
Maintainer review
No maintainer verification is recorded for this version.
A maintainer review records the library version checked. It does not assert that it is the latest upstream release. Reported outcomes are separate evidence.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_J7iMuhKg20tUX00MUc0zKQ" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Use priority queue by number of url finds <span>— Vectle</span></a>