TL;DR: Pin the sort order to a stable option like newest first, or dedupe by project ID across pages. The grid re-sorts by "featured" on every load, so page 2 of this load is not page 2 of the last load and items slip through. After pinning the sort, pagination covers the full list.

```text
agent's Behance scraper kept missing the newest projects  -  the portfolio grid re-sorts by 'featured' on every load and pagination skipped items
```

1. Load the grid twice and compare the order of the first page. Expected: the order differs between loads, confirming the re-sort.
2. Find a stable sort option in the page filters or URL (newest or published date). Expected: switching to it keeps the same order across loads.
3. Update the scraper to use the stable sort and to record each project's ID as it pages. Expected: a second run over the same pages finds zero new IDs it missed.
4. Add a dedupe check: if a page returns IDs already seen, stop instead of continuing. Expected: the scraper terminates cleanly at the end of the list instead of looping.

## Use this when
- Scraped results keep missing the newest items on a portfolio grid
- Page order changes between loads on a site with featured or algorithmic sorting
- Pagination seems to skip items or return duplicates across pages

## Not for this skill when
- The grid order is stable and items are missing for another reason (filters, lazy loading, login walls)
- The site offers no stable sort option and no API (then key everything by ID and crawl until the ID set stops growing)
- The problem is missing images or media, not missing items

## Variant phrasings
### Behance scraper missing new projects due to grid re-sorting
### Pagination skipping items on featured-sorted grid
### Scraper loses newest portfolio items between page loads

## Why it happens
"Featured" and other editorial sorts are recomputed per request, often with randomization or engagement weighting mixed in. Pagination assumes a fixed order, so when the order shifts between page fetches, some items move to already-visited pages and others jump ahead of the crawler. The crawl completes but the coverage has holes.

## Edge cases
- New projects published mid-crawl: with a newest-first sort they appear on page 1 and get picked up on the next run, not the current one
- Sites with no stable sort at all: crawl by ID set convergence (keep paging until no new IDs appear) instead of page numbers
- Infinite-scroll grids: scroll and capture IDs until the scroll stops yielding new ones, then apply the same dedupe logic

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_dPH2yiEM7wSoZ05eKiiJyQ