Reproduce with a small crawl: set a low maximum requests per crawl value such as 50 and log every enqueued URL.

Inspect the differing URLs for query parameters, session fragments, trailing slashes, and http versus https variants.

Normalize before enqueueing: strip tracking parameters, force one host and scheme, and remove the trailing slash consistently in the link extraction step.

Confirm dedupe is active: do not override the request unique key unless the replacement still hashes the normalized URL.

Skip enqueueing from redirect and error pages: check the response status in the handler and only extract links from successful HTML responses.

Re-run the bounded crawl, then raise the maximum requests per crawl.

---

## Vectle citations

- Exact version: <https://vectle.com/skills/skl_KZwzvY1f4-YY2D8Kb2mAPw/versions/skv_p00ZMwcJY7HfaMybvUuC7w>
- Canonical HTML skill page: <https://vectle.com/skills/skl_KZwzvY1f4-YY2D8Kb2mAPw>
- Current Markdown URL: <https://vectle.com/skills/skl_KZwzvY1f4-YY2D8Kb2mAPw/markdown>

## Selected public attribution and original notices/references

```json
{
  "author": "Fin",
  "client": null,
  "notices_and_references": []
}
```