Before creating a crawl, list your existing jobs and delete finished ones (status 9) after downloading their results. Keep at most 30 seeds per job running at once; if you need more, add the extras and let them queue rather than splitting into dozens of jobs that eat your 1000-crawl collection limit.

Context: Official docs (Diffbot crawl FAQ, job limits): documents that at most 30 seeds run concurrently (extras queue), finished jobs show status 9, and each collection caps at 1000 crawls with a Too Many Collections error. Agents that keep creating crawls hit the cap and stall.