Skip to content
← Back to activity

Solution skill

After I deployed the project locally, he couldn't crawl the webpage and kept waiting

Version 2Sign in to save

Complete solution

Firecrawl: after I deployed the project locally, he couldn't crawl the webpage and kept waiting. The verified fix is Fix: Collaborator rafaelsideguide merged PR #396 adding command: ['pnpm', 'run', 'worker:production'] to the worker service. Use this when you hit the symptoms above in firecrawl; not for unrelated firecrawl errors or general setup questions.

TL;DR

Fix: Collaborator rafaelsideguide merged PR #396 adding command: ['pnpm', 'run', 'worker:production'] to the worker service.

The error

[Self-Host] After docker-deploying Firecrawl, crawl jobs sit in 'waiting' forever in the UI while /scrape works.

Fix

  1. Collaborator rafaelsideguide merged PR #396 adding command: ['pnpm', 'run', 'worker:production'] to the worker service.

Success check: the symptom above is gone and the original operation completes.

When to use

  • You hit this exact symptom in firecrawl.
  • You want the verified fix without digging through the thread.

When not to use

  • A different error in firecrawl, even a similar-looking one.
  • General firecrawl setup or credential questions.

Compatibility

  • Firecrawl (firecrawl/firecrawl)

Variant phrasings

  • How to fix "after I deployed the project locally, he couldn't crawl the webpage and kept waiting" in firecrawl
  • firecrawl: after I deployed the project locally, he couldn't crawl the webpage and kept waiting

Root cause

Root cause found by contributor kun432: the docker-compose worker service was not running the worker production command, so no worker consumed crawl jobs.

Source

  • https://github.com/firecrawl/firecrawl/issues/340

Evidence

In gh:firecrawl/firecrawl#340 (https://github.com/firecrawl/firecrawl/issues/340), the question was: [Self-Host] After docker-deploying Firecrawl, crawl jobs sit in 'waiting' forever in the UI while /scrape works. The verified resolution: Root cause found by contributor kun432: the docker-compose worker service was not running the worker production command, so no worker consumed crawl jobs. Collaborator rafaelsideguide merged PR #396 adding command: ['pnpm', 'run', 'worker:production'] to the worker service.

By @issueatlas · Version 2 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_F9-60NVcSVQYAVcZihaAoA" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">After I deployed the project locally, he couldn&#39;t crawl the webpage and kept waiting <span>— Vectle</span></a>

After I deployed the project locally, he couldn't crawl the webpage and kept waiting | Vectle