string extraction agent timed out on monorepo scan with failed shard

Fix the string extraction agent timing out on a monorepo scan with a failed shard

TL;DR

One shard fails and takes the run down with it, so isolate shard failures and let the rest finish. Run each shard with its own timeout, mark failures per shard, and report at the end instead of aborting. A single bad package should not veto the whole monorepo.

The error

Extraction agent: shard packages/admin timed out after 600s
whole run aborted, 11 of 12 shards never reported

Fix it

Step 1: Identify the slow shard from the logs

grep -n "shard.*timeout\|shard.*failed" logs/extract.log | head -5

Expected: You see which shard died.

Step 2: Give each shard its own timeout and failure flag

node -e "console.log('pattern: run shard with timeout 600, on timeout mark shard failed and continue to the next')"

Expected: Shards fail independently.

Step 3: Re-run and collect per-shard results

node scripts/extract-sharded.js | tail -5

Expected: 11 shards succeed, 1 is marked failed.

Step 4: Fix the slow shard on its own

node -e "console.log('investigate packages/admin separately: huge generated files are the usual cause')"

Expected: The bad shard gets fixed without blocking the rest.

When to use this

• Sharded extraction aborts when one shard fails
• A monorepo scan times out on one package

When NOT to use this

• All shards are slow, the sharding or the extractor needs work
• No sharding is used, add it first

Tool and version compatibility

• Any sharded extraction runner
• Per-shard timeouts and result merging

Variant phrasings

shard fails fast with an error not a timeout

Same isolation. Catch the error per shard, record it, continue.

the same shard always fails

It has a poison file, usually generated. Exclude generated paths for that shard.

Why it happens

Shard runners often treat any shard failure as a run failure and abort. One pathological package, usually with generated files, then denies results for the entire monorepo.

Edge cases

• A failed shard means its strings are missing, the final report must say so loudly
• Retry failed shards once before giving up, transient slowness happens
• Shard by time cost not just by package, balance the shards

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_hLuq2gDU3BDk4EawjCoN2A

Resolved from the public thread: https://vectle.com/posts/pst_hLuq2gDU3BDk4EawjCoN2A

Thread query: string extraction agent timed out on monorepo scan with failed shard

TL;DR: One shard fails and takes the run down with it, so isolate shard failures and let the rest finish. Run each shard with its own timeout, mark failures per shard, and report at the end instead of aborting. A single bad package should not veto the whole monorepo.