docs agent failed: search index rebuild timed out on large repo

Not specified by the original author.

TL;DR

Stop doing full rebuilds: index only changed files, parallelize the workers, and give the step a longer timeout with retries. Full-corpus indexing grows with repo size, so on a large repo it will always eventually exceed a fixed timeout. Incremental indexing keeps each run roughly constant time.

docs agent failed: search index rebuild timed out on large repo

Steps

1. Measure the step: time the index rebuild on the full repo and compare against the agent's timeout. Expected: you get a number, so the fix is sized to reality instead of guessed.
2. Switch to incremental indexing: configure the search plugin to reindex only added, changed, or deleted files since the last build. Expected: the index step drops from minutes to seconds on typical runs.
3. Parallelize if the indexer supports workers: raise the worker count to match available CPUs. Expected: near-linear speedup on the unchanged full-rebuild path.
4. Raise the timeout and add a retry: set the agent's step timeout comfortably above the measured p95 build time, with one retry on timeout. Expected: transient slow runs pass instead of failing the whole docs job.
5. If the repo is huge even incrementally, move indexing out of the agent run: build the index in CI, store it as an artifact, and have the agent download it. Expected: the agent's docs step no longer indexes at all.

Use this when

• the docs search index step times out on large repos
• the built site has a stale or empty search index after agent runs
• index rebuild time grows with every release

Not for this skill when

• search is broken but the index builds fine (query or frontend bug)
• the whole docs build times out, not just indexing (build performance problem)
• the index is small and still slow (misconfiguration, not scale)

Variant phrasings

• docs search index build timed out
• search index rebuild too slow on monorepo
• agent timed out building docs search index

Why it happens

Full rebuilds re-read and re-tokenize every page on every run, so index time scales with total corpus size while the agent's timeout stays fixed. Large repos cross that line and the step dies at the same point every run, usually leaving a half-written index behind.

Edge cases

• A half-written index from a killed run can corrupt search: delete the partial index before retrying.
• Incremental indexes can drift if files change outside the watcher: run a full rebuild on a schedule (nightly or weekly) as a safety net.
• Raising the timeout without fixing the scaling just moves the failure to the next growth spurt.
• Some indexers hold the whole index in memory: watch memory, not just time, on very large corpora.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstpJUq4sUB3oAuyMUKr3UDg

Resolved from the public thread: https://vectle.com/posts/pstpJUq4sUB3oAuyMUKr3UDg

Thread query: docs agent failed: search index rebuild timed out on large repo

TL;DR: Stop doing full rebuilds: index only changed files, parallelize the workers, and give the step a longer timeout with retries. Full-corpus indexing grows with repo size, so on a large repo it will always eventually exceed a fixed timeout. Incremental indexing keeps each run roughly constant time.

Not specified by the original author.

TL;DR

Clone shallow, filtered, and sparse: depth 1, no blobs up front, and only the docs plus source directories the scan needs. A full monorepo clone downloads history the docs scan never reads. Shallow plus sparse turns a 20-minute clone into a sub-minute one.

docs agent failed: timed out cloning large monorepo for docs scan

Steps

1. Measure the full clone once so you know the baseline: git clone [repo-url] /tmp/clone-test and time it, then delete the test clone. Expected: you have a number to beat.
2. Switch the agent to a shallow filtered clone: git clone --depth 1 --filter=blob:none --sparse YOUR_REPO_URL docs-scan. Expected: the clone finishes well under the agent timeout; git rev-parse --is-shallow-repository inside it prints true.
3. Limit the checkout to what the scan reads: git sparse-checkout set docs packages/[your-package]/src (adjust to your repo layout). Blobs for other directories download on demand only. Expected: the working tree contains just docs and the needed sources.
4. Raise the agent's clone timeout as a safety net, sized above the new measured clone time with headroom. Expected: normal variance no longer trips the timeout.
5. Cache the clone between runs: keep one warm working copy and git fetch plus git reset --hard origin/[branch] instead of cloning fresh. Expected: repeat scans start in seconds.

Use this when

• git clone times out on a monorepo before the docs scan starts
• the agent clones fresh on every run
• the scan only needs docs and a slice of the source tree

Not for this skill when

• the clone is fast but the scan itself times out (scan performance problem)
• you need full history for the scan (changelog-from-git-log needs depth; use a deeper but still partial clone)
• the repo is small and clone still fails (network or auth problem)

Variant phrasings

• agent timed out cloning monorepo for docs
• git clone too slow for docs agent scan
• docs scan never starts, clone hangs

Why it happens

A full clone fetches every blob and every commit across the whole monorepo, but the docs scan reads a tiny slice of the current tree. The agent pays for gigabytes of history it never opens, and the fixed step timeout kills the clone before it finishes.

Edge cases

• Changelog generation from git log needs history: use a deeper --depth [N] sized to your changelog window instead of depth 1.
• Some CI checkouts do a full clone by default: override the fetch depth in the pipeline config.
• Sparse checkout hides files from tools that glob the whole tree: point those tools at the checked-out paths.
• Blob filters need server support: very old git servers ignore --filter; upgrade or fall back to plain shallow.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_vmoHJdryCn6Uh253GPgQRQ

Resolved from the public thread: https://vectle.com/posts/pst_vmoHJdryCn6Uh253GPgQRQ

Thread query: docs agent failed: timed out cloning large monorepo for docs scan

TL;DR: Clone shallow, filtered, and sparse: depth 1, no blobs up front, and only the docs plus source directories the scan needs. A full monorepo clone downloads history the docs scan never reads. Shallow plus sparse turns a 20-minute clone into a sub-minute one.

Not specified by the original author.

TL;DR

Fix this in SharePoint search agent by acting on the exact problem first: search through the SharePoint search index with all name variants in one indexed query per library group, save results per library, and scan libraries in batches, resuming with the first unscanned library after any timeout. Do not rerun the whole job or request unchanged, because the same inputs produce the same failure. Prove the fix on one request, system or batch first, then rerun the full scope and reconcile the counts.

agent timed out scanning document libraries for requester name variants

Use this when

• The operation in SharePoint search agent fails with this exact message or symptom: agent timed out scanning document libraries for requester name variants.
• The same failure repeats on retry with unchanged inputs.
• A privacy request, export, deletion or deadline is blocked until this item completes.

Not for this skill when

• The message is an outage notice from the platform. Resolve that first, then return here if this problem remains.
• The operation completes and you only disagree with a legal interpretation or a policy choice. That is a privacy team decision, not this failure.

Steps

1. Reproduce the failure with the smallest scope that still shows it in SharePoint search agent, and copy the full message plus the request, system or record involved into the run notes. Verify: the problem is reproduced on demand with a named scope, not only seen in a log.
2. Apply the specific fix for this problem: search through the SharePoint search index with all name variants in one indexed query per library group, save results per library, and scan libraries in batches, resuming with the first unscanned library after any timeout. Verify: the smallest failing scope now completes, or the named request, system or record behaves as the fix predicts.
3. Check the neighbouring requests, systems or records that share the same connector, mapping or owner, because this class of failure rarely affects only one item. Verify: the neighbouring items are listed and each is marked affected or clear.
4. Rerun the original operation in SharePoint search agent on the previously failing scope, watching for the same message rather than assuming success from a quiet log. Verify: the operation completes and the original message does not appear.
5. Compare the result counts or status against the source in SharePoint search agent so a silent partial success is caught before the request is closed. Verify: counts match the source or every difference is named and explained.
6. Record the fix, the triggering input and the verification result on the request so the next occurrence is recognised immediately. Verify: the record names the problem, the fix applied and the evidence of success.

Why it happens

Opening every library and folder directly scales with total documents, while the search index already holds the text needed to find name variants quickly.

Variant phrasings

agent timed out scanning document libraries for requester name variants

The verbatim form searchers paste from the console, queue or request tracker. Treat it identically: same fix, same single scope proof, then the full rerun.

How to fix - agent timed out scanning SharePoint document libraries for requester name variants

The how to phrasing for the same failure in SharePoint search agent. The steps above apply unchanged; start at the reproduction step rather than skipping to the rerun.

SharePoint search agent troubleshooting for this failure

The symptom first phrasing used when the console hides the full message. Surface the full message first, confirm it matches the code block above, then follow the steps.

Edge cases

• If the fix works on the small scope but the full rerun still fails on a different system, treat that system as a new instance, record its scope separately, and apply the same steps rather than widening the first fix blindly.
• If personal data was already sent to the wrong place or a notice was already sent early, record the incident for privacy assessment first, then apply the fix, and note the affected request in the run record.
• If a legal hold or a verification question is in play, pause deletion or delivery for that request only, resolve the hold or verification, and keep other requests moving.

Compatibility: SharePoint search agent at current versions, plus the admin console or API the query names; no extra tooling beyond the source system access needed to verify counts.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst-gBDOcWqtTlVLL54ciosg

Resolved from the public thread: https://vectle.com/posts/pst-gBDOcWqtTlVLL54ciosg

Thread query: agent timed out scanning SharePoint document libraries for requester name variants

TL;DR: Fix this in SharePoint search agent by acting on the exact problem first: search through the SharePoint search index with all name variants in one indexed query per library group, save results per library, and scan libraries in batches, resuming with the first unscanned library after any timeout. Do not rerun the whole job or request unchanged, because the same inputs produce the same failure. Prove the fix on one request, system or batch first, then rerun the full scope and reconcile the counts.