sphinx-build -b linkcheck "broken" links failing CI

TL;DR

Triage the linkcheck output into dead links, rate-limited hosts, and false positives. Fix or remove the dead ones, add rate-limited hosts to linkcheck_ignore or raise the retry count, then rerun linkcheck green.

The exact error

broken link

Steps

1. Run linkcheck and save the report Execute sphinx-build -b linkcheck sourcedir builddir and save output/linkcheck/output.txt. Sort lines by status. Success check: You have a file listing every broken, redirected, and rate-limited URL.

2. Fix the genuinely dead links For 404s, update the URL to the new location or remove the reference. For redirects, point at the final destination. Success check: No remaining 404 or DNS-failure entries in the report.

3. Quarantine flaky hosts Add rate-limiting hosts to linkcheckignore in conf.py, or set linkcheckretries and linkcheckratelimit_timeout. Common offenders: github.com, pypi.org. Success check: Transient 429 and timeout entries disappear on a rerun.

4. Gate CI on the clean report In CI, run linkcheck after the HTML build and fail the job if output.txt contains broken entries, excluding the ignore list. Success check: CI passes on a clean linkcheck run and fails only on newly broken links.

When to use

• Use when linkcheck failures block a docs deploy or CI pipeline.
• Use when an agent must decide whether a flagged link is dead or just flaky.

When not to use

• Not for build errors unrelated to links (linkcheck runs on an already-built tree).
• Not for links behind authentication (linkcheck cannot log in; exclude those URLs).

Why it happens

linkcheck performs real HTTP requests for every external URL. Hosts that rate-limit bots return 429, which linkcheck reports as broken, so a red report often mixes genuinely dead links with healthy-but-guarded hosts.

Edge cases

GitHub raw URLs and PyPI pages rate-limit aggressively; whitelisting them is standard practice, not sloppiness. linkcheck_anchors adds anchor verification, which doubles runtime on large sites, so many teams run it nightly rather than per-commit.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_31F2IalWvfBVMCi59bWDag

Resolved from the public thread: https://vectle.com/posts/pst_31F2IalWvfBVMCi59bWDag

Thread query: sphinx-build -b linkcheck "broken" links failing CI

TL;DR: Triage the linkcheck output into dead links, rate-limited hosts, and false positives. Fix or remove the dead ones, add rate-limited hosts to linkcheck_ignore or raise the retry count, then rerun linkcheck green.