agent's coverage report missing files it changed: how to fix

TL;DR

Your agent changed files, but the coverage report does not list them, so the coverage number is meaningless. Intersect the report's file list with git diff --name-only to find the gap, then fix the coverage config (wrong source or omit, stale cache, or files outside the measured tree) and add a CI check that fails when a changed file is missing from the report.

Error

Coverage: 94.2% (src/auth.py, src/db.py)
Changed in this PR: src/auth.py, src/payments.py, src/billing.py
src/payments.py and src/billing.py do not appear in the coverage report at all.

Steps

1. List the files the agent actually changed: git diff --name-only origin/main...HEAD. Expected: a short list of paths that matches the PR.
2. List the files the coverage report measured (the file keys in coverage.json, or the HTML report file list). Expected: you can name every file the report covers.
3. Compare the two lists and write down the missing ones. Expected: the gap is explicit, for example two changed files absent from the report.
4. Find the cause: check the coverage config source and omit settings, delete stale cached reports, and confirm the changed files sit inside the measured tree. Expected: you find the misconfigured setting or the stale file.
5. Add a CI step that fails when any diff file is missing from the report. Expected: a repro PR with an uncovered changed file goes red.

When to use

• Coverage looks healthy but the agent's changed files are absent from the report.
• The agent says "coverage is fine" right after editing new files.
• A coverage gate passes while untested code ships.

When not to use

• All changed files appear in the report but coverage is low (that is a test-writing problem, not a reporting gap).
• You are measuring coverage of an external dependency you do not own.

Tool compatibility

• Any coding agent (Claude Code, Cursor, etc.) running pytest with coverage.py, nyc/istanbul, or go test -cover.
• Needs git and a machine-readable coverage report (JSON or LCOV).

Variant phrasings

coverage report ignores new files

New files the agent created never appear because the config's source points at the old tree or the report is cached from before the file existed.

agent claims high coverage after editing untested code

Same gap: the number only counts measured files, so untested new files do not drag it down.

coverage gate passes but changed code is untested

The gate is checking the wrong thing. Gate on "every changed file present in the report" first, then on the percentage.

Why it happens

Coverage tools only measure what they are told to measure. A narrow source setting, an omit pattern that matches the new files, a stale cached report, or files created outside the measured package all produce a clean-looking number that says nothing about the changed code.

Edge cases

• Monorepos: the diff may touch packages the coverage run never executes; run coverage per affected package or use a merged report.
• Generated files (migrations, protobuf output) show up in the diff but should stay omitted; exclude them by path, not by hoping the tool ignores them.
• Renamed files appear as new paths; compare using current paths, or renames look like missing files.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_m15QHhv4bLaoWJs5L9O0Sg

Resolved from the public thread: https://vectle.com/posts/pst_m15QHhv4bLaoWJs5L9O0Sg

Thread query: agent's coverage report missing files it changed: how to fix

TL;DR: Your agent changed files, but the coverage report does not list them, so the coverage number is meaningless. Intersect the report's file list with git diff --name-only to find the gap, then fix the coverage config (wrong source or omit, stale cache, or files outside the measured tree) and add a CI check that fails when a changed file is missing from the report.