Skip to content

feat(tools): add code coverage tools - #107

Merged
CalvinAllen merged 1 commit into
mainfrom
feat/tools/coverage-tools
Sep 3, 2026
Merged

feat(tools): add code coverage tools#107
CalvinAllen merged 1 commit into
mainfrom
feat/tools/coverage-tools

Conversation

@CalvinAllen

Copy link
Copy Markdown
Contributor

Resolves #104

Adds three MCP tools for code coverage.

Tool Behavior Availability
coverage_analyze Run all tests with coverage collection Enterprise through VS 2022; all editions from VS 2026
coverage_report Module / class / method tree with line and block counts every edition, both VS versions
coverage_show Open the Code Coverage Results window same as analyze

This corrects my own analysis on #95 and #104

I told you twice that class-level coverage wasn't reachable from the VS API, and that reading results was VS 2026-only. Both were wrong. I had been looking at Microsoft.CodeCoverage.VisualStudio.Contracts (IReportService), which is VS 2026-only and genuinely is file-and-line only.

There is a better assembly I missed: Microsoft.CodeCoverage.IO.dll, under Common7\IDE\CommonExtensions\Microsoft\TestWindow\VsTest. It is present in VS 2022 and VS 2026, in every edition, and gives:

CoverageFileUtility.ReadCoverageFile(path) -> CoverageData
CoverageData.Modules -> ModuleWrapper { Name, Path, TargetFramework, Functions }
Function : CoverageStatistics { Name, NamespaceName, TypeName }
CoverageStatistics { LinesCovered, LinesPartiallyCovered, LinesNotCovered,
                     BlocksCovered, BlocksNotCovered }

Function carries TypeName and NamespaceName, so grouping by declaring type produces the "specific Module\class Coverage percentage or the fully expanded coverage tree" from the original request in #95 — the item I'd written off.

ModuleWrapper and Function both derive from CoverageStatistics, so per-module and per-function counts come straight from the reader. Only the class level is computed here, which also retires the "we'd have to do the rollup arithmetic ourselves and could get it subtly wrong" concern I raised — it's now one grouping and a sum.

Implementation notes

V1 utility on purpose. CoverageFileUtility has a parameterless constructor; CoverageFileUtilityV2 requires an ICoverageFileConfiguration, which can't be implemented without a compile-time reference. Both expose the same underlying data.

Loaded by path, not strong name. Unlike the TestWindow and Terminal assemblies, this one has no binding redirect or codeBase in devenv.exe.config — I checked. It's loaded from %VSAPPIDDIR%\CommonExtensions\Microsoft\TestWindow\VsTest\, preferring an already-loaded copy.

Unknown command vs disabled command. coverage_analyze distinguishes Commands.Item throwing (this edition has no coverage at all → explain the edition gate) from IsAvailable == false (momentarily disabled → suggest retrying). Those need different advice and would otherwise both surface as a generic failure.

Selectable detail. summary / class (default) / method, plus a name filter. A method-level tree over a large solution is far too large to return by default.

Partially covered lines count against the total, matching how Visual Studio reports line coverage — 8 covered + 1 partial + 1 uncovered reads as 80%, not 90%. There's a test pinning that specifically because it's the kind of thing that silently drifts.

Workflow this enables

coverage_analyze → poll test_status (from #105) until Completedcoverage_report. The coverage run is a test run, so the state tracking added in #105 is what makes knowing when to read results possible.

Testing

dotnet build -c Release clean, 0 warnings. 69/69 tests pass (19 added).

Same approach as #105/#106: stand-ins declared in the genuine Microsoft.CodeCoverage.IO namespaces, supplied via an injectable assembly resolver, so the reflective path runs end to end. The stand-in reader throws if the wrong ReadCoverageFile overload is selected.

The arithmetic gets the heaviest coverage — grouping, module rollup, overall rollup, partial-line handling, and a theory over the divide-by-zero edge cases (zero lines must read 0%, not NaN). One of these caught a wrong expectation I'd written while drafting, which is the point of them.

Known limitation

Locating the .coverage file is heuristic. ICodeCoverageHost has no "last report" accessor — VS passes the report around in memory as byte[]. coverage_report defaults to the newest .coverage beneath the solution's TestResults folder and accepts an explicit coverageFile path as the reliable escape hatch. I could not verify the default against a real VS coverage run, so that's the first thing worth checking in a live instance.

Adds three MCP tools: coverage_analyze, coverage_report and coverage_show.

Reading results goes through Microsoft.CodeCoverage.IO rather than
Microsoft.CodeCoverage.VisualStudio.Contracts. The contracts assembly ships only
in VS 2026 and exposes coverage per file and per line, with no class or module
rollup. Microsoft.CodeCoverage.IO ships in both VS 2022 and VS 2026 under
CommonExtensions\Microsoft\TestWindow\VsTest, in every edition, and its
CoverageFileUtility yields modules whose functions carry NamespaceName and
TypeName. Grouping functions by declaring type produces the module, class and
method tree the original request asked for, on both Visual Studio versions.

ModuleWrapper and Function both derive from CoverageStatistics, so per-module
and per-function counts come straight from the reader. Only the class level is
derived here.

The V1 CoverageFileUtility is used deliberately: it has a parameterless
constructor, whereas CoverageFileUtilityV2 requires an ICoverageFileConfiguration
that cannot be implemented without a compile-time reference.

Unlike the TestWindow and Terminal assemblies, this one has no binding redirect
or codeBase entry in devenv.exe.config, so it is loaded by path from VSAPPIDDIR
rather than by strong name.

Running coverage stays edition-gated: Enterprise only through VS 2022, all
editions from VS 2026. coverage_analyze distinguishes an unknown command, which
means the edition has no coverage support, from a known but disabled one, which
means it is momentarily unavailable, because those need different advice.

coverage_report defaults to the newest .coverage file beneath the solution's
TestResults folder and accepts an explicit path. Detail level is selectable
because a method-level tree over a large solution is far too large to return by
default.
@CalvinAllen CalvinAllen mentioned this pull request Sep 3, 2026
@CalvinAllen
CalvinAllen merged commit 1fd01d7 into main Sep 3, 2026
2 checks passed
@CalvinAllen
CalvinAllen deleted the feat/tools/coverage-tools branch September 3, 2026 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

feat(tools): add code coverage tools

1 participant