feat(tools): add code coverage tools - #107
Merged
Merged
Conversation
Adds three MCP tools: coverage_analyze, coverage_report and coverage_show. Reading results goes through Microsoft.CodeCoverage.IO rather than Microsoft.CodeCoverage.VisualStudio.Contracts. The contracts assembly ships only in VS 2026 and exposes coverage per file and per line, with no class or module rollup. Microsoft.CodeCoverage.IO ships in both VS 2022 and VS 2026 under CommonExtensions\Microsoft\TestWindow\VsTest, in every edition, and its CoverageFileUtility yields modules whose functions carry NamespaceName and TypeName. Grouping functions by declaring type produces the module, class and method tree the original request asked for, on both Visual Studio versions. ModuleWrapper and Function both derive from CoverageStatistics, so per-module and per-function counts come straight from the reader. Only the class level is derived here. The V1 CoverageFileUtility is used deliberately: it has a parameterless constructor, whereas CoverageFileUtilityV2 requires an ICoverageFileConfiguration that cannot be implemented without a compile-time reference. Unlike the TestWindow and Terminal assemblies, this one has no binding redirect or codeBase entry in devenv.exe.config, so it is loaded by path from VSAPPIDDIR rather than by strong name. Running coverage stays edition-gated: Enterprise only through VS 2022, all editions from VS 2026. coverage_analyze distinguishes an unknown command, which means the edition has no coverage support, from a known but disabled one, which means it is momentarily unavailable, because those need different advice. coverage_report defaults to the newest .coverage file beneath the solution's TestResults folder and accepts an explicit path. Detail level is selectable because a method-level tree over a large solution is far too large to return by default.
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #104
Adds three MCP tools for code coverage.
coverage_analyzecoverage_reportcoverage_showThis corrects my own analysis on #95 and #104
I told you twice that class-level coverage wasn't reachable from the VS API, and that reading results was VS 2026-only. Both were wrong. I had been looking at
Microsoft.CodeCoverage.VisualStudio.Contracts(IReportService), which is VS 2026-only and genuinely is file-and-line only.There is a better assembly I missed:
Microsoft.CodeCoverage.IO.dll, underCommon7\IDE\CommonExtensions\Microsoft\TestWindow\VsTest. It is present in VS 2022 and VS 2026, in every edition, and gives:FunctioncarriesTypeNameandNamespaceName, so grouping by declaring type produces the "specific Module\class Coverage percentage or the fully expanded coverage tree" from the original request in #95 — the item I'd written off.ModuleWrapperandFunctionboth derive fromCoverageStatistics, so per-module and per-function counts come straight from the reader. Only the class level is computed here, which also retires the "we'd have to do the rollup arithmetic ourselves and could get it subtly wrong" concern I raised — it's now one grouping and a sum.Implementation notes
V1 utility on purpose.
CoverageFileUtilityhas a parameterless constructor;CoverageFileUtilityV2requires anICoverageFileConfiguration, which can't be implemented without a compile-time reference. Both expose the same underlying data.Loaded by path, not strong name. Unlike the TestWindow and Terminal assemblies, this one has no binding redirect or codeBase in
devenv.exe.config— I checked. It's loaded from%VSAPPIDDIR%\CommonExtensions\Microsoft\TestWindow\VsTest\, preferring an already-loaded copy.Unknown command vs disabled command.
coverage_analyzedistinguishesCommands.Itemthrowing (this edition has no coverage at all → explain the edition gate) fromIsAvailable == false(momentarily disabled → suggest retrying). Those need different advice and would otherwise both surface as a generic failure.Selectable detail.
summary/class(default) /method, plus a namefilter. A method-level tree over a large solution is far too large to return by default.Partially covered lines count against the total, matching how Visual Studio reports line coverage — 8 covered + 1 partial + 1 uncovered reads as 80%, not 90%. There's a test pinning that specifically because it's the kind of thing that silently drifts.
Workflow this enables
coverage_analyze→ polltest_status(from #105) untilCompleted→coverage_report. The coverage run is a test run, so the state tracking added in #105 is what makes knowing when to read results possible.Testing
dotnet build -c Releaseclean, 0 warnings. 69/69 tests pass (19 added).Same approach as #105/#106: stand-ins declared in the genuine
Microsoft.CodeCoverage.IOnamespaces, supplied via an injectable assembly resolver, so the reflective path runs end to end. The stand-in reader throws if the wrongReadCoverageFileoverload is selected.The arithmetic gets the heaviest coverage — grouping, module rollup, overall rollup, partial-line handling, and a theory over the divide-by-zero edge cases (zero lines must read 0%, not NaN). One of these caught a wrong expectation I'd written while drafting, which is the point of them.
Known limitation
Locating the
.coveragefile is heuristic.ICodeCoverageHosthas no "last report" accessor — VS passes the report around in memory asbyte[].coverage_reportdefaults to the newest.coveragebeneath the solution'sTestResultsfolder and accepts an explicitcoverageFilepath as the reliable escape hatch. I could not verify the default against a real VS coverage run, so that's the first thing worth checking in a live instance.