feat(tools): add Test Explorer run, debug and stats tools - #105
Merged
Conversation
Adds seven MCP tools over Test Explorer: test_run_all, test_debug_all, test_run, test_debug, test_cancel, test_status and test_stats. The extension cannot reference Microsoft.VisualStudio.TestWindow.Interfaces.dll at compile time. It is absent from the Microsoft.VisualStudio.SDK metapackage, nuget.org carries nothing newer than 11.0.61030 (2012), and $(DevEnvDir) is undefined under dotnet build, which is how this repo builds locally and in CI. TestExplorerInterop therefore resolves ITestExplorerStatsService and IOperationState by MEF contract name and reads them reflectively. The surface needed is one bool, one struct of four ints and one event, and it is identical in VS 2022 17.14 and VS 2026 18.0. Run tools start the run and return immediately, matching the non-blocking contract the build tools were corrected to in #96. Completion is observed by subscribing to IOperationState rather than by blocking, so the UI thread is never held; test_status reports the collapsed run state and current counts. test_stats distinguishes "Test Explorer not initialized" from "no tests", so an uninitialized window cannot be misread as a solution with zero tests. The *InContext commands act on the caret, so test_run and test_debug resolve the requested class or method through the workspace symbol search, open the file and position the caret before issuing the command. Test Explorer exposes no documented cancel command, so test_cancel probes candidate names and uses the first the running Visual Studio recognises. Commands are probed with Command.IsAvailable before execution, because ExecuteCommand throws when a command is disabled, which for Test Explorer is the normal state before discovery finishes.
This was referenced Sep 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #101
Resolves #102
Adds seven MCP tools over Test Explorer.
test_run_alltest_debug_alltest_runtest_debugtest_canceltest_statustest_statsOne deviation from the issues, with reason
Both issues called for referencing
Microsoft.VisualStudio.TestWindow.Interfaces.dllfrom the VS install directory via$(DevEnvDir). That doesn't work. I verified before writing any code:$(DevEnvDir)is only set when building from a Visual Studio / developer command prompt context. This repo builds withdotnet buildboth locally and in CI (vsix-build.ymlstep 3), so a HintPath reference would have broken the build outright. The assembly is also absent from theMicrosoft.VisualStudio.SDKmetapackage, and nuget.org carries nothing newer than 11.0.61030 (2012), which long predatesITestExplorerStatsService.TestExplorerInteroptherefore resolves the two services by MEF contract name and reads them reflectively. The alternative — an MSBuild target shelling out tovswhere— would have made a Visual Studio installation a hard build prerequisite, which it currently isn't. The tradeoff is losing compile-time type safety across six members; the reflection is confined to one file with the rationale in its<remarks>.Design notes
Non-blocking, consistent with #96. The run tools start the run and return immediately. Completion is observed by subscribing to
IOperationState.StateChanged, not by blocking, so the UI thread is never held — this is specifically not a reintroduction of theBuild(true)problem that #96 fixed.test_statuscollapses the rawTestOperationStatesvalues intoNoRunObserved/Discovering/Running/Canceling/Completed/Canceled.A discovery pass doesn't erase a run outcome. Editing code after a run triggers discovery; the state machine keeps the last execution result rather than overwriting it. Covered by a test.
"Not initialized" is not "no tests".
ITestExplorerStatsServicereports zeros before Test Explorer starts.test_statschecksIsTestExplorerStatsServiceRunningand returns an explicit message instead, so an agent can't misread an unopened window as a solution with no tests.Caret positioning.
TestExplorer.RunAllTestsInContextacts on wherever the caret is, sotest_run/test_debugresolve the name through the existing workspace symbol search, open the file, and place the caret before issuing the command. Resolution prefers exact fully-qualified match, then exact simple name, then trailing segment, and reports when the match was ambiguous.Command probing.
ExecuteCommandthrows when a command is disabled, which for Test Explorer is normal before discovery finishes. Commands are checked withCommand.IsAvailablefirst so that's an ordinary result rather than an exception.test_cancelcaveat. Test Explorer exposes no documented cancel command ID —RunAllTests,DebugAllTests,RepeatLastRunand friends are documented, cancel is not. It probesTestExplorer.CancelTestRunthenTestExplorer.CancelTestsand uses whichever the running VS recognizes, reporting cleanly if neither does. Flagging it as the one piece here that would benefit from a check against a live Test Explorer.Testing
dotnet build -c Releaseclean, 0 warnings. 36/36 tests pass (26 added).Rather than only testing the pure helpers, the tests stand up a real MEF
CompositionContainerholding stand-in parts declared in the genuineMicrosoft.VisualStudio.TestWindow.Extensibilitynamespace and shaped like the shipped ones — including implementingStateChangedexplicitly, asOperationBrokerdoes. That exercises the contract-name lookup, interface discovery, property reads and event subscription, which is where the risk in this approach actually lives.Not covered automatically: the DTE command invocations and caret positioning, which need a live VS instance.
Not included
The list of individual failed / not-run test names. As documented in #95, that data is internal and gated behind
InternalsVisibleToon Microsoft's strong-name key.test_stats' description says counts only, so the limitation is visible to the agent rather than silently surprising.