Skip to content

Replace Datadog integration with verified read-only VS Code tools - #1

Merged
Azayzel merged 5 commits into
mainfrom
feat/vscode-readonly-tool-selection
Oct 31, 2026
Merged

Azayzel merged 5 commits into
mainfrom
feat/vscode-readonly-tool-selection

Conversation

Copilot AI commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Replace the Datadog-specific allowlist and runner with native, searchable VS Code MCP/extension tool selection.
  • Require fresh user verification of read-only tools before every live test or evidence collection; exclude unmarked tools and confirm every invocation. Saved selections/imports cannot grant permission.
  • Give both variants the same tool catalog, preserve frozen-evidence prompt testing, invalidate stale snapshots, and retain partial tool activity on failures/cancellation.
  • Update UI, reports, documentation, and regression coverage while preserving historical Datadog and arbitrary-tool exports.

Safety and limitations

Read-only verification is a user declaration, not sandboxing or proof of provider behavior. Writes, commands, code execution, and shared-state changes are prohibited; independently verified tools and read-only server permissions remain necessary. Only text tool results are supported.

Validation

  • npm run build passed: JavaScript syntax and extension assets verified; all 217 tests passed.
  • git diff --check passed.
  • Live VS Code/MCP integration was not exercised in this CLI session.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Azayzel and others added 4 commits October 30, 2026 21:51
- Introduced a new markdown file for a GitHub issue triage demo showcasing read-only verification and model comparisons.
- Created a GitHub issue triage agent with specific instructions for evidence collection and output formatting.
- Updated app.js to improve UI elements for model comparison, including run summary and tool selection.
- Enhanced CSS for better layout and responsiveness of comparison controls and tool setup sections.
- Modified package.json to reflect new publisher information and set the extension as public.
- Refined copilot.js and extension.js for improved model selection and run actions.
- Updated prompt-evidence.js to ensure proper evidence collection and validation.
- Added tests to verify new UI elements and functionality for model comparison and tool selection.
- Implemented a new webview panel in `panel.js` for visualizing model comparisons and prompt reviews.
- Added HTML structure and styling for user interactions, including model selection, prompt overrides, and response editing.
- Introduced advanced settings for model runs and evidence collection.

test: add architecture tests for module dependencies

- Created `architecture.test.js` to ensure acyclic internal dependencies among source modules.
- Validated that only the extension entry binds VS Code, while other modules load outside the editor.

refactor: reorganize imports in tests for clarity

- Updated import paths in various test files to reflect the new module structure under `comparison` and `providers`.
- Ensured all tests reference the correct modules for functionality and maintainability.

fix: correct history and orchestration references in tests

- Adjusted references in `history.test.js`, `orchestration.test.js`, and related tests to point to the new `records` and `comparison` modules.
- Ensured all tests pass with the updated module paths and structures.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…management

- Added a sidebar launcher view to the CompareLab extension with an icon and welcome message.
- Updated package.json to include new activation events and views configuration.
- Modified build script to include icons in the build process.
- Introduced a new release version management script to handle versioning and GitHub release checks.
- Implemented tests for release versioning logic and sidebar functionality.
- Changed publisher name in package.json and added an icon for the extension.
@Azayzel
Azayzel merged commit 239cf0c into main Oct 31, 2026
1 check passed
Azayzel added a commit that referenced this pull request Oct 31, 2026
Replace Datadog integration with verified read-only VS Code tools
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants