diff --git a/.gitignore b/.gitignore index 0082af2..3379211 100644 --- a/.gitignore +++ b/.gitignore @@ -9,6 +9,16 @@ node_modules/ # but not tracked in git) dist/ +# Demo dashboard artifacts (generated by scripts/build-demo-dashboard.mjs). +# The hero screenshot is committed for the README; the workspace + full +# render are local build artifacts. +docs/demo-workspace/ +docs/demo-dashboard.png +docs/demo-dashboard-full.png + +# Internal planning notes — kept local, not published to the public repo. +docs/visibility-roadmap.md + # Local env / secrets .env .env.local diff --git a/README.md b/README.md index dc28588..6eb652e 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ [![npm version](https://img.shields.io/npm/v/codecartographer-pi.svg)](https://www.npmjs.com/package/codecartographer-pi) [![Node](https://img.shields.io/badge/node-%3E%3D20-brightgreen.svg)](package.json) -> **Evidence-backed software cartography for coding agents.** Turn an unfamiliar repository into validated architecture, contracts, defects, and a reimplementation specification—then combine explicitly confirmed specifications with a product vision to produce a provenance-backed implementation plan. +> **Understand an unfamiliar codebase with an AI agent — and get a validated spec you can rebuild from.** CodeCartographer turns a repository into layered architecture, behavioral contracts, defect findings, and a language-agnostic reimplementation spec, with each phase validated before the next one runs. Works with Pi, Claude Code, Cursor, Codex, or any MCP-capable agent. ```text ● CodeCartographer @@ -19,6 +19,24 @@ ⎿ extracting behavioral contracts from server/index.ts… ``` +

+ CodeCartographer dashboard — a mid-run pipeline with architecture and defect-scan phases complete, contracts in progress, per-phase token and tool-use telemetry. +

+ +--- + +## Why CodeCartographer + +Asking an LLM to "analyze this repo" loses context halfway through, hallucinates findings, and leaves no artifact the next session can pick up. CodeCartographer fixes three things: + +1. **The filesystem is the memory, not the conversation.** Each phase writes a smaller, templated, evidence-tagged artifact to `.codecarto/findings/`. Later phases re-read the specific upstream files they need. A new session — or a context compaction — picks up from `status.yaml` without losing progress. + +2. **Every phase is validated before the pipeline advances.** Completion criteria are real: a `FAIL` output stops the run. You can't accidentally build a reimplementation spec on top of hallucinated architecture. + +3. **The output is a spec, not a chat log.** The final `reimplementation-spec.md` is language-agnostic, module-inventoried, and carries acceptance scenarios plus known unknowns. Hand it to another agent to rebuild from. + +Every finding is tagged with an evidence level: `observed fact`, `strong inference`, `portability hazard`, or `open question`. + --- ## At a glance @@ -36,6 +54,8 @@ Publish completed reimplementation specs from Pi or MCP, then run the `synthesis` pipeline to turn a product vision and explicitly confirmed library entries into a conflict-aware `project-plan.md` with a decision-level provenance ledger. +> **If CodeCartographer saves you a day of codebase archaeology, star the repo** — it helps the next person find it. + OpenAI Build Week reviewers: see the [new-vs-existing scope and one-command demo](docs/build-week-2026.md). --- @@ -70,6 +90,8 @@ pi -e /absolute/path/to/CodeCartographer/extensions/codecarto/index.ts Use this when your coding agent isn't Pi — Claude Code, Codex, opencode, Cursor, Claude Desktop, or anything else that speaks MCP. The host drives the conversation and runs the LLM; CodeCartographer provides phase prompts, validation, and experimental library publish/list/reindex operations. +> **30-second setup for Claude Code, Cursor, Codex, and Claude Desktop: see the [MCP quickstart](docs/mcp-quickstart.md).** + ```bash npm install --global codecartographer-pi ``` diff --git a/docs/demo-dashboard-hero.png b/docs/demo-dashboard-hero.png new file mode 100644 index 0000000..470f1c9 Binary files /dev/null and b/docs/demo-dashboard-hero.png differ diff --git a/docs/mcp-quickstart.md b/docs/mcp-quickstart.md new file mode 100644 index 0000000..b2d6ade --- /dev/null +++ b/docs/mcp-quickstart.md @@ -0,0 +1,174 @@ +# Add CodeCartographer to your coding agent in 30 seconds + +CodeCartographer works with any MCP-capable agent. The MCP server returns phase prompts and validation; your agent drives the conversation and runs the model. One install, one config block, done. + +## Step 1 — Install + +```bash +npm install --global codecartographer-pi +``` + +Verify the binary is on your `PATH`: + +```bash +which codecarto-mcp +``` + +`codecarto-mcp` is a stdio MCP server — running it directly will start it and wait for JSON-RPC input rather than printing anything. That's expected; your agent launches it for you. Press Ctrl-C if you started it by hand. + +## Step 2 — Add to your agent + +Pick your agent below and paste the config block into the right file. That's it. + +### Claude Code + +Add it with the CLI — no config file editing needed: + +```bash +claude mcp add codecartographer -- codecarto-mcp +``` + +Add `--scope user` to make it available in every project instead of just the current one. To check it registered, run `claude mcp list`. + +To share the server with everyone working on a repo, commit a `.mcp.json` at the repo root instead: + +```json +{ + "mcpServers": { + "codecartographer": { + "command": "codecarto-mcp" + } + } +} +``` + +Then in any repo you want to analyze: + +``` +Use the codecartographer MCP server to analyze this repo. Start with codecarto_init, then run codecarto_next repeatedly until the pipeline finishes. +``` + +### Cursor + +Edit `~/.cursor/mcp.json` for all projects, or `.cursor/mcp.json` in a repo root for just that project: + +```json +{ + "mcpServers": { + "codecartographer": { + "command": "codecarto-mcp" + } + } +} +``` + +Then in the Cursor chat, in the repo you want to analyze: + +``` +Use the codecartographer MCP tools to analyze this codebase. Start with codecarto_init, then walk the pipeline with codecarto_next. +``` + +### Codex (OpenAI) + +Codex uses TOML, not JSON. Add this to `~/.codex/config.toml`: + +```toml +[mcp_servers.codecartographer] +command = "codecarto-mcp" +``` + +Then in the repo you want to analyze: + +``` +Use the codecartographer MCP server. Run codecarto_init to set up the workspace, then run codecarto_next to advance through the analysis phases. +``` + +### Claude Desktop + +Edit `claude_desktop_config.json` (macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`): + +```json +{ + "mcpServers": { + "codecartographer": { + "command": "codecarto-mcp" + } + } +} +``` + +Then ask Claude to analyze a repo's code (point it at a local checkout). + +### opencode + +opencode uses an `mcp` key with its own shape — not `mcpServers`, and `command` is an array. Edit `~/.config/opencode/opencode.json`: + +```json +{ + "mcp": { + "codecartographer": { + "type": "local", + "command": ["codecarto-mcp"], + "enabled": true + } + } +} +``` + +## Step 3 — Run + +In the repo you want to understand, ask your agent to run the pipeline. The canonical flow: + +1. **`codecarto_init`** — copies `.codecarto/` into the repo, picks the pipeline variant. +2. **`codecarto_next`** — returns the next phase's prompt. Your agent runs it (reads source, writes findings). +3. **`codecarto_validate`** — checks the phase output against completion criteria. +4. **`codecarto_complete`** — advances `status.yaml`. +5. Repeat `codecarto_next` → validate → complete until the pipeline finishes. + +The final artifact is `.codecarto/findings/reimplementation-spec/reimplementation-spec.md` — a language-agnostic build spec with module inventory, acceptance scenarios, and known unknowns. + +## Pipeline variants + +| Variant | Phases | Use when | +|---|---|---| +| **Full with deep audit** (default) | 7 | Complete analysis with split defect scan | +| **Lite** | 3 | Understand behavior without porting plans | +| **Architecture only** | 1 | Quick structural overview | +| **Synthesis** | 4 | Turn a vision + library specs into an implementation plan | + +Pass `pipeline: ""` to `codecarto_init` to choose. See the [pipeline variants table](../README.md#pipeline-variants) in the README for the full list. + +## What you get + +- `findings/architecture/architecture-map.md` — layers, dependency direction, public surfaces +- `findings/defect-scan-mechanical/mechanical-defects.md` — logic, security, concurrency, API bugs with file:line evidence +- `findings/contracts/behavioral-contracts.md` — behavioral contracts with defaults and acceptance tests +- `findings/protocols/protocols-and-state.md` — event flows, state machines, persistence formats +- `findings/defect-scan-semantic/semantic-defects.md` — deeper semantic defects, run after protocols +- `findings/porting/reverse-engineering-bundle.md` — synthesis bundle with priority rankings +- `findings/reimplementation-spec/reimplementation-spec.md` — the final build spec + +Every finding is tagged: `observed fact`, `strong inference`, `portability hazard`, or `open question`. + +## No agent? Use the drop-in template + +If your tool doesn't speak MCP, copy the template directly: + +```bash +cp -r /path/to/CodeCartographer/.codecarto /path/to/your-repo/ +``` + +Then in any LLM session: `Read .codecarto/GUIDE.md and begin the analysis.` + +The analysis pipeline works fully in drop-in mode. Library publish and synthesis workflows require the MCP server or Pi extension. + +## Troubleshooting + +- **`codecarto_init` says the workspace exists** — pass `force: true` to overwrite (backs up the old `.codecarto/` first). +- **`codecarto_next` returns "no eligible phase"** — all phases are complete. Check `codecarto_status`. +- **`codecarto_validate` returns FAIL** — open the phase's output file, fix the gap, re-run validation. The pipeline won't advance past a FAIL. +- **Agent can't find the MCP server** — confirm `codecarto-mcp` is on your `PATH` (`which codecarto-mcp`). If not, reinstall globally or use the full path in the config. + +## Official MCP Registry + +[io.github.HuginnIndustries/codecartographer](https://registry.modelcontextprotocol.io/?search=CodeCartographer) diff --git a/scripts/build-demo-dashboard.mjs b/scripts/build-demo-dashboard.mjs new file mode 100644 index 0000000..c7b861b --- /dev/null +++ b/scripts/build-demo-dashboard.mjs @@ -0,0 +1,202 @@ +#!/usr/bin/env node +// Build a realistic demo .codecarto workspace (hand-authored findings, no LLM run) +// and render a real dashboard.html via the same writeDashboard() the Pi extension uses. +// Output: /docs/demo-workspace/.codecarto/dashboard.html +// Intended for a README hero screenshot. Safe to delete after the screenshot is taken. + +import { mkdir, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { writeDashboard } from "../dist/extensions/codecarto/dashboard-writer.js"; +import { PACKAGE_VERSION } from "../dist/core/workspace.js"; + +const root = join(process.cwd(), "docs", "demo-workspace"); +const ws = join(root, ".codecarto"); +const findings = join(ws, "findings"); +const workflow = join(ws, "workflow"); +const closeouts = join(ws, "closeouts"); +const scratch = join(ws, "scratch"); + +await mkdir(findings, { recursive: true }); +await mkdir(workflow, { recursive: true }); +await mkdir(closeouts, { recursive: true }); +await mkdir(scratch, { recursive: true }); +await mkdir(join(scratch, "handoffs"), { recursive: true }); + +// ── Pipeline (use the real default shipped pipeline) ──────────────────────── +const pipelineSrc = join(process.cwd(), ".codecarto", "workflow", "pipeline-full-with-deep-audit.yaml"); +const { readFileSync } = await import("node:fs"); +const pipelineYaml = readFileSync(pipelineSrc, "utf8"); +await writeFile(join(workflow, "pipeline-full-with-deep-audit.yaml"), pipelineYaml, "utf8"); + +// ── status.yaml: mid-run, architecture + defect-scan-mechanical complete, contracts running ─ +const now = new Date().toISOString(); +const statusYaml = `project_name: express-starter +schema_version: 1 +pipeline: workflow/pipeline-full-with-deep-audit.yaml +current_phase: contracts +last_updated: ${now} +next_actions: + - "Review the architecture map at findings/architecture/architecture-map.md" + - "Confirm or reject the 3 high-severity defects in findings/defect-scan-mechanical/" + - "Run /codecarto-next to continue the contracts phase" +phases: + architecture: + status: complete + outputs_present: + - findings/architecture/architecture-map.md + owner_notes: + - "Layered MVC confirmed; router/service/model separation is clean" + open_questions: + - id: oq-1 + kind: needs-maintainer-decision + description: "Is the legacy /api/v1 surface still supported, or can the reimplementation drop it?" + deferred_reason: "Affects contracts phase scope" + carry_forward: + - target_phase: contracts + kind: defer-to-phase + description: "v1 surface decision needed before contracts can finalize endpoint coverage" + defect-scan-mechanical: + status: complete + outputs_present: + - findings/defect-scan-mechanical/mechanical-defects.md + owner_notes: + - "3 high-severity, 7 medium, 12 low — all with file:line evidence" + defect-scan-sem: + status: pending + contracts: + status: in-progress + outputs_present: [] + open_questions: + - id: oq-2 + kind: needs-runtime-test + description: "Token-refresh behavior under concurrent requests is inferred from code, not confirmed by a test." + protocols: + status: pending + porting: + status: pending + reimplementation-spec: + status: pending +post_pipeline: + - source_phase: architecture + status: pending + description: "Spike: benchmark alternative ORMs vs the current raw-SQL layer before the porting phase" +`; +await writeFile(join(workflow, "status.yaml"), statusYaml, "utf8"); + +// ── Findings: realistic hand-authored artifacts ───────────────────────────── +await mkdir(join(findings, "architecture"), { recursive: true }); +await mkdir(join(findings, "defect-scan-mechanical"), { recursive: true }); +await mkdir(join(findings, "contracts"), { recursive: true }); + +const archMd = [ + `# Architecture map — express-starter`, + ``, + `## Layers`, + `| Layer | Responsibility | Key files |`, + `|---|---|---|`, + `| HTTP routing | Request parsing, route dispatch | src/routes/*.ts |`, + `| Service | Business logic, validation | src/services/*.ts |`, + `| Data access | Raw SQL via pg, connection pooling | src/db/*.ts |`, + `| Auth | JWT issue/refresh, middleware guards | src/auth/*.ts |`, + ``, + `## Dependency direction`, + `routes → services → data-access (one-way, no back-edges observed).`, + ``, + `## Public surfaces`, + `- REST: /api/v2/* (documented), /api/v1/* (legacy, undocumented)`, + `- CLI: \`npm run migrate\` wraps src/db/migrate.ts`, + ``, + `## Evidence levels`, + `\`observed fact\` (layer boundaries), \`strong inference\` (v1 deprecation intent), \`open question\` (concurrent refresh).`, + ``, +].join("\n"); +await writeFile(join(findings, "architecture", "architecture-map.md"), archMd, "utf8"); + +const defectsMd = [ + `# Defect report — express-starter`, + ``, + `## High severity`, + `| ID | Title | Location | Evidence |`, + `|---|---|---|---|`, + `| D-001 | SQL injection in user-search | src/db/users.ts:42 | String concatenation into query, user input unsanitized |`, + `| D-002 | Refresh token not rotated on use | src/auth/refresh.ts:88 | Old token remains valid after refresh |`, + `| D-003 | Migration runner has no transaction wrapper | src/db/migrate.ts:120 | Partial migration leaves schema half-applied |`, + ``, + `## Medium severity`, + `9 medium findings omitted for brevity — see full report.`, + ``, +].join("\n"); +await writeFile(join(findings, "defect-scan-mechanical", "mechanical-defects.md"), defectsMd, "utf8"); + +// contracts/artifact.md intentionally absent — phase is in-progress (shows realistic state) + +// ── Closeouts for completed phases ────────────────────────────────────────── +const archCloseout = [ + `# Closeout — architecture phase`, + ``, + `## Summary`, + `Architecture map produced. 4-layer MVC confirmed with clean one-way dependency direction. One open question (v1 surface support) deferred to contracts.`, + ``, +].join("\n"); +const archDate = new Date(Date.now() - 1000 * 60 * 60 * 26).toISOString().slice(0, 10); +await writeFile(join(closeouts, `${archDate}-architecture.md`), archCloseout, "utf8"); + +const defectCloseout = [ + `# Closeout — defect-scan-mechanical phase`, + ``, + `## Summary`, + `Mechanical defect scan complete. 3 high-severity findings (SQL injection, token rotation, migration transaction gap), all with file:line evidence. Semantic scan pending.`, + ``, +].join("\n"); +const defectDate = new Date(Date.now() - 1000 * 60 * 60 * 10).toISOString().slice(0, 10); +await writeFile(join(closeouts, `${defectDate}-defect-scan-mechanical.md`), defectCloseout, "utf8"); + +// ── Usage log (realistic token/tool/duration telemetry) ───────────────────── +const archTs = new Date(Date.now() - 1000 * 60 * 60 * 26).toISOString(); +const defectTs = new Date(Date.now() - 1000 * 60 * 60 * 10).toISOString(); +const usageYaml = `version: 1 +runs: + - phase: architecture + timestamp: ${archTs} + status: completed + turn_count: 25 + tool_uses: 76 + duration_ms: 268000 + tokens: + input: 980000 + output: 28000 + cache_write: 14000 + session_file: .codecarto/scratch/sessions/architecture.jsonl + compactions: + successful: 1 + failed: 0 + aborted: 0 + reasons: + threshold: 1 + overflow: 0 + manual: 0 + - phase: defect-scan-mechanical + timestamp: ${defectTs} + status: completed + turn_count: 39 + tool_uses: 91 + duration_ms: 425000 + tokens: + input: 2400000 + output: 24000 + cache_write: 9000 + session_file: .codecarto/scratch/sessions/defect-scan-mechanical.jsonl + compactions: + successful: 2 + failed: 0 + aborted: 0 + reasons: + threshold: 2 + overflow: 0 + manual: 0 +`; +await writeFile(join(workflow, ".usage.local.yaml"), usageYaml, "utf8"); + +// ── Render the dashboard via the real writer ─────────────────────────────── +await writeDashboard(root, PACKAGE_VERSION); +console.log(`Dashboard written to ${join(ws, "dashboard.html")}`); \ No newline at end of file