Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,8 @@ Built on `mongo-migrate-ts`. Migrations are TypeScript files with `up()` and `do
## Development

```bash
pnpm start # Interactive guide: pick a task, check prerequisites, run the right command (alias: pnpm scope)
pnpm start --list # Every contributor command, grouped by topic
pnpm docker:up:infra # Start backing services (MongoDB, Redis, Azurite, Lowkey Vault)
pnpm docker:dev:copilot # Full stack with Copilot worker + portal (hot reload)
pnpm docker:dev:all # All workers + portal + report generator (hot reload)
Expand All @@ -204,6 +206,12 @@ build cache, or volumes. Any cleanup must target verified Scope resources and
requires explicit user approval; warn that named volumes may contain local
database or authentication data.

> **Contributor guide catalog**: `pnpm start` only runs existing root scripts listed in
> `scripts/guide/catalog.ts` (menu questions live in `scripts/guide/tree.ts`). When you add,
> rename or remove a contributor-facing root script, update its recipe (title, summary,
> prerequisites, caution) so the guide stays accurate; unregistered scripts only appear under
> "Other scripts". `scripts/guide/*.test.ts` fails if a recipe references a missing script.

### Shared Dev Infrastructure (CosmosDB)

For testing against real Azure CosmosDB (e.g. index behavior), a shared dev instance can be provisioned. Each worktree gets its own isolated database. See [docs/shared-dev-infra.md](docs/shared-dev-infra.md) for setup and usage.
Expand Down
32 changes: 32 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ Scope is a pnpm workspaces monorepo (TypeScript, with a Rust component for the A

```bash
pnpm install # Install all workspace dependencies
pnpm start # Interactive guide to the commands below (alias: pnpm scope)
```

CI installs with `pnpm install --frozen-lockfile`; commit an updated `pnpm-lock.yaml` when you change
Expand All @@ -62,6 +63,37 @@ and its supported JavaScript implementation preserves that behavior without a
Python/compiler toolchain. Keep required dependency build scripts allowlisted in
`pnpm-workspace.yaml`; do not replace this narrow policy with `--ignore-scripts`.

### Interactive contributor guide

Not sure which command to run? `pnpm start` (or its alias `pnpm scope`) asks a few
questions, such as what you want to do and which worker you need, then:

1. checks the prerequisites for that task (such as the Docker daemon, mkcert,
`GITHUB_TOKEN`, `ANTHROPIC_API_KEY`, Rust, built workspace packages or the
generated `.env`), and explains how to fix anything missing;
2. shows the exact `pnpm` command or commands, with a warning when they delete
data, spend provider credits or are experimental;
3. runs them only after you confirm. Destructive commands default to **No**.

```bash
pnpm start # Guided menu; after a run, it offers to repeat your last choice
pnpm start --list # Print every command, grouped by topic
pnpm start --dry-run # Answer the questions and see the commands without running them
```

If `GITHUB_TOKEN` is missing but the GitHub CLI is signed in, the guide offers to
pass `gh auth token` to that one command; the token is never printed or saved.
The last choice is stored per worktree in `.dev-state/guide/last.json`. On a fresh
clone, `pnpm start` offers to run `pnpm install --frozen-lockfile` first. In
non-interactive shells (CI, pipes), it prints the `--list` output instead of
prompting. The guide only runs existing root scripts; you can always run them
directly.

When you add, rename or remove a contributor-facing root script, update its
recipe in [`scripts/guide/catalog.ts`](./scripts/guide/catalog.ts) (and the menu
in [`scripts/guide/tree.ts`](./scripts/guide/tree.ts) if it belongs in the guided
flow). Scripts without a recipe still appear under "Other scripts".

## Local development

### Choose how to run Scope
Expand Down
6 changes: 6 additions & 0 deletions docs/architecture/devcontainer.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,7 @@ Stop native mode or the auth spike first (`pnpm dev:down`); both use the edge po
```bash
pnpm dev:up --mode=kind --services=api,portal --build # first run: builds and loads images
pnpm dev:up --mode=kind --services=api,portal # later runs: reuse loaded images
pnpm dev:logs --service=api # last 100 lines of any Deployment or Job (portal, mongodb, db-migrate…)
pnpm dev:debug --mode=kind --service=api # inspector on 127.0.0.1:9230
pnpm dev:job --service=tests # run a suspended one-shot Job
pnpm dev:smoke --mode=kind # credential-free up -> checks -> down
Expand All @@ -153,6 +154,11 @@ than 10 GiB is free; the warning does not block the build. The devcontainer
configuration forwards port `8080` automatically, so Codespaces users do not
need to add a port mapping manually.

In Kind mode, `pnpm dev:logs` reads the logs of the named Deployment or Job with
`kubectl logs` instead of native log files. To follow logs live or inspect pods,
point kubectl at the workspace kubeconfig (`dev:up` never writes the default one):
`KUBECONFIG=.dev-state/runtime/kubeconfig kubectl -n scope-dev logs -f deployment/api`.

The browser reaches the stack through one origin. The node maps NodePort 30080
to loopback `EDGE_PORT`, and the in-cluster nginx edge routes by path.

Expand Down
2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
"private": true,
"description": "Scoped - coding agent workers (Claude Code, Copilot ACP, VS Code Web) + API + CLI",
"scripts": {
"start": "node scripts/guide/launch.mjs",
"scope": "node scripts/guide/launch.mjs",
"build": "pnpm -r build",
"cli": "pnpm --filter cli exec tsx src/index.ts",
"build:cli": "pnpm --filter shared build && pnpm --filter cli build",
Expand Down
34 changes: 32 additions & 2 deletions scripts/dev/cluster.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,37 @@ export function kubectl(args: string[], env = process.env): void {
execute("kubectl", ["--kubeconfig", kubeconfig, "--context", context, "--namespace", namespace, ...args], env);
}

/**
* Find the rendered Deployment or Job named `name`. Every Kind workload, app or
* backing service (e.g. mongodb, dev-edge, db-migrate), is named after itself.
*
* @throws When no Deployment or Job has that name; the message lists the valid names.
*/
export function findWorkload(env: NodeJS.ProcessEnv, name: string): { kind: "Deployment" | "Job"; resource: Record<string, unknown> } {
const workloads = renderKindStack({ ...env, SCOPE_IMAGE_TAG: project }).filter((item) => item.kind === "Deployment" || item.kind === "Job");
const resource = workloads.find((item) => (item.metadata as { name: string }).name === name);

if (!resource) {
const names = workloads.map((item) => (item.metadata as { name: string }).name).sort();
throw new Error(`Unknown Kind workload ${name}; available: ${names.join(", ")}`);
}
return { kind: resource.kind as "Deployment" | "Job", resource };
}

/** kubectl arguments for a workload's last 100 log lines; for a Deployment, kubectl picks one of its pods. */
export function logsArguments(kind: "Deployment" | "Job", name: string): string[] {
return ["logs", `${kind === "Job" ? "job" : "deployment"}/${name}`, "--tail=100"];
}

/** Print a Kind workload's recent logs, then how to follow them live with kubectl. */
export function showLogs(env: NodeJS.ProcessEnv, name: string): void {
const { kind } = findWorkload(env, name);
const [, target] = logsArguments(kind, name);

kubectl(logsArguments(kind, name), env);
console.log(`\nFollow live: KUBECONFIG=.dev-state/runtime/kubeconfig kubectl -n ${namespace} logs -f ${target}`);
}

export function inspectorCommand(command: string[], port: number): string[] {
if (command[0] !== "node" || !Number.isInteger(port) || port < 1024 || port > 65535) throw new Error("A Node entrypoint and valid inspector port are required");
return ["node", `--inspect-brk=0.0.0.0:${port}`, ...command.slice(1).filter((argument) => argument !== "--watch" && !argument.startsWith("--inspect"))];
Expand All @@ -27,8 +58,7 @@ export function inspectorCommand(command: string[], port: number): string[] {
export async function debugRuntime(env: NodeJS.ProcessEnv, name: string, args: string[] = []): Promise<void> {
const entry = runtimeByName(name);
if (!entry.inspector) throw new Error(`${name} has no Node inspector; use its dedicated debugger profile`);
const resource = renderKindStack({ ...env, SCOPE_IMAGE_TAG: project }).find((item) => (item.kind === "Deployment" || item.kind === "Job") && (item.metadata as { name: string }).name === name);
if (!resource) throw new Error(`Missing workload ${name}`);
const { resource } = findWorkload(env, name);
const workload = resource.kind === "Job" ? "job" : "deployment";
type Container = { name: string; command: string[]; startupProbe?: unknown; readinessProbe?: unknown };
const spec = resource.spec as { suspend?: boolean; template: { spec: { containers: Container[] } } };
Expand Down
8 changes: 8 additions & 0 deletions scripts/dev/control.ts
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,14 @@ async function main(): Promise<void> {
const state = readState();
if (command === "logs") {
const name = option("service") || "api";

// Kind workloads log to the cluster, not to native log files or the native Compose project.
if (state?.mode === "kind" && state.active) {
const cluster = await import("./cluster.js");
cluster.showLogs(inputEnv, name);
return;
}

runtimeByName(name);
const path = resolve(stateDirectory, `${name}.log`);
if (existsSync(path)) execute("tail", ["-n", "100", path]);
Expand Down
20 changes: 19 additions & 1 deletion scripts/dev/kind.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
import { describe, expect, it } from "vitest";
import { MONGO_REPLICA_SET_INIT, renderKindStack } from "./kind.js";
import { runtimes } from "./inventory.js";
import { inspectorCommand } from "./cluster.js";
import { findWorkload, inspectorCommand, logsArguments } from "./cluster.js";

describe("complete Kind stack", () => {
const resources = renderKindStack({});
Expand Down Expand Up @@ -57,4 +57,22 @@ describe("complete Kind stack", () => {
expect(local).toContain("listen 8080 ssl");
expect(local).toContain('"targetPort":8080');
});
});

describe("Kind workload logs", () => {
it("resolves apps and backing services to Deployments and one-shot steps to Jobs", () => {
expect(findWorkload({}, "api").kind).toBe("Deployment");
expect(findWorkload({}, "mongodb").kind).toBe("Deployment");
expect(findWorkload({}, "dev-edge").kind).toBe("Deployment");
expect(findWorkload({}, "db-migrate").kind).toBe("Job");
});

it("lists the valid workload names for an unknown name", () => {
expect(() => findWorkload({}, "nope")).toThrow(/Unknown Kind workload nope; available: .*api.*mongodb/);
});

it("reads the last 100 lines of the matching workload", () => {
expect(logsArguments("Deployment", "api")).toEqual(["logs", "deployment/api", "--tail=100"]);
expect(logsArguments("Job", "db-migrate")).toEqual(["logs", "job/db-migrate", "--tail=100"]);
});
});
105 changes: 105 additions & 0 deletions scripts/guide/catalog.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

import { describe, expect, it } from "vitest";
import {
CATEGORIES,
formatRecipeCommand,
formatStep,
groupByCategory,
HIDDEN_SCRIPTS,
otherScriptRecipes,
PNPM_BUILTINS,
RECIPES,
stepScript,
type Recipe,
} from "./catalog.js";
import { readRootScripts, ROOT } from "./main.js";

const rootScripts = readRootScripts(ROOT);

describe("RECIPES", () => {
it("only runs root scripts that exist, or pnpm built-ins", () => {
const unknown = RECIPES.flatMap((recipe) => recipe.steps.map(stepScript)).filter(
(script) => !(script in rootScripts) && !PNPM_BUILTINS.has(script),
);

expect(unknown).toEqual([]);
});

it("uses unique ids", () => {
const ids = RECIPES.map((recipe) => recipe.id);
expect(new Set(ids).size).toBe(ids.length);
});

it("only allows the last step to be long-running", () => {
// A long-running step never exits on its own, so any later step would never run.
const misplaced = RECIPES.filter((recipe) => recipe.steps.slice(0, -1).some((recipeStep) => recipeStep.longRunning));
expect(misplaced.map((recipe) => recipe.id)).toEqual([]);
});

it("uses known categories and never the generated 'other' category", () => {
const known = new Set<string>(CATEGORIES.map((category) => category.id));

for (const recipe of RECIPES) {
expect(known.has(recipe.category), recipe.id).toBe(true);
expect(recipe.category, recipe.id).not.toBe("other");
}
});
});

describe("HIDDEN_SCRIPTS", () => {
it("only lists scripts that still exist, so stale entries get cleaned up", () => {
const stale = Object.keys(HIDDEN_SCRIPTS).filter((script) => !(script in rootScripts));
expect(stale).toEqual([]);
});

it("includes the guide's own entry points", () => {
expect(rootScripts.start).toBe("node scripts/guide/launch.mjs");
expect(rootScripts.scope).toBe("node scripts/guide/launch.mjs");
expect(HIDDEN_SCRIPTS).toHaveProperty("start");
expect(HIDDEN_SCRIPTS).toHaveProperty("scope");
});
});

describe("otherScriptRecipes", () => {
const curated: Recipe[] = [
{ id: "unit", title: "Unit tests", summary: "", category: "test", steps: [{ pnpm: ["test"] }] },
];

it("wraps unreferenced, non-hidden scripts in sorted fallback recipes", () => {
const scripts = { test: "vitest run", zeta: "echo z", alpha: "echo a", start: "node scripts/guide/launch.mjs" };

const recipes = otherScriptRecipes(scripts, curated);

expect(recipes.map((recipe) => recipe.id)).toEqual(["script:alpha", "script:zeta"]);
expect(recipes[0]).toMatchObject({ title: "alpha", category: "other", summary: "Runs: echo a", steps: [{ pnpm: ["alpha"] }] });
});

it("truncates very long commands in the summary", () => {
const [recipe] = otherScriptRecipes({ long: "x".repeat(500) }, curated);
expect(recipe.summary.length).toBeLessThanOrEqual("Runs: ".length + 160);
expect(recipe.summary.endsWith("…")).toBe(true);
});
});

describe("formatting", () => {
it("prints steps as typed, quoting only arguments a shell would split", () => {
expect(formatStep({ pnpm: ["dev:up", "--mode=kind", "--services=api,portal"] })).toBe("pnpm dev:up --mode=kind --services=api,portal");
expect(formatStep({ pnpm: ["cli", "submit", "-m", "hello world"] })).toBe('pnpm cli submit -m "hello world"');
});

it("joins multi-step recipes with &&", () => {
const recipe = RECIPES.find((candidate) => candidate.id === "pr-check");
expect(recipe && formatRecipeCommand(recipe)).toBe("pnpm lint && pnpm test && pnpm headers:check && pnpm notice:check");
});

it("groups recipes in category order and skips empty categories", () => {
const groups = groupByCategory([
{ id: "b", title: "B", summary: "", category: "test", steps: [] },
{ id: "a", title: "A", summary: "", category: "setup", steps: [] },
]);

expect(groups.map((group) => group.id)).toEqual(["setup", "test"]);
});
});
Loading
Loading