diff --git a/AGENTS.md b/AGENTS.md index 8742c75e5..3b6782963 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -317,7 +317,7 @@ fail-closed and only pauses when the agent actually proposed a reviewed output. │ ├── github-app-token/ # GitHub App token minter (bundled to github-app-token.js; mints installation token in Agent + Detection when engine.github-app-token is set) │ ├── executor-e2e/ # Stage 3 safe-output E2E test harness (not a bundle; runs deterministic scenarios against a real ADO project and files a GitHub issue on failure) │ ├── compiler-smoke-e2e/ # Smoke E2E orchestrator (not a bundle): stages each case in `tests/smoke/cases.json` to the fixed `.smoke/pipeline.yml` path on its own per-case `ado-aw-mirror` ref, queues it against its credential *lane* definition, and asserts they go green. Two modes via `SMOKE_COMPILER_SOURCE`: `candidate` (compiler built from this commit, pinned pipeline-artifact) and `released` (latest release asset, release URLs required). Built to `test-bin/` by `build:compiler-smoke-e2e`, listed in `NON_BUNDLE_DIRS`. -│ ├── prepare-pr-base/ # create-pull-request preparer (bundled to prepare-pr-base.js): Agent mode uses ADO diff metadata + bounded dual-ref fallback to make the merge-base reachable; SafeOutputs mode fetches only the target worktree tip +│ ├── prepare-pr-base/ # create-pull-request preparer (bundled to prepare-pr-base.js): Agent mode uses ADO diff metadata + bounded fallback; SafeOutputs fetches the target tip; cross-org targets use isolated credentials + exact remote matching │ ├── ado-proxy/ # Credential-isolated ADO policy proxy (bundled to ado-proxy.js). The pipeline mounts it into node:20-slim and starts it before AWF; AWF attaches the trusted container via --topology-attach. scope.ts builds the organization-relative current/additional scope index; catalog.gen.json + ../shared/ado-proxy-catalog.types.gen.ts are generated from Rust by export-ado-proxy-catalog{,-schema} and drift-guarded; a catalog_version mismatch fails closed at startup. │ ├── trigger-e2e/ # Test-only gate-spec / trigger-evaluation harness (not a bundle): mirrors Rust `Fact::ALL` in `gate-spec.ts`; `fact-catalog.gen.json` is generated by `export-fact-catalog` and drift-guarded by CI │ └── shared/ # Shared modules across bundles (auth, ado-client, env-facts, types.gen.ts) diff --git a/docs/ado-proxy-design.md b/docs/ado-proxy-design.md index c268f7a5e..666edd937 100644 --- a/docs/ado-proxy-design.md +++ b/docs/ado-proxy-design.md @@ -267,7 +267,7 @@ verify the first certificate`. ### Credential delivery -`generate_acquire_ado_token` emits an `AzureCLI@2` +`acquire_ado_token_step` emits an `AzureCLI@3` step that mints an ADO-audience token from the ARM service connection and stores it as the secret pipeline variable `SC_READ_TOKEN`. diff --git a/docs/ado-script.md b/docs/ado-script.md index a633f1fd1..1fd93bff3 100644 --- a/docs/ado-script.md +++ b/docs/ado-script.md @@ -79,10 +79,13 @@ pipeline** as runtime helpers. Today it produces thirteen bundles: shallow source/target ranges and verifies the base locally. Ineligible or unavailable REST falls back to bounded dual-ref depths 200/500/2000, never an automatic full-history fetch. SafeOutputs `target-worktree` mode fetches only - the target tip at depth 1. Each allowed repo is passed as a typed - `--repo-dir` / `--source-ref` / `--target-branch` tuple; per-dir failures are - isolated and surfaced as ADO warnings. The bearer - (`SYSTEM_ACCESSTOKEN`) remains in masked env and spawned-git + the target tip at depth 1. Cross-org repositories are partitioned into a + separate trusted credential scope and passed with validated + organization/project/repository coordinates. The checkout remote must match + those coordinates exactly before the Bearer reaches REST or git; mismatch or + preparation failure stops the trusted task before Agent/executor execution. + Same-org per-dir failures remain isolated warnings. The bearer remains + shell-local or in masked `SYSTEM_ACCESSTOKEN` env and spawned-git `GIT_CONFIG_*`, never argv or `.git/config`. Runs outside AWF. See [`safe-outputs.md`](safe-outputs.md#create-pull-request). diff --git a/docs/front-matter.md b/docs/front-matter.md index 122c903f4..0288deb05 100644 --- a/docs/front-matter.md +++ b/docs/front-matter.md @@ -258,7 +258,7 @@ network: # optional network policy (standalone target only # variable-groups: # optional: import ADO Library variable groups (standalone/1es only) # - My Variable Group # each entry must be the exact ADO Library group name (see "Variable Groups" section) permissions: # optional ADO access token configuration (see docs/network.md#permissions-ado-access-tokens) - read: my-read-arm-connection # shorthand: proxy gets the ARM SC token; Agent/MCP/az get no real token + read: my-read-arm-connection # shorthand: AzureCLI@3 connectionType azureRM # read: # object form: narrow capabilities / add cross-org or project scope # service-connection: my-read-arm-connection # capabilities: [core, repos] # discovery is always enabled @@ -268,10 +268,15 @@ permissions: # optional ADO access token configuration (see do # - project: Shared # project-id: 33333333-3333-3333-3333-333333333333 # optional GUID-form calls # repositories: [shared-api] # empty/omitted => project reads only - write: my-write-arm-connection # OPTIONAL ARM SC for Stage 3 executor writes. - # Default: executor uses $(System.AccessToken). - # Set this only for cross-org writes or - # named-identity attribution. + write: my-write-arm-connection # shorthand: AzureCLI@3 connectionType azureRM + # write: # expanded form for scoped cross-org repo writes + # service-connection: ado-repository-writer + # connection-type: azureDevOps + # allow: + # - organization: partner-org + # projects: + # - project: Shared + # repositories: [shared-api] # permissions-required: # optional abstract capability requirements (usually set by an # read: true # imported component rather than authored directly); see # write: true # docs/imports.md#permissions-required. Unioned across all @@ -539,19 +544,20 @@ Each entry can be: | Form | Syntax | Description | |------|--------|-------------| -| **Shorthand** | `- org/repo` | Alias derived from last segment, type=git, ref=refs/heads/main, checkout=true | -| **Shorthand with alias** | `- alias=org/repo` | Explicit alias before `=` | -| **Object** | `- name: org/repo` | Full control over all fields | +| **Shorthand** | `- project/repo` | Alias derived from last segment, type=git, ref=refs/heads/main, checkout=true | +| **Shorthand with alias** | `- alias=project/repo` | Explicit alias before `=` | +| **Object** | `- name: project/repo` | Full control over all fields | Object fields: | Field | Default | Description | |---------------|------------------------|-------------| -| `name` | *(required)* | Full `org/repo` name (maps to ADO `name:`) | +| `name` | *(required)* | Azure Repos `project/repository` name (maps to ADO `name:`) | | `alias` | last segment of `name` | Repository alias (maps to ADO `repository:`) | | `type` | `git` | ADO repository resource type | | `ref` | `refs/heads/main` | Branch or tag reference | -| `endpoint` | *(none)* | Azure DevOps service connection. Required for `type: github`, `githubenterprise`, or `bitbucket`; not needed for same-org Azure Repos `git`. | +| `endpoint` | *(none)* | Checkout service connection. Required for external providers and cross-organization Azure Repos `git`. | +| `organization` | *(current organization)* | Target Azure DevOps organization for cross-org `type: git`; requires object form and `endpoint`. | | `checkout` | `true` | Whether the agent job clones this repo | | `fetch-depth` | *(ADO default)* | Shallow-clone depth for this repo's checkout (ADO `fetchDepth`). `0` = full history | | `fetch-tags` | *(ADO default)* | Whether to fetch git tags during checkout (ADO `fetchTags`) | @@ -560,12 +566,13 @@ Aliases must be unique case-insensitively because they become checkout directory names on Windows agents. `root`, `repo`, and `self` are reserved in every casing; `self` is the compiler-owned path for the pipeline repository. -> **Cross-organization `type: git` repositories.** A `type: git` entry with an -> `endpoint:` set (used for a repository outside the pipeline's own Azure -> DevOps organization) checks out correctly, but `create-pull-request` cannot -> yet target it: Stage 3 composes every ADO Git REST call from the pipeline's -> own organization/project. See the limitation note under -> [`create-pull-request`](safe-outputs.md#create-pull-request). +> **Cross-organization `type: git` repositories.** Use object form with both +> `organization:` and `endpoint:`. Checkout authorization and Stage 3 writes +> are separate: `endpoint` authenticates the repository resource, while +> expanded `permissions.write` with `connection-type: azureDevOps` and an exact +> organization/project/repository `allow` scope authorizes `create-pull-request`, +> `create-branch`, and `create-git-tag`. Incomplete entries compile with a +> warning and are rejected if targeted, including under `--dry-run`. ### Tuning checkout fetch behavior (`fetch-depth` / `fetch-tags`) diff --git a/docs/network.md b/docs/network.md index 576b991b3..8f8959056 100644 --- a/docs/network.md +++ b/docs/network.md @@ -239,8 +239,8 @@ See [`imports:`](imports.md) for the ADO-first compile-time `repository` and ## Permissions (ADO Access Tokens) -The ARM service-connection scope does not determine what its identity may do in -Azure DevOps. `permissions.read` and `permissions.write` describe intended +The service-connection resource scope does not determine what its identity may +do in Azure DevOps. `permissions.read` and `permissions.write` describe intended pipeline roles and token placement; operators must separately grant each underlying identity the minimum Azure DevOps permissions. The executor (Stage 3) always has a write-capable token; what changes is its *source* and @@ -249,7 +249,7 @@ underlying identity the minimum Azure DevOps permissions. The executor | Source | When | Identity | | ----------------------------------- | --------------------------------------------- | ----------------------------------------------- | | `$(System.AccessToken)` *(default)* | No `permissions.write` configured | `Project Collection Build Service (org)` | -| `$(SC_WRITE_TOKEN)` *(opt-in)* | `permissions.write: ` | The federated identity behind the ARM SC | +| `$(SC_WRITE_TOKEN)` *(opt-in)* | Any `permissions.write` service connection | The configured connection's Entra identity | The agent (Stage 1) never receives the executor's token. Stage separation — not token type — is the trust boundary. @@ -261,7 +261,7 @@ not token type — is the trust boundary. don't match (`PATCH _apis/build/builds/{id}`) and fetches PR metadata for Tier 2 filters (labels, draft status, changed files). Runs before the agent, outside the AWF sandbox. -2. **Stage 3 executor** — when no ARM write SC is configured (the default), +2. **Stage 3 executor** — when no write service connection is configured (the default), the executor's `SYSTEM_ACCESSTOKEN` env var is sourced from `$(System.AccessToken)`. @@ -287,9 +287,11 @@ agents. Set `permissions.write` only when you need: 1. **Cross-org or cross-project writes** — `System.AccessToken` is scoped to the host project. Targeting work items or repos in a different ADO - project / organization requires an ARM SC with broader scope. + project may use an Azure Resource Manager connection. Cross-organization + repository writes require an Azure DevOps connection and explicit allow + scope. 2. **Named-identity attribution** — `System.AccessToken` writes are - attributed to the `Project Collection Build Service` identity. An ARM SC + attributed to the `Project Collection Build Service` identity. A service connection attributes writes to its underlying federated identity (e.g. `safe-output-bot@contoso.com`), useful when audit logs or work-item notifications need a specific actor. @@ -343,6 +345,35 @@ agents. Set `permissions.write` only when you need: used **only** by the executor in Stage 3 (`SafeOutputs` job). Overrides the default `$(System.AccessToken)` for write operations. Never exposed to the agent. + + Scalar form preserves the Azure Resource Manager connection contract: + + ```yaml + permissions: + write: my-arm-connection + ``` + + Expanded form selects AzureCLI@3's connection type and adds exact + cross-organization repository scopes: + + ```yaml + permissions: + write: + service-connection: ado-repository-writer + connection-type: azureDevOps + allow: + - organization: other-org + projects: + - project: Other Project + repositories: [target-repo] + ``` + + `connection-type` is either `azureRM` or `azureDevOps`. Only + `azureDevOps` permits cross-organization repository writes. The Azure DevOps + service connection must be backed by Entra workload identity federation; its + identity must belong to the same tenant, be added to every target + organization, and receive repository ACLs there. `allow` is additive to the + implicit current organization and is deny-by-default for cross-org targets. - **Both omitted**: The agent has no ADO API access. The executor still has a write-capable token via `$(System.AccessToken)`, scoped by the pipeline's job-authorization settings. @@ -354,11 +385,22 @@ agents. Set `permissions.write` only when you need: permissions: read: my-read-sc -# Cross-org / named-identity attribution — executor writes via ARM SC. +# Named-identity attribution through an Azure Resource Manager connection. permissions: read: my-read-sc write: my-write-sc +# Cross-org repository writes through an Azure DevOps WIF connection. +permissions: + write: + service-connection: ado-repository-writer + connection-type: azureDevOps + allow: + - organization: other-org + projects: + - project: Other Project + repositories: [target-repo] + # Agent has no ADO read access; executor still writes via $(System.AccessToken). # (Empty front matter — no `permissions:` key at all.) ``` diff --git a/docs/safe-output-permissions.md b/docs/safe-output-permissions.md index f6ef4db90..95e1636f8 100644 --- a/docs/safe-output-permissions.md +++ b/docs/safe-output-permissions.md @@ -74,7 +74,7 @@ The toggle lives in three places (most-specific wins): > which carries `PullRequestContribute` by default. If `permissions.write:` is set in the agent's front matter, Stage 3 -uses the **ARM service connection's identity** instead, and none of +uses the configured **service connection's identity** instead, and none of the above applies — see [Option 1](#option-1-wire-a-write-service-connection-recommended). --- @@ -230,9 +230,8 @@ complementary. ### Option 1: Wire a write service connection (recommended) -Add an ARM service connection whose backing identity has the -permission you need on the target repository, and reference it from -the agent front matter: +For same-organization or same-organization cross-project writes, the scalar +form keeps the Azure Resource Manager connection behavior: ```yaml permissions: @@ -240,14 +239,30 @@ permissions: write: ado-aw-write # used by Stage 3 ``` +For cross-organization repository writes, use an Azure DevOps service +connection backed by Entra workload identity federation: + +```yaml +permissions: + write: + service-connection: ado-aw-repository-writer + connection-type: azureDevOps + allow: + - organization: other-org + projects: + - project: Other Project + repositories: [target-repo] +``` + Stage 3 will mint its token via that connection instead of using `$(System.AccessToken)`, so the build-service ACEs become irrelevant. This is the most explicit option: the identity used for writes is named in the front matter, audit logs attribute every action to that named principal, and the least-privilege grant lives entirely on the -service connection's identity. It also works unchanged for -cross-organization writes. +service connection's identity. The same identity can write in multiple +same-tenant organizations after it is added to each organization and granted +the repository permissions listed above. See [`docs/network.md`](network.md) (Permissions section) and the "Service Connections" page on the documentation site for the full @@ -306,7 +321,7 @@ Option 2 unless you have a specific reason to broaden the grant. | HTTP status | Body fragment | Most likely cause | |---|---|---| -| 401 Unauthorized | `TF400813: The user '...' is not authorized to access this resource` | Token is malformed or missing — usually a misconfigured service-connection step; check that the AzureCLI@2 mint succeeded. | +| 401 Unauthorized | `TF400813: The user '...' is not authorized to access this resource` | Token is malformed or missing — usually a misconfigured service-connection step; check that the AzureCLI@3 mint succeeded. | | 403 Forbidden | `TF401027: You need the Git 'PullRequestContribute' permission` | This page — Stage 3 identity lacks PR-contribute on the target repo. | | 403 Forbidden | `TF401027: You need the Git 'GenericContribute' permission` | Same diagnosis; need `Contribute` on the repo (typically because of `create-pull-request` or `create-branch`). | | 403 Forbidden | `VS800075: The project ... does not exist, or you do not have permission to access it.` | Cross-project request blocked because "Limit job authorization scope to current project" is ON. Use Option 1 with a write service connection that has cross-project rights, or move the resource into the calling project. | diff --git a/docs/safe-outputs.md b/docs/safe-outputs.md index a5ab9f356..c93a07934 100644 --- a/docs/safe-outputs.md +++ b/docs/safe-outputs.md @@ -292,7 +292,8 @@ Stage 3 `SafeOutputs` job and authenticate to Azure DevOps using `SYSTEM_ACCESSTOKEN`. By default this is `$(System.AccessToken)` — the pipeline's built-in OAuth token running as the *Project Collection Build Service* identity. Set `permissions.write` to override this with an -ARM-minted token, e.g. for cross-org writes or named-identity attribution. +AzureCLI@3-minted token, e.g. for cross-org writes or named-identity +attribution. See [`docs/network.md`](network.md) and [`docs/ir.md`](ir.md) for the typed SafeOutputs job wiring. @@ -1014,25 +1015,31 @@ Creates a pull request with code changes made by the agent. When invoked: 2. Saves the patch to the safe outputs directory 3. Creates a JSON record with PR metadata (title, description, source branch, repository) -During Stage 3 execution, the repository is validated against the allowed list (from `checkout:` + "self"), then the patch is applied and a PR is created in Azure DevOps. - -> **Cross-organization repositories are not yet supported.** Every ADO Git -> REST call the executor makes is composed from the pipeline's own -> organization/project. A `repos:` alias checked out from a **different** -> Azure DevOps organization (a `type: git` entry with an `endpoint:` service -> connection — see [`docs/front-matter.md`](front-matter.md#repositories-repos)) -> cannot be targeted: the compiler warns when `create-pull-request` and such an -> alias are both configured, and Stage 3 rejects the alias with a clear error -> (including under `--dry-run`) instead of silently composing a request against -> the wrong organization. +During Stage 3 execution, the repository is validated against the allowed list +(from `checkout:` + "self"), resolved to an exact +organization/project/repository target, then the patch is applied and a PR is +created in Azure DevOps. + +> **Cross-organization repositories.** `create-pull-request`, +> `create-branch`, and `create-git-tag` can target a checked-out Azure Repos +> repository in another organization when its `repos:` object declares +> `organization:` plus `endpoint:`, and expanded `permissions.write` uses +> `connection-type: azureDevOps` with an exact organization/project/repository +> allow scope. The Stage 3 token is never exposed to the Agent. Missing routing, +> connection type, or scope produces a compile warning and a target-time +> rejection, including under `--dry-run`; it never falls back to the pipeline +> organization. **Shallow-clone agent pools (automatic):** The diff base is computed at agent -time from the checked-out repository. For same-organization Azure Repos, +time from the checked-out repository. For Azure Repos, `prepare-pr-base.js` asks the ADO Diffs API for the exact `commonCommit`, `aheadCount`, and `behindCount`, then fetches only the source and target ranges -needed to make that base locally reachable. It verifies the server result with +needed to make that base locally reachable. Cross-org preparation runs in a +trusted AzureCLI@3 task and passes its short-lived bearer only to the bundle +child process; the credential is not persisted or exposed to the Agent. It +verifies the server result with `git merge-base --all` before the containerized SafeOutputs MCP server can -generate a patch. Non-Azure/cross-organization/unavailable-REST cases use bounded +generate a patch. Non-Azure/unavailable-REST cases use bounded dual-ref depths 200/500/2000 and fail clearly rather than silently fetching full history. @@ -1349,6 +1356,10 @@ safe-outputs: max: 1 # Maximum per run (default: 1) ``` +Cross-organization tag creation uses the same `repos.organization` and +expanded `permissions.write` contract described under +[`create-pull-request`](#create-pull-request). + ### add-build-tag Adds a tag to an Azure DevOps build. @@ -1385,6 +1396,10 @@ safe-outputs: max: 1 # Maximum per run (default: 1) ``` +Cross-organization branch creation uses the same `repos.organization` and +expanded `permissions.write` contract described under +[`create-pull-request`](#create-pull-request). + ### upload-workitem-attachment Uploads a workspace file as an attachment to an Azure DevOps work item. @@ -1539,7 +1554,7 @@ multiple uploads. **Notes:** - Single-file only; directory uploads are not supported. - When `build_id` is omitted and `allowed-build-ids` is configured, the allow-list check is skipped — the current build is implicitly trusted. -- Requires `BUILD_CONTAINERID`, `BUILD_BUILDID`, and `SYSTEM_TEAMPROJECTID` (all set automatically inside an Azure DevOps pipeline job) and `vso.build_execute` scope on the executor's token (granted to `$(System.AccessToken)` by default, and to the ARM-minted token when `permissions.write` is set). +- Requires `BUILD_CONTAINERID`, `BUILD_BUILDID`, and `SYSTEM_TEAMPROJECTID` (all set automatically inside an Azure DevOps pipeline job) and `vso.build_execute` scope on the executor's token (granted to `$(System.AccessToken)` by default, and to the configured service-connection token when `permissions.write` is set). ### cache-memory (moved to `tools:`) Memory is now configured as a first-class tool under `tools: cache-memory:` instead of `safe-outputs: memory:`. See the [Cache Memory section](./tools.md#cache-memory-cache-memory) in `docs/tools.md` for details. diff --git a/scripts/ado-script/src/executor-e2e/__tests__/ado-rest.test.ts b/scripts/ado-script/src/executor-e2e/__tests__/ado-rest.test.ts index 9c6a72e21..5c3cba3e4 100644 --- a/scripts/ado-script/src/executor-e2e/__tests__/ado-rest.test.ts +++ b/scripts/ado-script/src/executor-e2e/__tests__/ado-rest.test.ts @@ -19,6 +19,28 @@ describe("AdoRest.workItemTypeExists", () => { vi.unstubAllGlobals(); }); + describe("AdoRest authentication", () => { + afterEach(() => { + vi.unstubAllGlobals(); + }); + + it("uses Bearer auth when requested", async () => { + const fetchMock = stubFetch( + () => + new Response(JSON.stringify({ id: "repo" }), { + status: 200, + headers: { "content-type": "application/json" }, + }), + ); + + await new AdoRest({ ...options, authKind: "bearer" }).getRepository("repo"); + + expect(fetchMock.mock.calls[0]?.[1]).toMatchObject({ + headers: expect.objectContaining({ Authorization: "Bearer token" }), + }); + }); + }); + it("resolves true and encodes the project and type segments", async () => { const fetchMock = stubFetch( () => diff --git a/scripts/ado-script/src/executor-e2e/__tests__/cross-org-scenarios.test.ts b/scripts/ado-script/src/executor-e2e/__tests__/cross-org-scenarios.test.ts new file mode 100644 index 000000000..f117d41c7 --- /dev/null +++ b/scripts/ado-script/src/executor-e2e/__tests__/cross-org-scenarios.test.ts @@ -0,0 +1,227 @@ +import { afterEach, describe, expect, it, vi } from "vitest"; + +import { AdoRest } from "../ado-rest.js"; +import type { ScenarioContext } from "../scenario.js"; +import { SkipError } from "../scenario.js"; +import { + crossOrgSource, + createCrossOrgBranch, + createCrossOrgGitTag, + resolveCrossOrgEnv, + type CrossOrgEnv, +} from "../scenarios/cross-org.js"; + +function fakeCtx(): ScenarioContext { + return { + orgUrl: "https://dev.azure.com/current/", + project: "Current", + adoRepo: "repo", + buildId: "77", + token: "token", + adoAwBin: "ado-aw", + workDir: "/tmp", + rest: {} as ScenarioContext["rest"], + log: () => {}, + prefix: (tool) => `ado-aw-det-77-${tool}`, + }; +} + +afterEach(() => { + vi.unstubAllEnvs(); +}); + +describe("cross-org executor environment", () => { + it("skips when the pre-provisioned infrastructure is absent", () => { + expect(() => resolveCrossOrgEnv(fakeCtx())).toThrow(SkipError); + }); + + it("builds one exact repository and write scope from environment", () => { + vi.stubEnv("EXECUTOR_E2E_CROSS_ORG_ORGANIZATION", "other-org"); + vi.stubEnv("EXECUTOR_E2E_CROSS_ORG_PROJECT", "Other Project"); + vi.stubEnv("EXECUTOR_E2E_CROSS_ORG_REPOSITORY", "target-repo"); + vi.stubEnv("EXECUTOR_E2E_CROSS_ORG_ENDPOINT", "ado-write"); + vi.stubEnv("EXECUTOR_E2E_CROSS_ORG_TOKEN", "entra-token"); + + const env = resolveCrossOrgEnv(fakeCtx()); + const source = crossOrgSource(env); + + expect(env.orgUrl).toBe("https://dev.azure.com/other-org/"); + expect(source.repositories).toEqual([{ + name: "Other Project/target-repo", + alias: "cross-org-target", + organization: "other-org", + endpoint: "ado-write", + }]); + expect(source.writePermissions).toEqual({ + serviceConnection: "ado-write", + connectionType: "azureDevOps", + allow: [{ + organization: "other-org", + projects: [{ + project: "Other Project", + repositories: ["target-repo"], + }], + }], + }); + }); + + describe("cross-org branch and tag scenarios", () => { + function scenarioEnv( + getRefObjectId: ReturnType, + deleteRef = vi.fn(async () => {}), + ): CrossOrgEnv { + return { + organization: "other-org", + orgUrl: "https://dev.azure.com/other-org/", + project: "Other Project", + repository: "target-repo", + alias: "cross-org-target", + endpoint: "ado-write", + token: "entra-token", + rest: { + getRefObjectId, + deleteRef, + } as unknown as AdoRest, + }; + } + + it("builds and verifies a cross-org branch proposal", async () => { + const getRefObjectId = vi.fn(async () => "a".repeat(40)); + const deleteRef = vi.fn(async () => {}); + const state = { + env: scenarioEnv(getRefObjectId, deleteRef), + branch: "ado-aw-det-77-cross-branch", + base: "main", + }; + + const source = createCrossOrgBranch.source; + if (!source) throw new Error("cross-org branch scenario source is required"); + await expect(source(fakeCtx(), state)).resolves.toEqual(crossOrgSource(state.env)); + expect(createCrossOrgBranch.config(fakeCtx(), state)).toEqual({ + "allowed-repositories": ["cross-org-target"], + max: 1, + }); + await expect( + createCrossOrgBranch.ndjson(fakeCtx(), state), + ).resolves.toEqual({ + branch_name: state.branch, + source_branch: "main", + repository: "cross-org-target", + }); + await expect( + createCrossOrgBranch.assert( + fakeCtx(), + state, + { + name: "create_branch", + status: "succeeded", + }, + [], + ), + ).resolves.toBeUndefined(); + expect(getRefObjectId).toHaveBeenCalledWith( + "target-repo", + `heads/${state.branch}`, + ); + await createCrossOrgBranch.cleanup(fakeCtx(), state); + expect(deleteRef).toHaveBeenCalledWith( + "target-repo", + `refs/heads/${state.branch}`, + ); + }); + + it("builds, verifies, and cleans up a cross-org tag proposal", async () => { + const getRefObjectId = vi.fn(async () => "b".repeat(40)); + const deleteRef = vi.fn(async () => {}); + const state = { + env: scenarioEnv(getRefObjectId, deleteRef), + tag: "ado-aw-det-77-cross-tag", + }; + + const source = createCrossOrgGitTag.source; + if (!source) throw new Error("cross-org tag scenario source is required"); + await expect(source(fakeCtx(), state)).resolves.toEqual(crossOrgSource(state.env)); + expect(createCrossOrgGitTag.config(fakeCtx(), state)).toEqual({ + "allowed-repositories": ["cross-org-target"], + max: 1, + }); + await expect(createCrossOrgGitTag.ndjson(fakeCtx(), state)).resolves.toEqual({ + tag_name: state.tag, + message: expect.stringContaining("create-git-tag-cross-org"), + repository: "cross-org-target", + }); + await expect( + createCrossOrgGitTag.assert( + fakeCtx(), + state, + { + name: "create_git_tag", + status: "succeeded", + }, + [], + ), + ).resolves.toBeUndefined(); + expect(getRefObjectId).toHaveBeenCalledWith( + "target-repo", + `tags/${state.tag}`, + ); + await expect( + createCrossOrgGitTag.env!(fakeCtx(), state), + ).resolves.toEqual({ + SYSTEM_ACCESSTOKEN: "entra-token", + }); + await createCrossOrgGitTag.cleanup(fakeCtx(), state); + expect(deleteRef).toHaveBeenCalledWith( + "target-repo", + `refs/tags/${state.tag}`, + ); + }); + + it("fails verification when a cross-org tag is absent", async () => { + const state = { + env: scenarioEnv(vi.fn(async () => undefined)), + tag: "ado-aw-det-77-cross-tag", + }; + + await expect( + createCrossOrgGitTag.assert( + fakeCtx(), + state, + { + name: "create_git_tag", + status: "succeeded", + }, + [], + ), + ).rejects.toThrow(/was not created/); + }); + + it("fails verification when a cross-org branch is absent", async () => { + const state = { + env: scenarioEnv(vi.fn(async () => undefined)), + branch: "ado-aw-det-77-cross-branch", + base: "main", + }; + + await expect( + createCrossOrgBranch.assert( + fakeCtx(), + state, + { + name: "create_branch", + status: "succeeded", + }, + [], + ), + ).rejects.toThrow(/was not created/); + }); + }); + + it("treats unexpanded pipeline macros as absent", () => { + vi.stubEnv( + "EXECUTOR_E2E_CROSS_ORG_ORGANIZATION", + "$(EXECUTOR_E2E_CROSS_ORG_ORGANIZATION)", + ); + expect(() => resolveCrossOrgEnv(fakeCtx())).toThrow(SkipError); + }); +}); diff --git a/scripts/ado-script/src/executor-e2e/__tests__/execute-cli.test.ts b/scripts/ado-script/src/executor-e2e/__tests__/execute-cli.test.ts index c8170d697..2b76d964f 100644 --- a/scripts/ado-script/src/executor-e2e/__tests__/execute-cli.test.ts +++ b/scripts/ado-script/src/executor-e2e/__tests__/execute-cli.test.ts @@ -43,6 +43,38 @@ describe("renderSourceMarkdown", () => { expect(md).toContain('"set-github-issue-type": {"target-repo":"o/r"}'); expect(md.match(/^---$/gm)?.length).toBe(2); }); + + it("emits expanded write permissions and cross-org repository metadata", () => { + const md = renderSourceMarkdown({ + tool: "create-branch", + safeOutputs: { "create-branch": { "allowed-repositories": ["target"] } }, + source: { + repositories: [{ + name: "Other Project/target-repo", + alias: "target", + organization: "other-org", + endpoint: "ado-write", + }], + writePermissions: { + serviceConnection: "ado-write", + connectionType: "azureDevOps", + allow: [{ + organization: "other-org", + projects: [{ + project: "Other Project", + repositories: ["target-repo"], + }], + }], + }, + }, + }); + + expect(md).toContain("permissions:"); + expect(md).toContain('"connection-type":"azureDevOps"'); + expect(md).toContain( + ' - {"name":"Other Project/target-repo","alias":"target","organization":"other-org","endpoint":"ado-write"}', + ); + }); }); describe("renderNdjsonLine", () => { diff --git a/scripts/ado-script/src/executor-e2e/__tests__/index.test.ts b/scripts/ado-script/src/executor-e2e/__tests__/index.test.ts index 93ae55a10..2c266930a 100644 --- a/scripts/ado-script/src/executor-e2e/__tests__/index.test.ts +++ b/scripts/ado-script/src/executor-e2e/__tests__/index.test.ts @@ -25,6 +25,9 @@ describe("scenario registry", () => { expect(new Set(ids).size).toBe(ids.length); expect(ids).toContain("create-pull-request"); expect(ids).toContain("create-pull-request-self-multi-checkout"); + expect(ids).toContain("create-pull-request-cross-org"); + expect(ids).toContain("create-branch-cross-org"); + expect(ids).toContain("create-git-tag-cross-org"); }); it("registers the GitHub issue scenarios with unique ids", () => { diff --git a/scripts/ado-script/src/executor-e2e/ado-rest.ts b/scripts/ado-script/src/executor-e2e/ado-rest.ts index 65dd24fab..c7da667ea 100644 --- a/scripts/ado-script/src/executor-e2e/ado-rest.ts +++ b/scripts/ado-script/src/executor-e2e/ado-rest.ts @@ -2,9 +2,8 @@ * Minimal, self-contained Azure DevOps REST client for the deterministic * executor E2E harness. * - * Uses the global `fetch` (Node 20+) with Basic auth (empty user + token), - * matching how the `ado-aw` Rust executor authenticates - * (`reqwest ... .basic_auth("", Some(token))`). Endpoints and api-versions are + * Uses the global `fetch` (Node 20+) with Basic auth by default and Bearer auth + * for Azure DevOps service-connection tokens. Endpoints and api-versions are * chosen to line up with the executors under test so setup/assert/cleanup hit * the same surfaces the executor writes to. * @@ -15,6 +14,7 @@ export interface AdoRestOptions { orgUrl: string; project: string; token: string; + authKind?: "basic" | "bearer"; log?: (msg: string) => void; } @@ -42,7 +42,10 @@ export class AdoRest { constructor(opts: AdoRestOptions) { this.base = opts.orgUrl.replace(/\/+$/, ""); this.project = opts.project; - this.authHeader = "Basic " + Buffer.from(":" + opts.token).toString("base64"); + this.authHeader = + opts.authKind === "bearer" + ? `Bearer ${opts.token}` + : "Basic " + Buffer.from(":" + opts.token).toString("base64"); this.log = opts.log ?? (() => {}); this.timeoutMs = Number(process.env.EXECUTOR_E2E_REST_TIMEOUT_MS) || 30_000; } diff --git a/scripts/ado-script/src/executor-e2e/execute-cli.ts b/scripts/ado-script/src/executor-e2e/execute-cli.ts index e4db83a54..7e1fa0210 100644 --- a/scripts/ado-script/src/executor-e2e/execute-cli.ts +++ b/scripts/ado-script/src/executor-e2e/execute-cli.ts @@ -16,7 +16,7 @@ import { mkdir, readFile, writeFile } from "node:fs/promises"; import { existsSync } from "node:fs"; import { join, resolve, sep } from "node:path"; -import type { ExecutedRecord, PriorEntry } from "./scenario.js"; +import type { ExecutedRecord, PriorEntry, ScenarioSource } from "./scenario.js"; const SAFE_OUTPUT_FILENAME = "safe_outputs.ndjson"; const EXECUTED_FILENAME = "safe-outputs-executed.ndjson"; @@ -27,6 +27,7 @@ export interface RenderSourceOptions { safeOutputs: Record>; /** ADO repo name for repo-targeting tools (emits a `repos:` block). */ adoRepo?: string; + source?: ScenarioSource; } /** @@ -45,7 +46,21 @@ export function renderSourceMarkdown(opts: RenderSourceOptions): string { "engine:", " id: copilot", ]; - if (opts.adoRepo) { + if (opts.source?.writePermissions) { + const write = opts.source.writePermissions; + lines.push("permissions:"); + lines.push(` write: ${JSON.stringify({ + "service-connection": write.serviceConnection, + "connection-type": write.connectionType, + allow: write.allow, + })}`); + } + if (opts.source?.repositories) { + lines.push("repos:"); + for (const repository of opts.source.repositories) { + lines.push(` - ${JSON.stringify(repository)}`); + } + } else if (opts.adoRepo) { lines.push("repos:"); // JSON-stringify both the alias and the repo name (valid YAML) to stay // consistent with the rest of the rendered front-matter and guard against @@ -87,6 +102,7 @@ export interface RunExecuteOptions { */ priorEntries?: PriorEntry[]; adoRepo?: string; + source?: ScenarioSource; orgUrl: string; project: string; token: string; @@ -137,7 +153,12 @@ export async function runExecute(opts: RunExecuteOptions): Promise( // Guard the auxiliary scenario methods too: a harness-level bug in any of // these must record a failed result and let the rest of the suite run, // not propagate out of runScenario and abort runAll early. - let config, entry, files, extraEnv, priorEntries; + let config, entry, files, extraEnv, priorEntries, source; try { config = scenario.config(ctx, state); entry = await scenario.ndjson(ctx, state); priorEntries = scenario.priorEntries ? await scenario.priorEntries(ctx, state) : undefined; files = scenario.files ? await scenario.files(ctx, state) : undefined; extraEnv = scenario.env ? await scenario.env(ctx, state) : undefined; + source = scenario.source ? await scenario.source(ctx, state) : undefined; } catch (err) { return finish({ ok: false, phase: "execute", message: errMessage(err) }); } @@ -88,6 +89,7 @@ export async function runScenario( entry, priorEntries, adoRepo: scenario.targetsAdoRepo ? ctx.adoRepo : undefined, + source, orgUrl: ctx.orgUrl, project: ctx.project, token: ctx.token, diff --git a/scripts/ado-script/src/executor-e2e/scenario.ts b/scripts/ado-script/src/executor-e2e/scenario.ts index 578f4b18f..be8b3315b 100644 --- a/scripts/ado-script/src/executor-e2e/scenario.ts +++ b/scripts/ado-script/src/executor-e2e/scenario.ts @@ -49,6 +49,28 @@ export interface PriorEntry { readonly config: Record; } +export interface ScenarioSourceRepository { + readonly name: string; + readonly alias: string; + readonly organization?: string; + readonly endpoint?: string; +} + +export interface ScenarioSource { + readonly repositories?: ScenarioSourceRepository[]; + readonly writePermissions?: { + readonly serviceConnection: string; + readonly connectionType: "azureDevOps"; + readonly allow: Array<{ + readonly organization: string; + readonly projects: Array<{ + readonly project: string; + readonly repositories: string[]; + }>; + }>; + }; +} + /** Shared, read-only context handed to every scenario phase. */export interface ScenarioContext { /** ADO collection URI, e.g. https://dev.azure.com/msazuresphere/ */ readonly orgUrl: string; @@ -93,6 +115,8 @@ export interface Scenario { * per-tool `allowed-repositories` config. */ readonly targetsAdoRepo?: boolean; + /** Optional trusted front-matter additions for cross-org executor scenarios. */ + source?(ctx: ScenarioContext, state: State): Promise; /** Per-tool `safe-outputs: :` front-matter config fragment. */ config(ctx: ScenarioContext, state: State): Record; /** diff --git a/scripts/ado-script/src/executor-e2e/scenarios/create-pull-request.ts b/scripts/ado-script/src/executor-e2e/scenarios/create-pull-request.ts index dc6346798..16fafb4d9 100644 --- a/scripts/ado-script/src/executor-e2e/scenarios/create-pull-request.ts +++ b/scripts/ado-script/src/executor-e2e/scenarios/create-pull-request.ts @@ -29,6 +29,11 @@ import { join } from "node:path"; import type { Scenario, ScenarioContext } from "../scenario.js"; import { partialOutput } from "../execute-cli.js"; import { detBody, numResult, Teardown } from "./common.js"; +import { + crossOrgSource, + resolveCrossOrgEnv, + type CrossOrgEnv, +} from "./cross-org.js"; interface CreatePrState { repo: string; @@ -42,13 +47,17 @@ interface CreatePrState { sourcesDir: string; /** Actual git checkout beneath sourcesDir. */ checkoutDir: string; + rest: ScenarioContext["rest"]; + executorToken: string; + repositorySelector: string; + crossOrg?: CrossOrgEnv; /** PR id, populated in assert() so cleanup can abandon it. */ prId?: number; } interface CreatePrScenarioOptions { readonly id: string; - readonly repositorySelector: "named" | "self"; + readonly repositorySelector: "named" | "self" | "cross-org"; readonly patchRelPath: string; readonly changedFileSuffix?: string; } @@ -124,13 +133,28 @@ async function setupCreatePullRequest( ctx: ScenarioContext, options: CreatePrScenarioOptions, ): Promise { - const repo = ctx.adoRepo; - const authHeader = "Basic " + Buffer.from(":" + ctx.token).toString("base64"); + const crossOrg = + options.repositorySelector === "cross-org" + ? resolveCrossOrgEnv(ctx) + : undefined; + const repo = crossOrg?.repository ?? ctx.adoRepo; + const orgUrl = crossOrg?.orgUrl ?? ctx.orgUrl; + const project = crossOrg?.project ?? ctx.project; + const token = crossOrg?.token ?? ctx.token; + const rest = crossOrg?.rest ?? ctx.rest; + const authHeader = crossOrg + ? `Bearer ${token}` + : "Basic " + Buffer.from(":" + token).toString("base64"); const sourcesDir = join(ctx.workDir, options.id, "src-checkout"); await mkdir(sourcesDir, { recursive: true }); - const checkoutDir = join(sourcesDir, repo); + const repositorySelector = crossOrg?.alias ?? + (options.repositorySelector === "self" ? "self" : repo); + const checkoutDir = join( + sourcesDir, + crossOrg?.alias ?? repo, + ); - const cloneUrl = `${ctx.orgUrl.replace(/\/+$/, "")}/${encodeURIComponent(ctx.project)}/_git/${encodeURIComponent(repo)}`; + const cloneUrl = `${orgUrl.replace(/\/+$/, "")}/${encodeURIComponent(project)}/_git/${encodeURIComponent(repo)}`; ctx.log(`[${options.id}] cloning ${repo}`); await git(ctx, ["clone", cloneUrl, checkoutDir], sourcesDir, authHeader, options.id); @@ -188,6 +212,10 @@ async function setupCreatePullRequest( patchContent, sourcesDir, checkoutDir, + rest, + executorToken: token, + repositorySelector, + crossOrg, }; } @@ -197,7 +225,7 @@ function createPullRequestScenario( return { id: options.id, tool: "create-pull-request", - targetsAdoRepo: true, + targetsAdoRepo: options.repositorySelector === "named", config: (_ctx, state) => ({ // Target the repo's actual default branch (state.targetBranch), which is // also where base_commit was taken from, rather than hardcoding "main". @@ -208,11 +236,16 @@ function createPullRequestScenario( "include-stats": false, }), setup: (ctx) => setupCreatePullRequest(ctx, options), + source: async (_ctx, state) => + state.crossOrg ? crossOrgSource(state.crossOrg) : {}, files: async (_ctx, state) => ({ [state.patchRelPath]: state.patchContent }), env: async (_ctx, state) => { const env: Record = { BUILD_SOURCESDIRECTORY: state.sourcesDir, }; + if (state.crossOrg) { + env.SYSTEM_ACCESSTOKEN = state.executorToken; + } if (options.repositorySelector === "self") { Object.assign(env, { ADO_AW_SELF_REPOSITORY_DIRECTORY: state.checkoutDir, @@ -235,7 +268,7 @@ function createPullRequestScenario( description: detBody(ctx, options.id), source_branch: state.sourceBranch, patch_file: state.patchRelPath, - repository: options.repositorySelector === "self" ? "self" : state.repo, + repository: state.repositorySelector, agent_labels: [], base_commit: state.baseCommit, patch_sha256: state.patchSha256, @@ -245,9 +278,9 @@ function createPullRequestScenario( // Record the PR id up front so cleanup abandons it even if a later // assertion (or the getPullRequest call itself) throws. state.prId = prId; - const pr = await ctx.rest.getPullRequest(state.repo, prId); + const pr = await state.rest.getPullRequest(state.repo, prId); if (pr.status === "abandoned") throw new Error(`PR #${prId} is abandoned`); - const sha = await ctx.rest.getRefObjectId(state.repo, `heads/${state.sourceBranch}`); + const sha = await state.rest.getRefObjectId(state.repo, `heads/${state.sourceBranch}`); if (!sha) throw new Error(`source branch '${state.sourceBranch}' was not pushed`); }, cleanup: async (ctx, state) => { @@ -257,11 +290,11 @@ function createPullRequestScenario( const teardown = new Teardown(); if (state.prId !== undefined) { const prId = state.prId; - teardown.add("abandon PR", () => ctx.rest.abandonPullRequest(state.repo, prId)); + teardown.add("abandon PR", () => state.rest.abandonPullRequest(state.repo, prId)); } await teardown .add("delete source branch", () => - ctx.rest.deleteRef(state.repo, `refs/heads/${state.sourceBranch}`), + state.rest.deleteRef(state.repo, `refs/heads/${state.sourceBranch}`), ) // Remove the cloned checkout so repeated local runs don't accumulate it. .add("remove local checkout", () => @@ -285,7 +318,15 @@ export const createPullRequestSelfMultiCheckout = createPullRequestScenario({ changedFileSuffix: "-self-multi-checkout", }); +export const createPullRequestCrossOrg = createPullRequestScenario({ + id: "create-pull-request-cross-org", + repositorySelector: "cross-org", + patchRelPath: "create-pr-cross-org.patch", + changedFileSuffix: "-cross-org", +}); + export const createPullRequestScenarios: Scenario[] = [ createPullRequest, createPullRequestSelfMultiCheckout, + createPullRequestCrossOrg, ]; diff --git a/scripts/ado-script/src/executor-e2e/scenarios/cross-org.ts b/scripts/ado-script/src/executor-e2e/scenarios/cross-org.ts new file mode 100644 index 000000000..2f7b7c4f2 --- /dev/null +++ b/scripts/ado-script/src/executor-e2e/scenarios/cross-org.ts @@ -0,0 +1,174 @@ +/** + * Optional cross-organization Azure Repos write scenarios. + * + * These require pre-provisioned same-tenant Azure DevOps WIF infrastructure; + * they skip when any required environment value is absent. + */ +import { AdoRest } from "../ado-rest.js"; +import type { Scenario, ScenarioContext, ScenarioSource } from "../scenario.js"; +import { SkipError } from "../scenario.js"; +import { detBody } from "./common.js"; + +export interface CrossOrgEnv { + organization: string; + orgUrl: string; + project: string; + repository: string; + alias: string; + endpoint: string; + token: string; + rest: AdoRest; +} + +function cleanVar(value: string | undefined): string | undefined { + const trimmed = value?.trim(); + if (!trimmed || /^\$\([^)]+\)$/.test(trimmed)) return undefined; + return trimmed; +} + +export function resolveCrossOrgEnv(ctx: ScenarioContext): CrossOrgEnv { + const organization = cleanVar(process.env.EXECUTOR_E2E_CROSS_ORG_ORGANIZATION); + const project = cleanVar(process.env.EXECUTOR_E2E_CROSS_ORG_PROJECT); + const repository = cleanVar(process.env.EXECUTOR_E2E_CROSS_ORG_REPOSITORY); + const endpoint = cleanVar(process.env.EXECUTOR_E2E_CROSS_ORG_ENDPOINT); + const token = cleanVar(process.env.EXECUTOR_E2E_CROSS_ORG_TOKEN); + const missing = [ + ["EXECUTOR_E2E_CROSS_ORG_ORGANIZATION", organization], + ["EXECUTOR_E2E_CROSS_ORG_PROJECT", project], + ["EXECUTOR_E2E_CROSS_ORG_REPOSITORY", repository], + ["EXECUTOR_E2E_CROSS_ORG_ENDPOINT", endpoint], + ["EXECUTOR_E2E_CROSS_ORG_TOKEN", token], + ].filter(([, value]) => !value).map(([name]) => name); + if (missing.length > 0) { + throw new SkipError( + `cross-org repository scenarios require ${missing.join(", ")}`, + ); + } + if (!organization || !project || !repository || !endpoint || !token) { + throw new SkipError( + "cross-org repository scenario configuration is incomplete", + ); + } + const orgUrl = `https://dev.azure.com/${organization}/`; + return { + organization, + orgUrl, + project, + repository, + alias: "cross-org-target", + endpoint, + token, + rest: new AdoRest({ + orgUrl, + project, + token, + authKind: "bearer", + log: ctx.log, + }), + }; +} + +export function crossOrgSource(env: CrossOrgEnv): ScenarioSource { + return { + repositories: [{ + name: `${env.project}/${env.repository}`, + alias: env.alias, + organization: env.organization, + endpoint: env.endpoint, + }], + writePermissions: { + serviceConnection: env.endpoint, + connectionType: "azureDevOps", + allow: [{ + organization: env.organization, + projects: [{ + project: env.project, + repositories: [env.repository], + }], + }], + }, + }; +} + +function defaultBranch(defaultBranch: string | undefined): string { + return defaultBranch?.replace(/^refs\/heads\//, "") || "main"; +} + +export const createCrossOrgBranch: Scenario<{ + env: CrossOrgEnv; + branch: string; + base: string; +}> = { + id: "create-branch-cross-org", + tool: "create-branch", + setup: async (ctx) => { + const env = resolveCrossOrgEnv(ctx); + const repository = await env.rest.getRepository(env.repository); + return { + env, + branch: ctx.prefix("create-branch-cross-org"), + base: defaultBranch(repository.defaultBranch), + }; + }, + source: async (_ctx, state) => crossOrgSource(state.env), + config: (_ctx, state) => ({ + "allowed-repositories": [state.env.alias], + max: 1, + }), + env: async (_ctx, state) => ({ + SYSTEM_ACCESSTOKEN: state.env.token, + }), + ndjson: async (_ctx, state) => ({ + branch_name: state.branch, + source_branch: state.base, + repository: state.env.alias, + }), + assert: async (_ctx, state) => { + const sha = await state.env.rest.getRefObjectId( + state.env.repository, + `heads/${state.branch}`, + ); + if (!sha) throw new Error(`cross-org branch '${state.branch}' was not created`); + }, + cleanup: async (_ctx, state) => + state.env.rest.deleteRef(state.env.repository, `refs/heads/${state.branch}`), +}; + +export const createCrossOrgGitTag: Scenario<{ + env: CrossOrgEnv; + tag: string; +}> = { + id: "create-git-tag-cross-org", + tool: "create-git-tag", + setup: async (ctx) => ({ + env: resolveCrossOrgEnv(ctx), + tag: `ado-aw-det-${ctx.buildId}-cross-org-tag`, + }), + source: async (_ctx, state) => crossOrgSource(state.env), + config: (_ctx, state) => ({ + "allowed-repositories": [state.env.alias], + max: 1, + }), + env: async (_ctx, state) => ({ + SYSTEM_ACCESSTOKEN: state.env.token, + }), + ndjson: async (ctx, state) => ({ + tag_name: state.tag, + message: detBody(ctx, "create-git-tag-cross-org"), + repository: state.env.alias, + }), + assert: async (_ctx, state) => { + const sha = await state.env.rest.getRefObjectId( + state.env.repository, + `tags/${state.tag}`, + ); + if (!sha) throw new Error(`cross-org tag '${state.tag}' was not created`); + }, + cleanup: async (_ctx, state) => + state.env.rest.deleteRef(state.env.repository, `refs/tags/${state.tag}`), +}; + +export const crossOrgScenarios: Scenario[] = [ + createCrossOrgBranch, + createCrossOrgGitTag, +]; diff --git a/scripts/ado-script/src/executor-e2e/scenarios/index.ts b/scripts/ado-script/src/executor-e2e/scenarios/index.ts index 9165e8fad..e2d277abb 100644 --- a/scripts/ado-script/src/executor-e2e/scenarios/index.ts +++ b/scripts/ado-script/src/executor-e2e/scenarios/index.ts @@ -5,6 +5,7 @@ import type { Scenario } from "../scenario.js"; import { buildScenarios } from "./build.js"; import { createPullRequestScenarios } from "./create-pull-request.js"; +import { crossOrgScenarios } from "./cross-org.js"; import { gitScenarios } from "./git.js"; import { githubIssueScenarios } from "./github-issue.js"; import { prScenarios } from "./pr.js"; @@ -19,6 +20,7 @@ export const allScenarios: Scenario[] = [ ...wikiScenarios, ...prScenarios, ...gitScenarios, + ...crossOrgScenarios, ...buildScenarios, ...createPullRequestScenarios, ...githubIssueScenarios, diff --git a/scripts/ado-script/src/prepare-pr-base/__tests__/index.test.ts b/scripts/ado-script/src/prepare-pr-base/__tests__/index.test.ts index b81afa933..88bcc7ba3 100644 --- a/scripts/ado-script/src/prepare-pr-base/__tests__/index.test.ts +++ b/scripts/ado-script/src/prepare-pr-base/__tests__/index.test.ts @@ -37,6 +37,7 @@ function dependencies(opts: { deps: PrepareDependencies; calls: Array<{ args: string[]; env?: Record }>; dirs: string[]; + getCommitDiffMetadata: ReturnType; } { const calls: Array<{ args: string[]; env?: Record }> = []; const dirs: string[] = []; @@ -79,6 +80,7 @@ function dependencies(opts: { deps: { runners, chdir, getCommitDiffMetadata }, calls, dirs, + getCommitDiffMetadata, }; } @@ -132,6 +134,50 @@ describe("parseArgs", () => { }); }); + it("parses complete cross-organization repository coordinates", () => { + expect( + parseArgs([ + "--mode", + "patch-base", + "--repo-dir", + "/src", + "--organization", + "other-org", + "--project", + "Other Project", + "--repository", + "target-repo", + "--target-branch", + "main", + ]), + ).toEqual({ + mode: "patch-base", + repos: [{ + dir: "/src", + organization: "other-org", + project: "Other Project", + repository: "target-repo", + sourceRef: undefined, + target: "main", + }], + fallbackTarget: "main", + fallbackSourceRef: undefined, + }); + }); + + it("rejects partial cross-organization repository coordinates", () => { + expect(() => + parseArgs([ + "--repo-dir", + "/src", + "--organization", + "other-org", + "--target-branch", + "main", + ]), + ).toThrow(/must set --organization, --project, and --repository together/); + }); + it("rejects unknown modes", () => { expect(() => parseArgs(["--mode", "everything"])).toThrow(/Unsupported/); }); @@ -172,11 +218,12 @@ describe("prepare-pr-base main", () => { vi.spyOn(process.stdout, "write").mockImplementation(() => true); const error = Object.assign(new Error("forbidden"), { statusCode: 403 }); const { deps, calls } = dependencies({ metadataError: error }); - await main( + const rc = await main( patchArgs(), { SYSTEM_COLLECTIONURI: "https://dev.azure.com/org/" }, deps, ); + expect(rc).toBe(0); const fetches = calls.filter((call) => call.args[0] === "fetch"); expect(fetches).toHaveLength(1); expect(fetches[0]!.args).toContain("--depth=200"); @@ -188,6 +235,149 @@ describe("prepare-pr-base main", () => { ); }); + it("uses explicit cross-org coordinates and bearer for REST and fetch", async () => { + vi.spyOn(process.stdout, "write").mockImplementation(() => true); + const { deps, calls, getCommitDiffMetadata } = dependencies({ + remote: "https://dev.azure.com/other-org/Other%20Project/_git/target-repo", + }); + const rc = await main( + { + mode: "patch-base", + repos: [{ + dir: "/src", + target: "main", + sourceRef: "refs/heads/feature", + organization: "other-org", + project: "Other Project", + repository: "target-repo", + }], + fallbackTarget: "main", + }, + { SYSTEM_ACCESSTOKEN: "cross-token" }, + deps, + ); + + expect(getCommitDiffMetadata).toHaveBeenCalledWith( + "Other Project", + "target-repo", + "main", + HEAD, + "https://dev.azure.com/other-org/", + ); + const fetches = calls.filter((call) => call.args[0] === "fetch"); + expect(fetches[0]!.env).toMatchObject({ + GIT_CONFIG_VALUE_0: "Authorization: bearer cross-token", + }); + }); + + it("does not send the bearer when explicit coordinates mismatch the remote", async () => { + vi.spyOn(process.stdout, "write").mockImplementation(() => true); + const { deps, calls, getCommitDiffMetadata } = dependencies({ + remote: "https://dev.azure.com/unexpected/Other%20Project/_git/target-repo", + }); + const rc = await main( + { + mode: "patch-base", + repos: [{ + dir: "/src", + target: "main", + sourceRef: "refs/heads/feature", + organization: "other-org", + project: "Other Project", + repository: "target-repo", + }], + fallbackTarget: "main", + }, + { SYSTEM_ACCESSTOKEN: "must-not-leak" }, + deps, + ); + + expect(getCommitDiffMetadata).not.toHaveBeenCalled(); + expect(calls.some((call) => call.args[0] === "fetch")).toBe(false); + expect(rc).toBe(1); + }); + + it("rejects explicit coordinates that point at the current organization", async () => { + vi.spyOn(process.stdout, "write").mockImplementation(() => true); + const { deps, calls, getCommitDiffMetadata } = dependencies({ + remote: "https://dev.azure.com/org/Project/_git/repo", + }); + const rc = await main( + { + mode: "patch-base", + repos: [{ + dir: "/src", + target: "main", + organization: "org", + project: "Project", + repository: "repo", + }], + fallbackTarget: "main", + }, + { + SYSTEM_COLLECTIONURI: "https://dev.azure.com/org/", + SYSTEM_ACCESSTOKEN: "must-not-leak", + }, + deps, + ); + + expect(getCommitDiffMetadata).not.toHaveBeenCalled(); + expect(calls.some((call) => call.args[0] === "fetch")).toBe(false); + expect(rc).toBe(1); + }); + + it("rejects locale-equivalent but code-point-distinct repository names", async () => { + vi.spyOn(process.stdout, "write").mockImplementation(() => true); + const { deps, calls, getCommitDiffMetadata } = dependencies({ + remote: "https://dev.azure.com/other-org/Other%20Project/_git/%EF%BD%92%EF%BD%85%EF%BD%90%EF%BD%8F", + }); + const rc = await main( + { + mode: "patch-base", + repos: [{ + dir: "/src", + target: "main", + organization: "other-org", + project: "Other Project", + repository: "repo", + }], + fallbackTarget: "main", + }, + { SYSTEM_ACCESSTOKEN: "must-not-leak" }, + deps, + ); + + expect(getCommitDiffMetadata).not.toHaveBeenCalled(); + expect(calls.some((call) => call.args[0] === "fetch")).toBe(false); + expect(rc).toBe(1); + }); + + it("rejects Unicode names that JavaScript lowercases to an ASCII identifier", async () => { + vi.spyOn(process.stdout, "write").mockImplementation(() => true); + const { deps, calls, getCommitDiffMetadata } = dependencies({ + remote: "https://dev.azure.com/other-org/Other%20Project/_git/%E2%84%AAepo", + }); + const rc = await main( + { + mode: "patch-base", + repos: [{ + dir: "/src", + target: "main", + organization: "other-org", + project: "Other Project", + repository: "kepo", + }], + fallbackTarget: "main", + }, + { SYSTEM_ACCESSTOKEN: "must-not-leak" }, + deps, + ); + + expect(getCommitDiffMetadata).not.toHaveBeenCalled(); + expect(calls.some((call) => call.args[0] === "fetch")).toBe(false); + expect(rc).toBe(1); + }); + it("prefers the self resource ref over the triggering repository branch", async () => { vi.spyOn(process.stdout, "write").mockImplementation(() => true); const { deps, calls } = dependencies({ @@ -238,6 +428,33 @@ describe("prepare-pr-base main", () => { ]); }); + it("target-worktree sends the bearer to a matching explicit cross-org remote", async () => { + vi.spyOn(process.stdout, "write").mockImplementation(() => true); + const { deps, calls } = dependencies({ + remote: "https://dev.azure.com/other-org/Other%20Project/_git/target-repo", + }); + await main( + { + mode: "target-worktree", + repos: [{ + dir: "/src", + target: "main", + organization: "other-org", + project: "Other Project", + repository: "target-repo", + }], + fallbackTarget: "main", + }, + { SYSTEM_ACCESSTOKEN: "cross-token" }, + deps, + ); + + const fetch = calls.find((call) => call.args[0] === "fetch"); + expect(fetch?.env).toMatchObject({ + GIT_CONFIG_VALUE_0: "Authorization: bearer cross-token", + }); + }); + it("does not send the ADO bearer to a non-Azure origin", async () => { vi.spyOn(process.stdout, "write").mockImplementation(() => true); const { deps, calls } = dependencies({ @@ -281,6 +498,40 @@ describe("prepare-pr-base main", () => { expect(calls.filter((call) => call.args[0] === "fetch")).toHaveLength(1); }); + it("returns failure after processing later repos when a required target fails", async () => { + vi.spyOn(process.stdout, "write").mockImplementation(() => true); + const visited: string[] = []; + const { deps, calls } = dependencies({ + remote: "https://dev.azure.com/org/project/_git/repo", + chdir: (dir) => void visited.push(dir), + }); + const rc = await main( + { + mode: "target-worktree", + repos: [ + { + dir: "/required-cross-org", + target: "main", + organization: "other-org", + project: "Other Project", + repository: "target-repo", + }, + { dir: "/same-org", target: "main" }, + ], + fallbackTarget: "main", + }, + { + SYSTEM_COLLECTIONURI: "https://dev.azure.com/org/", + SYSTEM_ACCESSTOKEN: "cross-token", + }, + deps, + ); + + expect(rc).toBe(1); + expect(visited).toEqual(["/required-cross-org", "/same-org"]); + expect(calls.filter((call) => call.args[0] === "fetch")).toHaveLength(1); + }); + it("uses BUILD_SOURCESDIRECTORY and BUILD_SOURCEBRANCH for legacy no-arg calls", async () => { vi.spyOn(process.stdout, "write").mockImplementation(() => true); const { deps, dirs } = dependencies({ remote: "https://github.com/org/repo" }); diff --git a/scripts/ado-script/src/prepare-pr-base/index.ts b/scripts/ado-script/src/prepare-pr-base/index.ts index 5cc636699..e1b0f873d 100644 --- a/scripts/ado-script/src/prepare-pr-base/index.ts +++ b/scripts/ado-script/src/prepare-pr-base/index.ts @@ -39,6 +39,9 @@ export interface RepoTarget { dir: string; target: string; sourceRef?: string; + organization?: string; + project?: string; + repository?: string; } export interface PrepareArgs { @@ -56,6 +59,7 @@ export interface PrepareDependencies { repository: string, targetBranch: string, sourceCommit: string, + organizationUrl?: string, ) => Promise; } @@ -77,6 +81,12 @@ function oneLine(value: unknown, maxLength = 500): string { return text.length <= maxLength ? text : `${text.slice(0, maxLength)}...`; } +function sameAdoName(left: string, right: string): boolean { + const foldAscii = (value: string): string => + value.replace(/[A-Z]/g, (character) => character.toLowerCase()); + return foldAscii(left) === foldAscii(right); +} + function flushPending( repos: RepoTarget[], pending: Partial | null, @@ -84,10 +94,24 @@ function flushPending( fallbackSourceRef?: string, ): void { if (!pending?.dir) return; + const targetIdentity = [ + pending.organization, + pending.project, + pending.repository, + ]; + if (targetIdentity.some((value) => value !== undefined) && + targetIdentity.some((value) => value === undefined)) { + throw new Error( + `Repository '${pending.dir}' must set --organization, --project, and --repository together.`, + ); + } repos.push({ dir: pending.dir, target: pending.target ?? fallbackTarget, sourceRef: pending.sourceRef ?? fallbackSourceRef, + organization: pending.organization, + project: pending.project, + repository: pending.repository, }); } @@ -120,6 +144,15 @@ export function parseArgs(argv: string[]): PrepareArgs { if (pending?.dir) pending.target = target; else fallbackTarget = target; i++; + } else if (flag === "--organization") { + if (pending?.dir) pending.organization = value; + i++; + } else if (flag === "--project") { + if (pending?.dir) pending.project = value; + i++; + } else if (flag === "--repository") { + if (pending?.dir) pending.repository = value; + i++; } } flushPending(repos, pending, fallbackTarget, fallbackSourceRef); @@ -167,18 +200,55 @@ async function preparePatchBase( const remote = runners.gitOk(["remote", "get-url", "origin"]) ?? ""; const identity = parseAdoRepoUrl(remote); + const explicitIdentity = + repo.organization && repo.project && repo.repository + ? { + collectionUri: `https://dev.azure.com/${repo.organization}/`, + organization: repo.organization.toLowerCase(), + project: repo.project, + repository: repo.repository, + } + : null; + const identityMatchesExplicit = + identity !== null && + explicitIdentity !== null && + identity.organization === explicitIdentity.organization && + sameAdoName(identity.project, explicitIdentity.project) && + sameAdoName(identity.repository, explicitIdentity.repository); const sameOrgAdo = identity !== null && isCurrentAdoOrganization(identity, env); - const repoFetchEnv = sameOrgAdo ? fetchEnv : {}; + const eligibleIdentity = identityMatchesExplicit + ? explicitIdentity + : sameOrgAdo + ? identity + : null; + const repoFetchEnv = eligibleIdentity ? fetchEnv : {}; + if (explicitIdentity && sameOrgAdo) { + warnRepo( + repo.dir, + repo.target, + "compiler-resolved cross-organization target points at the current Azure DevOps organization", + ); + return false; + } + if (explicitIdentity && !identityMatchesExplicit) { + warnRepo( + repo.dir, + repo.target, + "checkout remote does not match the compiler-resolved Azure DevOps target", + ); + return false; + } const restDisabled = env.ADO_AW_PREPARE_PR_BASE_DISABLE_REST === "1"; if (restDisabled) { restReason = "ADO REST disabled for deterministic fallback testing"; - } else if (identity && sameOrgAdo) { + } else if (eligibleIdentity) { try { const metadata = await deps.getCommitDiffMetadata( - identity.project, - identity.repository, + eligibleIdentity.project, + eligibleIdentity.repository, repo.target, headSha, + eligibleIdentity.collectionUri, ); const exact = ensureExactMergeBaseFetched( repo.target, @@ -227,8 +297,38 @@ function prepareTargetWorktree( ): boolean { const remote = deps.runners.gitOk(["remote", "get-url", "origin"]) ?? ""; const identity = parseAdoRepoUrl(remote); + const explicitMatches = + identity !== null && + repo.organization !== undefined && + repo.project !== undefined && + repo.repository !== undefined && + identity.organization === repo.organization.toLowerCase() && + sameAdoName(identity.project, repo.project) && + sameAdoName(identity.repository, repo.repository); + if ( + repo.organization && + identity && + isCurrentAdoOrganization(identity, env) + ) { + warnRepo( + repo.dir, + repo.target, + "compiler-resolved cross-organization target points at the current Azure DevOps organization", + ); + return false; + } + if (repo.organization && !explicitMatches) { + warnRepo( + repo.dir, + repo.target, + "checkout remote does not match the compiler-resolved Azure DevOps target", + ); + return false; + } const repoFetchEnv = - identity && isCurrentAdoOrganization(identity, env) ? fetchEnv : {}; + identity && (isCurrentAdoOrganization(identity, env) || explicitMatches) + ? fetchEnv + : {}; const fetched = ensureTargetTipFetched(repo.target, repoFetchEnv, deps.runners); if (!fetched.ok) { warnRepo(repo.dir, repo.target, fetched.reason); @@ -258,21 +358,25 @@ export async function main( ]; } const fetchEnv = bearerEnv(env.SYSTEM_ACCESSTOKEN); + let requiredTargetFailed = false; for (const repo of repos) { try { deps.chdir(repo.dir); } catch (err) { warnRepo(repo.dir, repo.target, `could not enter checkout: ${oneLine(err)}`); + if (repo.organization) requiredTargetFailed = true; continue; } + let prepared: boolean; if (args.mode === "target-worktree") { - prepareTargetWorktree(repo, env, fetchEnv, deps); + prepared = prepareTargetWorktree(repo, env, fetchEnv, deps); } else { - await preparePatchBase(repo, env, fetchEnv, deps); + prepared = await preparePatchBase(repo, env, fetchEnv, deps); } + if (!prepared && repo.organization) requiredTargetFailed = true; } - return 0; + return requiredTargetFailed ? 1 : 0; } if ( diff --git a/scripts/ado-script/src/shared/__tests__/auth.test.ts b/scripts/ado-script/src/shared/__tests__/auth.test.ts index 1bc555261..67d7a603d 100644 --- a/scripts/ado-script/src/shared/__tests__/auth.test.ts +++ b/scripts/ado-script/src/shared/__tests__/auth.test.ts @@ -51,4 +51,38 @@ describe("getWebApi", () => { const b = await getWebApi(); expect(a).toBe(b); }); + + it("uses a bearer handler for compiler-minted access tokens", async () => { + process.env.SYSTEM_COLLECTIONURI = "https://example.visualstudio.com/"; + process.env.SYSTEM_ACCESSTOKEN = "entra-token"; + process.env.ADO_AW_ACCESS_TOKEN_KIND = "bearer"; + + const api = await getWebApi(); + + expect(api.authHandler.constructor.name).toBe("BearerCredentialHandler"); + }); + + it("retains PAT handling for local/manual tokens", async () => { + process.env.SYSTEM_COLLECTIONURI = "https://example.visualstudio.com/"; + process.env.SYSTEM_ACCESSTOKEN = "pat-token"; + delete process.env.ADO_AW_ACCESS_TOKEN_KIND; + + const api = await getWebApi(); + + expect(api.authHandler.constructor.name).toBe( + "PersonalAccessTokenCredentialHandler", + ); + }); + + it("replaces a cached client when the short-lived token rotates", async () => { + process.env.SYSTEM_COLLECTIONURI = "https://example.visualstudio.com/"; + process.env.ADO_AW_ACCESS_TOKEN_KIND = "bearer"; + process.env.SYSTEM_ACCESSTOKEN = "first-token"; + const first = await getWebApi(); + + process.env.SYSTEM_ACCESSTOKEN = "second-token"; + const second = await getWebApi(); + + expect(second).not.toBe(first); + }); }); diff --git a/scripts/ado-script/src/shared/ado-client.ts b/scripts/ado-script/src/shared/ado-client.ts index 7d3e7c26f..4ebf3968f 100644 --- a/scripts/ado-script/src/shared/ado-client.ts +++ b/scripts/ado-script/src/shared/ado-client.ts @@ -130,10 +130,11 @@ export async function getCommitDiffMetadata( repositoryId: string, targetBranch: string, sourceCommit: string, + organizationUrl?: string, ): Promise { const sourceSha = requireSha("sourceCommit", sourceCommit); return withRetry("getCommitDiffMetadata", async () => { - const git = await (await getWebApi()).getGitApi(); + const git = await (await getWebApi(organizationUrl)).getGitApi(); const branch = await git.getBranch(repositoryId, targetBranch, project); const targetSha = requireSha("target branch commit", branch.commit?.commitId); const result = await git.getCommitDiffs( diff --git a/scripts/ado-script/src/shared/auth.ts b/scripts/ado-script/src/shared/auth.ts index 889290f2f..b6d0531b8 100644 --- a/scripts/ado-script/src/shared/auth.ts +++ b/scripts/ado-script/src/shared/auth.ts @@ -19,6 +19,8 @@ * * Env-var contract: * - `SYSTEM_ACCESSTOKEN` ← `$(System.AccessToken)` + * - `ADO_AW_ACCESS_TOKEN_KIND=bearer` for compiler-minted OAuth/Entra tokens; + * omitted for local/manual PAT usage * - collection URI ← ADO's auto-injected `SYSTEM_COLLECTIONURI` * (falls back to `SYSTEM_TEAMFOUNDATIONCOLLECTIONURI`). Both are * predefined ADO variables auto-mapped into the env of every script @@ -30,17 +32,16 @@ import * as azdev from "azure-devops-node-api"; import type { WebApi } from "azure-devops-node-api"; import { logError } from "./vso-logger.js"; -let cached: WebApi | undefined; +const cached = new Map(); /** For tests only: clear the cached WebApi. */ export function _resetCacheForTesting(): void { - cached = undefined; + cached.clear(); } -export async function getWebApi(): Promise { - if (cached) return cached; - +export async function getWebApi(organizationUrl?: string): Promise { const orgUrl = + organizationUrl || process.env.SYSTEM_COLLECTIONURI || process.env.SYSTEM_TEAMFOUNDATIONCOLLECTIONURI; const token = process.env.SYSTEM_ACCESSTOKEN; @@ -55,8 +56,17 @@ export async function getWebApi(): Promise { logError(msg); throw new Error(msg); } + const tokenKind = + process.env.ADO_AW_ACCESS_TOKEN_KIND === "bearer" ? "bearer" : "pat"; + const cacheKey = `${tokenKind}:${orgUrl}`; + const existing = cached.get(cacheKey); + if (existing?.token === token) return existing.client; - const handler = azdev.getPersonalAccessTokenHandler(token); - cached = new azdev.WebApi(orgUrl, handler); - return cached; + const handler = + tokenKind === "bearer" + ? azdev.getBearerHandler(token) + : azdev.getPersonalAccessTokenHandler(token); + const client = new azdev.WebApi(orgUrl, handler); + cached.set(cacheKey, { token, client }); + return client; } diff --git a/site/src/content/docs/reference/ado-script.mdx b/site/src/content/docs/reference/ado-script.mdx index 56883c4c9..a4c24d407 100644 --- a/site/src/content/docs/reference/ado-script.mdx +++ b/site/src/content/docs/reference/ado-script.mdx @@ -24,7 +24,7 @@ pipelines** as runtime helpers. Today it produces fifteen bundles: - **`conclusion.js`** — Conclusion job reporter that files/comments ADO work items for pipeline failures and diagnostic signals (Conclusion job) - **`approval-summary.js`** — renders a sanitized per-tool summary of proposed safe outputs into the build's `ado-aw-safe-outputs` summary tab (end of Agent job) - **`github-app-token.js`** — mints (and revokes) a GitHub App installation token for the Copilot engine when `engine.github-app-token` is configured (Agent + Detection jobs) -- **`prepare-pr-base.js`** — Agent `patch-base` mode uses ADO diff metadata to fetch and verify only the source/target history needed for the merge-base (bounded 200/500/2000 dual-ref fallback); SafeOutputs `target-worktree` mode fetches only the target tip at depth 1 (issue #1453) +- **`prepare-pr-base.js`** — Agent `patch-base` mode uses ADO diff metadata to fetch and verify only the source/target history needed for the merge-base (bounded 200/500/2000 dual-ref fallback); SafeOutputs `target-worktree` mode fetches only the target tip at depth 1. Authorized cross-org repos run in a separate trusted credential scope and must match compiler-resolved organization/project/repository coordinates before receiving the bearer.