Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions .ai/skills/hack-repo-verify/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,11 @@ does not mean CI ran.
Hosted CI runs the `bun run test` file set on eight `test shard i/8` jobs;
light PRs run only the five Linux shards and report selected macOS paths as
deferred to the integration run. `ci:mac`/`ci:e2e` opt into selected tiers;
`ci:full` opts into all ordinary PR tiers. Ready Rust-input PRs retain Linux
`ci:full` opts into all ordinary PR tiers. Ready CLI-source PRs (`src/`, `index.ts`, `packages/cli/`) retain Linux x64
Config compiler and relocated acceptance, including refusal HOME/output contracts.
The required test fan-in rejects selected compiler failures, cancellations and
unexpected skips. This adds no default Mac minutes; the existing Linux lane also
builds the release bundle. Ready Rust-input PRs retain Linux
candidate/compiler checks while deferring macOS. Full tiers run automatically on
ready PRs changing protected CI inputs or targeting
`main`; the latter retains Intel. Light PRs skip build smoke, macOS and Docker;
Expand All @@ -160,7 +164,9 @@ The required `test` check is their fan-in. Every planned shard and the enabled
Linux process-lifetime job must succeed; failure, cancellation or an unexpected
skip fails. A planned out-of-scope lifetime skip passes. Drafts deliberately fail with
`skipped: draft`. Next pushes can select tests only after a successful predecessor;
superseded pushes cancel. Main and explicit full dispatches retain release-host
the active next push finishes while newer pushes replace only the pending run.
After a batch merges, no further merge proceeds until a next run completes,
unless the batch verifier ran the same checks on the combined tree. Main and explicit full dispatches retain release-host
coverage. Prerelease plan/publish require exact-head `release-full` evidence from
the selected run's latest attempt, including every shard and compiler host.
[`scripts/test-shard-plan.ts`](../../../scripts/test-shard-plan.ts) splits the files
Expand Down
23 changes: 21 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ concurrency:
# Let the active next run finish, including jobs waiting for macOS capacity.
# New next pushes replace only the pending run. PRs still cancel stale runs;
# main pushes and explicit release qualification keep unique groups.
# The lead waits for a completed next run between batches unless its batch
# verifier ran the same checks: pending revisions may still be replaced.
group: ${{ github.workflow }}-${{ github.event_name == 'pull_request' && format('pr-{0}', github.event.pull_request.number) || (github.event_name == 'push' && github.ref == 'refs/heads/next' && 'push-next') || format('push-{0}', github.run_id) }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

Expand Down Expand Up @@ -90,6 +92,7 @@ jobs:
# This guard is independent of the selector and its fallback list.
full=false
linux_rust=false
relocated_cli=false
base=""
if [ "$EVENT_NAME" = "pull_request" ]; then
parents=$(git rev-list --parents -n 1 HEAD) || parents=""
Expand All @@ -114,6 +117,10 @@ jobs:
else
if [ ! -s "$RUNNER_TEMP/ci-paths" ]; then full=true; fi
while IFS= read -r -d '' path; do
case "$path" in
src/*|index.ts|packages/cli/*|scripts/check-*-cli.ts|scripts/check-native-plan-clis.ts)
relocated_cli=true ;;
esac
case "$path" in
packages/runtime-core/*.md|packages/config-compiler/*.md) ;;
packages/runtime-core/*|packages/config-compiler/*|packages/relay-guest/*)
Expand Down Expand Up @@ -200,6 +207,8 @@ jobs:
compiler='[{"os":"ubuntu-latest","platform":"linux","arch":"x64"},{"os":"ubuntu-24.04-arm","platform":"linux","arch":"arm64"}]'
native=true
echo 'compiler_rust=true' >> "$GITHUB_OUTPUT"
elif [ "$PR_DRAFT" != "true" ] && [ "$relocated_cli" = "true" ]; then
compiler='[{"os":"ubuntu-latest","platform":"linux","arch":"x64"}]'
fi
printf 'compiler_hosts=%s\nruntime_core=%s\ntest_shards=[4,5,6,7,8]\ndefer_macos=true\n' "$compiler" "$native" >> "$GITHUB_OUTPUT"
else
Expand All @@ -208,11 +217,15 @@ jobs:
compiler=$(sed -n 's/^compiler_hosts=//p' "$GITHUB_OUTPUT" | tail -n 1)
compiler=$(jq -c '. + [{"os":"ubuntu-latest","platform":"linux","arch":"x64"},{"os":"ubuntu-24.04-arm","platform":"linux","arch":"arm64"}] | unique_by(.os) | map(select(.os != "macos-15-intel"))' <<< "${compiler:-[]}")
printf 'compiler_hosts=%s\ncompiler_rust=true\nruntime_core=true\n' "$compiler" >> "$GITHUB_OUTPUT"
elif [ "$relocated_cli" = "true" ] && [ "$full" != "true" ]; then
compiler=$(sed -n 's/^compiler_hosts=//p' "$GITHUB_OUTPUT" | tail -n 1)
compiler=$(jq -c '. + [{"os":"ubuntu-latest","platform":"linux","arch":"x64"}] | unique_by(.os)' <<< "${compiler:-[]}")
printf 'compiler_hosts=%s\n' "$compiler" >> "$GITHUB_OUTPUT"
fi
fi
if [ "$e2e" != "true" ]; then echo 'docker_suite=false' >> "$GITHUB_OUTPUT"; fi
printf 'build_smoke=%s\n' "$builds" >> "$GITHUB_OUTPUT"
echo 'Light PRs defer macOS, Docker and build smoke to integration; Rust crate inputs keep Linux candidate/compiler checks. Labels opt in; protected inputs, unavailable policy/diff and ready PRs into main automatically run full tiers.' >> "$GITHUB_STEP_SUMMARY"
echo 'Light PRs defer macOS, Docker and build smoke to integration; CLI inputs keep Linux relocated acceptance, and Rust crate inputs keep Linux candidate/compiler checks. Labels opt in; protected inputs, unavailable policy/diff and ready PRs into main automatically run full tiers.' >> "$GITHUB_STEP_SUMMARY"
else
printf 'defer_macos=false\nbuild_smoke=true\n' >> "$GITHUB_OUTPUT"
fi
Expand Down Expand Up @@ -909,7 +922,7 @@ jobs:
# succeed; when it runs a focus instead, the focused job must. A cancelled
# result fails, as do a failure, a failed scope or any skip of
# the job in scope still fails rather than surfacing as a skipped check.
needs: [changes, test-shard, test-focus, linux-process-lifetime]
needs: [changes, test-shard, test-focus, linux-process-lifetime, config-compiler]
if: ${{ always() }}
runs-on: ubuntu-latest
timeout-minutes: 5
Expand All @@ -924,6 +937,8 @@ jobs:
TEST_FOCUS: ${{ needs.changes.outputs.test_focus }}
SHARD_RESULT: ${{ needs.test-shard.result }}
FOCUS_RESULT: ${{ needs.test-focus.result }}
COMPILER_ENABLED: ${{ needs.changes.outputs.compiler_hosts != '[]' }}
COMPILER_RESULT: ${{ needs.config-compiler.result }}
LIFETIME_ENABLED: ${{ needs.changes.outputs.docker_suite }}
LIFETIME_RESULT: ${{ needs.linux-process-lifetime.result }}
run: |
Expand All @@ -939,6 +954,10 @@ jobs:
echo "change scope did not succeed (result: $CHANGES_RESULT)"
exit 1
fi
case "$COMPILER_ENABLED:$COMPILER_RESULT" in
true:success|false:skipped) ;;
*) echo "Compiler acceptance evidence did not match its plan (enabled: $COMPILER_ENABLED, result: $COMPILER_RESULT)" >&2; exit 1 ;;
esac
if [ "$FOCUS_RESULT" != "success" ]; then
echo "test focus did not succeed (result: $FOCUS_RESULT)"
exit 1
Expand Down
36 changes: 33 additions & 3 deletions docs/reference/ci-test-selection.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,13 @@

A PR uses a light tier by default: typecheck, lint/privacy, secret scan, docs
checks when relevant, and selected Linux unit files on five Linux shards. It runs
no macOS jobs, Docker E2E shards, state-model job or CLI/release build smoke.
no macOS jobs, Docker E2E shards or state-model job. Ready PRs changing CLI
source (`src/`, `index.ts`, `packages/cli/`) retain the existing Linux x64
Config compiler job, including its release build and relocated CLI acceptance.
This deliberately covers activity recording, HOME writes, output envelopes and
refusals through any shared CLI source, rather than maintaining a fragile list
of lifecycle helpers. Changes to acceptance scripts retain that job through the
protected-script full-tier rule. Unrelated test/docs PRs retain the lighter tier.
Ready Rust-input PRs retain the Linux Candidate core leg and both Linux compiler
hosts (including Rust checks); macOS stays in the heavy tier. This covers
non-Markdown inputs under runtime-core/config-compiler (including pins, fixtures
Expand All @@ -28,7 +34,11 @@ fallback alone widens files; the independent shell guard also enables full tiers
Adding/removing labels, pushing, changing readiness or retargeting recomputes
scope; metadata edits can cancel the active run.

The required `test` fan-in also depends on `linux-process-lifetime`. When that
The required `test` fan-in also depends on `config-compiler` and
`linux-process-lifetime`. Every selected compiler host must succeed; its planned
out-of-scope skip passes. When native tiers are selected, the required check
also waits for their macOS compiler hosts, including Intel on release/full-main
runs. When that
tier is enabled, failure, cancellation, missing results or an unexpected skip
fails `test`, even when every unit shard passed. Its planned out-of-scope skip
passes. Branch rules are unchanged.
Expand Down Expand Up @@ -105,9 +115,21 @@ working directory and Bun preload.

The candidate job follows its crate, its config-compiler path dependency, guest, launcher, helper/toolchain and CI
inputs. The compiler's native host matrix follows its crate/build/toolchain/CI
inputs. Other compiler checks retain the previous Linux scope. These are job
inputs. CLI source changes retain Linux x64 compiler acceptance even when
macOS is deferred. These are job
input gates, separate from the unit import graph.

Cost: this adds **0 macOS minutes** to an ordinary CLI PR and one Linux x64
Config compiler job, without adding a second release build. A Linux measurement
on the 5002cf87 base with Bun 1.4.4 and Rust 1.97.1 took 59.5 seconds for the
eight relocated CLI scripts, 0.7 seconds for the CLI build, and 41.9 seconds for
compiler preparation with cached dependencies (102.1 seconds combined). These
are component timings, not the whole hosted lane: that job also runs schema,
transport/unit and release-install checks plus setup. Its separate release-install
check needs host sandbox privileges and is verified in hosted CI. The lead's
Mac batch reported roughly 80 seconds for all nine acceptance scripts once the
release build exists; that is Mac evidence, separate from this Linux measurement.

The Intel compiler lane runs on ready PRs into `main`, `main` pushes and explicit
full dispatches. Native-tier PRs into `next` and ordinary `next` integration runs
retain ARM macOS and both Linux compiler hosts. Intel-specific regressions on
Expand All @@ -121,6 +143,14 @@ including jobs waiting for macOS capacity. GitHub retains at most one pending ru
a newer push replaces that pending run, so intermediate revisions may not run.
As described above, a cancelled or missing predecessor forces full selection,
so replacing a pending run cannot omit its merged changes from coverage.
After one batch merges, the lead makes no further merge until a `next` run
completes, unless the batch verifier ran the same checks on the combined tree.
A cancelled or merely queued run does not satisfy that rule. The existing
non-cancelling workflow group protects the active run, but GitHub can replace
pending runs, so it cannot enforce a completed checkpoint between batches.
No job-class concurrency split is needed: it would duplicate scheduling and
could leave acceptance without matching platform evidence.

This preserves all selected macOS coverage without changing runner capacity or
the PR layout. It does not reserve macOS slots or guarantee a queue wait bound.
PRs still cancel stale runs for the same PR. `main` pushes retain unique,
Expand Down
4 changes: 2 additions & 2 deletions docs/reference/ci-test-selection.mmd
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ flowchart TD
guard -->|Yes or unavailable policy| allpr
event -->|Draft PR| draft[Draft Linux tier]
draft --> draftgate[Fan-ins fail: skipped draft]
labels -->|Default| light[Linux checks plus relevant Rust legs]
labels -->|Default| light[Linux plus CLI acceptance or Rust legs]
light --> deferred[Name deferred macOS files]
deferred --> prgate[Required test and docs checks]
deferred --> prgate[Test fan-in includes compiler acceptance]
labels -->|ci:mac or ci:e2e| tier[Selected opted-in tier]
labels -->|ci:full| allpr[Full PR tiers]
tier --> prgate
Expand Down
Loading
Loading