diff --git a/AGENTS.md b/AGENTS.md index dc5ef83a5..2909f17c6 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -36,8 +36,8 @@ - Daemon, service, shortcut, path, and packaging changes often need updates across `src/daemon/`, `src/paths/`, `src/systemd_user_service.rs`, `src/shortcut_hint.rs`, `configurator/src/app/daemon_setup/`, and `packaging/`. ## Validation -- Full local CI is `./tools/lint-and-test.sh`. -- That script runs version/package checks, `cargo fmt --all -- --check`, clippy with all targets/features, all-feature tests, and no-default-feature tests. +- Full local CI is `./tools/lint-and-test.sh`. It needs the .NET SDK selected by `global.json`. Without it, `cargo test` still runs the Rust source guards in `tests/repository_guards` (process sites, config writers, shared dependencies, no Python), but not the C# checks, including the source-coverage check that finds `.rs` files Cargo never compiles. +- That script runs the C# repository checks (assets, version, nixpkgs recipe, source coverage, legacy tools), the C# tool tests (which also run the packaging shell contracts), `cargo fmt --all -- --check`, clippy with all targets/features, all-feature tests, and no-default-feature tests. - Run `git diff --check` before handoff. When relevant files are untracked and the index must stay unchanged, check those files directly as well. - For docs-only `AGENTS.md` edits, make new files visible to Git before whitespace checks, for example `rg --files --hidden -g AGENTS.md -0 | xargs -0 git add -N --` followed by `git diff --check`. - On PowerShell, use `rg --files --hidden -g AGENTS.md | ForEach-Object { git add -N -- $_ }` followed by `git diff --check`. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 508542d86..a8c495fe5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -40,10 +40,13 @@ Git hash, and tells Cargo which Git metadata should trigger a rebuild. Cargo fea GitHub CI uses the C# file app. To run that route locally, build it once with `dotnet build tools/wayscriber.cs`, then run commands with -`dotnet run tools/wayscriber.cs --no-build -- ...`. The standalone scripts remain -available for development and installation on machines without .NET. Nix does +`dotnet run tools/wayscriber.cs --no-build -- ...`. Repository, release, and +packaging checks exist only there, so the complete local gate needs .NET. Nix does not provide .NET; `global.json` selects the required SDK when it is installed -separately. +separately. Without it, `cargo test` still runs the ownership guards in +`tests/repository_guards`, and the build, run, and install scripts stay usable; +the C# checks, including source coverage, need .NET. +The repository uses no Python. Build both packages without launching a window: @@ -128,12 +131,13 @@ Before submitting a broad or cross-package change, run the local CI entry point: ./tools/lint-and-test.sh ``` -It checks release/package metadata, all three retained release contracts, package layout, Rust -source coverage, formatting, strict all-feature Clippy, all-feature tests, and no-default-feature -tests. When the pinned .NET SDK is installed, it also builds and runs the C# repository-tool tests; -otherwise it reports that optional local check as skipped. The source-coverage gate uses current -rustc dep-info and rejects tracked or unignored `.rs` files that are outside the supported Cargo -target/feature matrix. +It needs the pinned .NET SDK and stops without it. It checks release/package metadata, Rust +source coverage, and the shell-tool inventory; builds and runs the C# repository-tool tests, +which also run the package-layout and release-packaging shell contracts; then checks +formatting, strict all-feature Clippy, all-feature tests, and no-default-feature tests. The +Rust tests include the repository guards in `tests/repository_guards`. The source-coverage +gate uses current rustc dep-info and rejects tracked or unignored `.rs` files that are outside +the supported Cargo target/feature matrix. The all-feature portal transport tests require `dbus-daemon`. Each fixture owns a private session bus and connects through its explicit address, leaving the desktop session bus alone. @@ -216,7 +220,8 @@ while preserving the original output identity and request. A second change durin active-output switch is terminal. Board PDF desktop captures retain a full-desktop generation check, including changes to other monitors even when the screenshot dimensions stay the same. -`./tools/code-health-report.sh` reports navigational maintainability metrics. Its CI artifact is +`dotnet run tools/wayscriber.cs --no-build -- report code-health` reports navigational +maintainability metrics. Its CI artifact is observational, not a global file/function-size gate; use the report to find code worth understanding, not as a reason for mechanical splitting. @@ -226,32 +231,33 @@ not as a reason for mechanical splitting. - Installation, service, and shortcut behavior belongs in `docs/SETUP.md` and packaging docs. - Main-crate architecture belongs in `docs/codebase-overview.md`. - Drafts under `docs/temp/` are planning material unless explicitly promoted. -- Version changes must go through `tools/bump-version.sh`; keep both package manifests, root - `Cargo.lock`, packaging metadata, and tag/release policy aligned. +- Version changes must go through the C# `version bump` command; keep both package manifests, + root `Cargo.lock`, packaging metadata, and tag/release policy aligned. - Close a user-visible "I don't have that setting / this build" report only after the change is in a tagged GitHub release. `main` is not what `arch-install.sh`, AUR `wayscriber-bin`, or other packaged installs ship. `--version` reports the crate version, not the git hash, so bump with - `tools/bump-version.sh` in the same change as a user-visible overlay, settings, or config toggle + `version bump` in the same change as a user-visible overlay, settings, or config toggle (or immediately before tagging that release). Otherwise two binaries can print the same `wayscriber 0.9.x` and look identical. See [tools/README.md](tools/README.md) for build, install, packaging, version, and release helpers. -Run `./tools/lint-and-test.sh` for the standalone local gate. It lints, builds -binaries, and tests the whole workspace with all features and with no default -features, alongside source, packaging, retained release-contract, and C# tool -checks when .NET is installed. CI runs the equivalent C# command and additionally checks dynamic +Run `./tools/lint-and-test.sh` for the complete local gate; it needs the .NET SDK +selected by `global.json`. It builds the C# tools, then runs the same steps as CI's +`ci lint-and-test`: the C# repository checks, C# formatting, and the C# tests +(which run the retained packaging shell contracts), then lints, builds binaries, +and tests the whole workspace with all features and with no default features. CI runs the equivalent C# command and additionally checks dynamic and static gtk4-layer-shell linkage and uploads its code-health report. GTK widget coverage runs separately with `./tools/test-gtk-widgets.sh` (Weston, -`dbus-run-session`, Python 3, `pkg-config`, Mesa software OpenGL, and Wayland -protocol XML required). It creates a private headless display and requires GTK -initialization; an unavailable display fails this check. The native popup tests -use another private Weston with software OpenGL and a protocol proxy to hold one -popup frame callback while its shared clock paints, then require capture to finish -after the popup's fresh render is acknowledged. They cover a plain `GtkPopover` -and `GtkPopoverMenu`'s empty proof overlay and menu restoration. Successful bodies -print `EXECUTED` markers. Ordinary widget tests without a display report an +`dbus-run-session`, and Mesa software OpenGL required). It creates a private +headless display and requires GTK initialization; an unavailable display fails +this check. The native popup tests use another private Weston with software +OpenGL and a Rust protocol proxy to hold one popup frame callback while its +shared clock paints, then require capture to finish after the popup's fresh +render is acknowledged. They cover a plain `GtkPopover` and `GtkPopoverMenu`'s +empty proof overlay and menu restoration. Successful bodies print `EXECUTED` +markers. Ordinary widget tests without a display report an optional skip; when a display is available, their GTK assertions run. The native popup tests require the dedicated GTK gate. Neither route proves layer-shell focus or screen capture behavior on a user's compositor. diff --git a/docs/RELEASING.md b/docs/RELEASING.md index bccbcbaf2..de2ef2025 100644 --- a/docs/RELEASING.md +++ b/docs/RELEASING.md @@ -6,10 +6,15 @@ Land tooling fixes, test routing, and release documentation as focused commits, then commit the version bump separately immediately before tagging. An uncommitted release preparation may contain these groups together; stage them separately at handoff. -1. Run `./tools/bump-version.sh X.Y.Z`. It updates both workspace versions and - package metadata without refreshing locked dependencies. Review `Cargo.lock`: +The version and tag steps use the C# repository tool. Build it once with +`dotnet build tools/wayscriber.cs`; the commands below then run it with `--no-build`. + +1. Run `dotnet run tools/wayscriber.cs --no-build -- version bump X.Y.Z`. It updates + both workspace versions and package metadata without refreshing locked dependencies, + then runs `version check --release-version X.Y.Z`. Review `Cargo.lock`: a version-only release should change only the two workspace package versions. - Prefetch dependencies first if the local Cargo cache is empty. + If the local Cargo cache is empty, the bump stops before it changes a file; prefetch + dependencies with `dotnet run tools/wayscriber.cs --no-build -- dev fetch` first. 2. Run `./tools/lint-and-test.sh` and `./tools/test-gtk-widgets.sh`. The canonical gate serializes the Rust test harness to avoid the observed parallel native-font crashes. It also runs each context-menu and board-picker retained-text rendering regression @@ -27,8 +32,11 @@ release preparation may contain these groups together; stage them separately at ## Publish and verify After reviewing and committing the release changes, push the branch and wait for -its GitHub checks. Use `./tools/publish-release-tag.sh --version X.Y.Z` only when -ready to publish. It creates and pushes the tag; the tag starts the Release workflow. +its GitHub checks. Use +`dotnet run tools/wayscriber.cs --no-build -- release publish-tag --version X.Y.Z` +only when ready to publish. It repeats the version check for that release version, +requires a clean working tree and an unused tag, and then creates and pushes the tag; +the tag starts the Release workflow. Add `--dry-run` to run the checks without tagging. Verify the whole Release workflow, including the GitHub assets, AUR recipes, and apt/rpm repository deployment. AUR waits for successful GitHub asset publication. diff --git a/docs/codebase-overview.md b/docs/codebase-overview.md index 489a2b7cc..26032b38a 100644 --- a/docs/codebase-overview.md +++ b/docs/codebase-overview.md @@ -326,7 +326,7 @@ capture suppression operates on the paired resources without runtime pairing che slot, quick color), which each rewrite one key. Everything else — the overlay's other controls, the daemon, the tray, startup, validation, migration preview, and shutdown — reads the file and leaves its bytes, mode, and mtime alone, including for a missing, read-only, or old-revision - file. `tools/check-config-writers.py` pins that set by name. + file. `tests/repository_guards/config_writers.rs` pins that set by name. - Every one of those writes goes through `ConfigDocument::save_with_backup`, which holds an advisory lock on a sibling `config.toml.lock` across the whole check-copy-rename window (`src/config/document/lock.rs`). The revision check and the atomic rename are separate syscalls @@ -423,7 +423,7 @@ capture suppression operates on the paired resources without runtime pairing che submission joins a staging queue in front of the channel and is pumped in as the worker makes room, so a burst that fills the channel cannot answer the newest gesture ahead of the older ones it was made after (which would leave their completions applying on top of it). That module is the - only production caller of the three editors, and is what `tools/check-config-writers.py` pins. + only production caller of the three editors, and is what `tests/repository_guards/config_writers.rs` pins. - Teardown is `finish_config_edits` (called by `shutdown_config_edits`, beside `shutdown_runtime_ui`). It drains `InputEffectDrain::DurableConfig` — the outbox-owned inventory of preset, quick-color, and recorded shortcut edits — one last time before stopping the worker, @@ -486,7 +486,7 @@ capture suppression operates on the paired resources without runtime pairing che then arbitrated by traversal order instead of being filtered away as an unauthored default, and the configurator's save status names which action kept the key: the resolution reaches `config.toml`, so the reloaded document has nothing left to report. -- Two guards keep it that way: `tools/check-config-writers.py` (in `tools/lint-and-test.sh`) fails +- Two guards keep it that way: `tests/repository_guards/config_writers.rs` (in every `cargo test`) fails when any source outside `src/config/document.rs`, `src/config/io.rs`, and `configurator/src/app/io.rs` names a config write primitive, when an unpinned file calls one of the narrow editors, or when the editors' path-taking `_at` twins stop being `#[cfg(test)]`-gated diff --git a/docs/daemon-protocol-v2.md b/docs/daemon-protocol-v2.md index 78da79286..8995b6599 100644 --- a/docs/daemon-protocol-v2.md +++ b/docs/daemon-protocol-v2.md @@ -63,7 +63,7 @@ process-start identity. The daemon does not mark that child ready until all thre exit is watched through the pidfd, while signals, tray intents, shortcut intents, and typed queue renames have owned wake descriptors; the daemon lifecycle has no periodic discovery tick. -The enforced process-site inventory is `tools/check-process-sites.py`. Direct process creation is +The enforced process-site inventory is `tests/repository_guards/process_sites.rs`. Direct process creation is limited to the broker, pre-runtime systemd setup, the separate configurator process, standalone About clipboard integration, and named test fixtures. The same check audits the raw-clone child stub: before `execve` it may reach only the fixed `fcntl`, `dup3`, `setpgid`, `close_range`, diff --git a/packaging/AGENTS.md b/packaging/AGENTS.md index 7bf77111e..a340f19f2 100644 --- a/packaging/AGENTS.md +++ b/packaging/AGENTS.md @@ -22,6 +22,6 @@ - Packaging changes may require `tools/`, `.github/`, setup docs, `src/systemd_user_service.rs`, `src/shortcut_hint.rs`, and configurator daemon setup updates. ## Validation -- Run `tools/check-version-consistency.sh` and `tools/test-package-repo-layout.sh` for package/version changes. -- Run `tools/check-nixpkgs-recipe.py` when dependencies, default features, or Nix build inputs change. +- Run `dotnet run tools/wayscriber.cs --no-build -- version check` and `tools/test-package-repo-layout.sh` for package/version changes. +- Run `dotnet run tools/wayscriber.cs --no-build -- check nixpkgs-recipe` when dependencies, default features, or Nix build inputs change. - Run `git diff --check` for metadata-only edits. diff --git a/packaging/nixpkgs/README.md b/packaging/nixpkgs/README.md index 7610a4d35..544b48c4b 100644 --- a/packaging/nixpkgs/README.md +++ b/packaging/nixpkgs/README.md @@ -7,7 +7,7 @@ We do not own that file — `nixpkgs` does. This copy exists so that: - packaging changes here (new system libraries, new installed files) are visible in the same commit as the change that requires them, and -- `tools/check-nixpkgs-recipe.py` can fail CI when a default Cargo feature needs +- `dotnet run tools/wayscriber.cs --no-build -- check nixpkgs-recipe` can fail CI when a default Cargo feature needs a system library the `nixpkgs` build does not declare. ## How versions reach nixpkgs diff --git a/packaging/nixpkgs/package.nix b/packaging/nixpkgs/package.nix index f18f01894..0c0f08b03 100644 --- a/packaging/nixpkgs/package.nix +++ b/packaging/nixpkgs/package.nix @@ -37,7 +37,7 @@ rustPlatform.buildRustPackage (finalAttrs: { # Keep in sync with the default feature set in Cargo.toml; the GTK inputs are # required by the `toolbar-gtk` default feature. - # Checked by tools/check-nixpkgs-recipe.py. + # Checked by `dotnet run tools/wayscriber.cs --no-build -- check nixpkgs-recipe`. buildInputs = [ cairo gtk4 diff --git a/src/config/io.rs b/src/config/io.rs index 6dc75014b..7ca5ba1d8 100644 --- a/src/config/io.rs +++ b/src/config/io.rs @@ -359,7 +359,7 @@ pub(crate) fn is_stale_source_error(error: &anyhow::Error) -> bool { /// non-idempotent validation step appears. Two things defend it in place of a /// behavioural test: `document_config_is_a_fixed_point_of_validation` below, /// which fails the moment validation stops being idempotent, and the -/// `authored_config()` check in `tools/check-config-writers.py`. +/// `authored_config()` check in `tests/repository_guards/config_writers.rs`. /// /// `verify` runs afterwards against the document the save parsed from the bytes /// it wrote — the merge output, not a re-read of the file — so a value that @@ -465,7 +465,7 @@ pub fn persist_keybinding_edit(action: Action, bindings: &[String]) -> Result) -> Result<()> { } pub struct Daemon { - pub(super) overlay_state: OverlayState, + pub(super) overlay: OverlayLifecycle, pub(super) should_quit: Arc, pub(super) visibility_intents: Arc, pub(super) initial_mode: Option, pub(super) initial_named_session_file: Option, - pub(super) active_named_session_file: Option, pub(super) instance_token: String, pub(super) freeze_on_show: bool, pub(super) tray_enabled: bool, @@ -97,16 +95,10 @@ pub struct Daemon { pub(super) tray_runtime: Option, pub(super) update_watch_thread: Option>, pub(super) global_shortcuts_listener: Option, - pub(super) overlay_child: OverlayChildOwner, - pub(super) overlay_active: Arc, pub(super) overlay_action_intents: Arc, - pub(super) pending_activation_token: Option, - pub(super) pending_toggle_request: Option, + pub(super) pending_launch: Option, pub(super) session_resume_override: Arc, pub(super) lock_file: Option, - pub(super) overlay_spawn_failures: u32, - pub(super) overlay_spawn_next_retry: Option, - pub(super) overlay_spawn_backoff_logged: bool, pub(super) last_plain_visibility_toggle_completed_at: Option, protocol_mode: DaemonControlProtocolMode, v2_command_owner: Option, @@ -131,13 +123,13 @@ impl Daemon { let override_state = Arc::new(AtomicU8::new(encode_session_override( session_resume_override, ))); + Self { - overlay_state: OverlayState::Hidden, + overlay: OverlayLifecycle::default(), should_quit: Arc::new(AtomicBool::new(false)), visibility_intents: Arc::new(VisibilityIntents::default()), initial_mode, initial_named_session_file, - active_named_session_file: None, instance_token: crate::daemon::generate_daemon_instance_token(), freeze_on_show: false, tray_enabled, @@ -145,61 +137,10 @@ impl Daemon { tray_runtime: None, update_watch_thread: None, global_shortcuts_listener: None, - overlay_child: OverlayChildOwner::default(), - overlay_active: Arc::new(AtomicBool::new(false)), - overlay_action_intents: Arc::new(OverlayActionIntents::default()), - pending_activation_token: None, - pending_toggle_request: None, - session_resume_override: override_state, - lock_file: None, - overlay_spawn_failures: 0, - overlay_spawn_next_retry: None, - overlay_spawn_backoff_logged: false, - last_plain_visibility_toggle_completed_at: None, - protocol_mode: DaemonControlProtocolMode::production(), - v2_command_owner: None, - v2_command_watcher: None, - v2_deadline_source: None, - v2_action_journal: None, - pending_action_admission_retry: Vec::new(), - action_admission_retry_at: None, - #[cfg(unix)] - signal_listener: None, - #[cfg(feature = "tray")] - tray_status: Arc::new(TrayStatusShared::new()), - } - } - - #[cfg(test)] - fn with_backend_runner_internal( - initial_mode: Option, - backend_runner: Arc, - ) -> Self { - let override_state = Arc::new(AtomicU8::new(SESSION_OVERRIDE_FOLLOW_CONFIG)); - Self { - overlay_state: OverlayState::Hidden, - should_quit: Arc::new(AtomicBool::new(false)), - visibility_intents: Arc::new(VisibilityIntents::default()), - initial_mode, - initial_named_session_file: None, - active_named_session_file: None, - instance_token: crate::daemon::generate_daemon_instance_token(), - freeze_on_show: false, - tray_enabled: true, - backend_runner: Some(backend_runner), - tray_runtime: None, - update_watch_thread: None, - global_shortcuts_listener: None, - overlay_child: OverlayChildOwner::default(), - overlay_active: Arc::new(AtomicBool::new(false)), overlay_action_intents: Arc::new(OverlayActionIntents::default()), - pending_activation_token: None, - pending_toggle_request: None, + pending_launch: None, session_resume_override: override_state, lock_file: None, - overlay_spawn_failures: 0, - overlay_spawn_next_retry: None, - overlay_spawn_backoff_logged: false, last_plain_visibility_toggle_completed_at: None, protocol_mode: DaemonControlProtocolMode::production(), v2_command_owner: None, @@ -220,20 +161,16 @@ impl Daemon { initial_mode: Option, backend_runner: Arc, ) -> Self { - Self::with_backend_runner_internal(initial_mode, backend_runner) + let mut daemon = Self::new(initial_mode, true, None, None); + daemon.backend_runner = Some(backend_runner); + + daemon } pub fn set_freeze_on_show(&mut self, enabled: bool) { self.freeze_on_show = enabled; } - pub(super) fn effective_named_session_file(&self) -> Option { - self.pending_toggle_request - .as_ref() - .and_then(|request| request.session_file.clone()) - .or_else(|| self.initial_named_session_file.clone()) - } - pub(super) fn session_resume_override(&self) -> Option { decode_session_override(self.session_resume_override.load(Ordering::Acquire)) } @@ -299,7 +236,7 @@ impl Daemon { return; } - let tray_overlay_active = self.overlay_active.clone(); + let tray_overlay_active = self.overlay.active_flag(); #[cfg(feature = "tray")] let tray_status = self.tray_status.clone(); #[cfg(not(feature = "tray"))] @@ -457,7 +394,7 @@ impl Daemon { #[cfg(not(feature = "tray"))] let update_sink = (); self.update_watch_thread = - start_update_watch(quit_event.clone(), self.overlay_active.clone(), update_sink); + start_update_watch(quit_event.clone(), self.overlay.active_flag(), update_sink); } match current_shortcut_runtime_backend() { @@ -556,7 +493,7 @@ impl Daemon { daemon_wake, self.v2_command_watcher.as_ref(), self.v2_deadline_source.as_ref(), - &self.overlay_child, + self.overlay.poll_fd(), )?; if readiness.deadline { self.v2_deadline_source @@ -649,7 +586,7 @@ impl Daemon { }; let mut admitted = Vec::with_capacity(actions.len()); let mut retry = Vec::new(); - let mut will_be_visible = self.overlay_state == OverlayState::Visible; + let mut will_be_visible = self.overlay.state() == OverlayState::Visible; let mut actions = actions.into_iter(); while let Some(action) = actions.next() { if !will_be_visible && matches!(action, crate::tray_action::TrayAction::LightDrawOff) { @@ -676,7 +613,7 @@ impl Daemon { // disposition. Admission is completed for the batch before side effects // begin, so an early runtime failure cannot silently lose the tail. for (action, prepared) in admitted { - if self.overlay_state == OverlayState::Hidden + if self.overlay.state() == OverlayState::Hidden && matches!(action, crate::tray_action::TrayAction::LightDrawOff) { let reason = "overlay remained hidden before LightDrawOff delivery"; @@ -692,7 +629,7 @@ impl Daemon { // A start held back by the spawn backoff is a failed delivery: an // entry left eligible would replay whenever the overlay next // started, long after the click that asked for it. - let delivery = if self.overlay_state == OverlayState::Hidden { + let delivery = if self.overlay.state() == OverlayState::Hidden { self.show_overlay() .and_then(ShowOutcome::require_shown) .and_then(|()| self.signal_overlay_action_ready(action)) @@ -765,7 +702,7 @@ impl Daemon { } if let Some(action) = legacy_request.overlay_action { - if self.overlay_state == OverlayState::Hidden + if self.overlay.state() == OverlayState::Hidden && matches!(action, crate::tray_action::TrayAction::LightDrawOff) { claimed.commit(EffectKind::NoOp)?; @@ -776,7 +713,7 @@ impl Daemon { claimed.defer()?; continue; } - if self.overlay_state == OverlayState::Hidden + if self.overlay.state() == OverlayState::Hidden && let Some(retry_in) = self.overlay_start_backoff() { claimed.reject(&overlay_start_backoff_reason(retry_in))?; @@ -794,7 +731,7 @@ impl Daemon { claimed.defer()?; continue; }; - let was_hidden = self.overlay_state == OverlayState::Hidden; + let was_hidden = self.overlay.state() == OverlayState::Hidden; let committed = claimed.commit(if was_hidden { EffectKind::StartAndDeliverAction } else { @@ -812,7 +749,7 @@ impl Daemon { continue; } - self.pending_toggle_request = Some(legacy_request); + self.replace_pending_launch_request(legacy_request); if was_hidden { if let Err(error) = self.show_overlay().and_then(ShowOutcome::require_shown) { let reason = format!("committed overlay start failed: {error:#}"); @@ -829,7 +766,7 @@ impl Daemon { journal.abandon_command(&command_identity, &prepared, &reason)?; return Err(error).context(reason); } - self.pending_toggle_request = None; + self.discard_pending_launch_options(); } continue; } @@ -848,7 +785,7 @@ impl Daemon { continue; } - let effect = if self.overlay_state == OverlayState::Visible { + let effect = if self.overlay.state() == OverlayState::Visible { EffectKind::HideReady } else { EffectKind::StartAndShow @@ -977,7 +914,7 @@ fn wait_for_daemon_lifecycle( daemon_wake: &RuntimeWakeSource, command_watcher: Option<&CommandQueueWatcher>, deadline_source: Option<&BootDeadlineSource>, - overlay_child: &OverlayChildOwner, + overlay_exit_fd: Option>, ) -> Result { let mut pollfds = vec![libc::pollfd { fd: daemon_wake.poll_fd().as_raw_fd(), @@ -1001,7 +938,7 @@ fn wait_for_daemon_lifecycle( }); index }); - let child_index = overlay_child.poll_fd().map(|fd| { + let child_index = overlay_exit_fd.map(|fd| { let index = pollfds.len(); pollfds.push(libc::pollfd { fd: fd.as_raw_fd(), @@ -1075,7 +1012,7 @@ fn wait_for_daemon_lifecycle( #[cfg(test)] impl Daemon { pub fn test_state(&self) -> OverlayState { - self.overlay_state + self.overlay.state() } } diff --git a/src/daemon/core/tests.rs b/src/daemon/core/tests.rs index 635298b5e..48ae00642 100644 --- a/src/daemon/core/tests.rs +++ b/src/daemon/core/tests.rs @@ -46,9 +46,7 @@ fn daemon_lifecycle_wait_wakes_for_v2_maintenance_deadline() { .unwrap(), ) .unwrap(); - let readiness = - wait_for_daemon_lifecycle(&wake, None, Some(&deadline), &OverlayChildOwner::default()) - .unwrap(); + let readiness = wait_for_daemon_lifecycle(&wake, None, Some(&deadline), None).unwrap(); assert_eq!( readiness, DaemonLifecycleReadiness { @@ -67,13 +65,8 @@ fn action_admission_retry_uses_the_existing_v2_deadline_source() { daemon.action_admission_retry_at = Some(BootDeadline::from_nanos(1)); daemon.arm_v2_lifecycle_deadline().unwrap(); - let readiness = wait_for_daemon_lifecycle( - &wake, - None, - daemon.v2_deadline_source.as_ref(), - &OverlayChildOwner::default(), - ) - .unwrap(); + let readiness = + wait_for_daemon_lifecycle(&wake, None, daemon.v2_deadline_source.as_ref(), None).unwrap(); assert_eq!( readiness, DaemonLifecycleReadiness { @@ -105,7 +98,7 @@ fn listener_failure_invalidates_v1_readiness_and_runs_existing_cleanup() { ) .unwrap(); listener.inject_read_error(libc::EIO); - wait_for_daemon_lifecycle(&wake, None, None, &OverlayChildOwner::default()).unwrap(); + wait_for_daemon_lifecycle(&wake, None, None, None).unwrap(); let mut daemon = Daemon::new(None, false, None, None); daemon.protocol_mode = DaemonControlProtocolMode::rollback_compatibility(); @@ -156,69 +149,74 @@ fn light_draw_off_request_does_not_show_hidden_overlay() { assert_eq!(called.load(AtomicOrdering::SeqCst), 0); assert_eq!(daemon.test_state(), OverlayState::Hidden); - assert!(daemon.pending_toggle_request.is_none()); - assert!(daemon.pending_activation_token.is_none()); + assert!(daemon.pending_launch.is_none()); } #[test] fn visible_overlay_rejects_different_named_session_request() { - let runner: Arc = Arc::new(|_| Ok(())); - let mut daemon = Daemon::with_backend_runner(None, runner); - daemon.overlay_state = OverlayState::Visible; - daemon.active_named_session_file = - Some(std::path::PathBuf::from("/tmp/current.wayscriber-session")); - - let err = daemon - .process_single_toggle( - Some(DaemonToggleRequest { - session_file: Some(std::path::PathBuf::from("/tmp/other.wayscriber-session")), - ..Default::default() - }), - None, - false, - ) - .expect_err("different visible named target should be rejected"); + super::super::overlay::tests::with_visible_overlay( + Some(PathBuf::from("/tmp/current.wayscriber-session")), + false, + |daemon| { + let err = daemon + .process_single_toggle( + Some(DaemonToggleRequest { + session_file: Some(PathBuf::from("/tmp/other.wayscriber-session")), + ..Default::default() + }), + None, + false, + ) + .expect_err("different visible named target should be rejected"); - assert!( - format!("{err:#}").contains("cannot switch named session target while overlay is visible"), - "{err:#}" - ); - assert_eq!(daemon.test_state(), OverlayState::Visible); - assert_eq!( - daemon.active_named_session_file.as_deref(), - Some(std::path::Path::new("/tmp/current.wayscriber-session")) + assert!( + format!("{err:#}") + .contains("cannot switch named session target while overlay is visible"), + "{err:#}" + ); + assert_eq!(daemon.test_state(), OverlayState::Visible); + assert_eq!( + daemon.overlay.active_named_session_file(), + Some(std::path::Path::new("/tmp/current.wayscriber-session")) + ); + }, ); } #[test] fn visible_overlay_rejection_writes_daemon_toggle_error_response() { let temp = crate::test_temp::tempdir().expect("tempdir"); - let runner: Arc = Arc::new(|_| Ok(())); - let mut daemon = Daemon::with_backend_runner(None, runner); - daemon.overlay_state = OverlayState::Visible; - daemon.active_named_session_file = - Some(std::path::PathBuf::from("/tmp/current.wayscriber-session")); - let command = DaemonToggleCommand { - daemon_token: "daemon-token".into(), - request: DaemonToggleRequest { - session_file: Some(std::path::PathBuf::from("/tmp/other.wayscriber-session")), - ..Default::default() - }, - request_path: temp.path().join("request.json"), - response_path: temp.path().join("responses").join("request.json"), - }; - - let mut suppress_overlay_action_signal = false; - daemon.process_queued_toggle_command(command.clone(), &mut suppress_overlay_action_signal); + super::super::overlay::tests::with_visible_overlay( + Some(PathBuf::from("/tmp/current.wayscriber-session")), + false, + |daemon| { + let command = DaemonToggleCommand { + daemon_token: "daemon-token".into(), + request: DaemonToggleRequest { + session_file: Some(std::path::PathBuf::from("/tmp/other.wayscriber-session")), + ..Default::default() + }, + request_path: temp.path().join("request.json"), + response_path: temp.path().join("responses").join("request.json"), + }; + + let mut suppress_overlay_action_signal = false; + daemon.process_queued_toggle_command( + command.clone(), + &mut suppress_overlay_action_signal, + ); - let err = read_daemon_toggle_response(&command.response_path) - .expect_err("visible target mismatch should be written to response"); - assert!( - format!("{err:#}").contains("cannot switch named session target while overlay is visible"), - "{err:#}" + let err = read_daemon_toggle_response(&command.response_path) + .expect_err("visible target mismatch should be written to response"); + assert!( + format!("{err:#}") + .contains("cannot switch named session target while overlay is visible"), + "{err:#}" + ); + assert_eq!(daemon.test_state(), OverlayState::Visible); + assert!(!suppress_overlay_action_signal); + }, ); - assert_eq!(daemon.test_state(), OverlayState::Visible); - assert!(!suppress_overlay_action_signal); } #[test] @@ -300,69 +298,38 @@ fn typed_visibility_toggle_request_is_not_debounced() { #[cfg(unix)] #[test] fn duplicate_plain_toggle_after_slow_hide_is_debounced() { - let broker = crate::process_broker::start_for_runtime().unwrap(); - let mut daemon = Daemon::new(None, false, None, None); - let child = broker - .broker() - .spawn( - crate::process_broker::HelperKind::TestSleep, - crate::process_broker::HelperLifetime::OwnedChild, - std::ffi::OsStr::new("sleep"), - [std::ffi::OsStr::new("10")], - Vec::new(), - ) - .expect("spawn slow-terminating test process"); - let child_pid = child.id(); - assert_eq!(unsafe { libc::kill(child_pid as i32, libc::SIGSTOP) }, 0); - let mut stopped = false; - for _ in 0..20 { - let mut status = 0; - let result = unsafe { - libc::waitpid( - child_pid as i32, - &mut status, - libc::WNOHANG | libc::WUNTRACED, - ) - }; - if result == child_pid as i32 && libc::WIFSTOPPED(status) { - stopped = true; - break; - } - thread::sleep(Duration::from_millis(10)); - } - assert!(stopped, "test child should stop before hide starts"); - daemon.overlay_child.reserve().unwrap(); - daemon.overlay_child.start(child).unwrap(); - daemon.overlay_child.mark_committing().unwrap(); - daemon.overlay_child.mark_ready().unwrap(); - daemon - .overlay_active - .store(true, std::sync::atomic::Ordering::Release); - daemon.overlay_state = OverlayState::Visible; - - let hide_started = Instant::now(); - daemon - .process_single_toggle(Some(DaemonToggleRequest::default()), None, false) - .unwrap(); - assert!( - hide_started.elapsed() >= DUPLICATE_SHORTCUT_SUPPRESSION_WINDOW, - "test setup should keep hide slow enough to cross the debounce window" - ); - assert_eq!(daemon.test_state(), OverlayState::Hidden); - - let called = Arc::new(AtomicUsize::new(0)); - let called_clone = Arc::clone(&called); - daemon.backend_runner = Some(Arc::new(move |_| { - called_clone.fetch_add(1, AtomicOrdering::SeqCst); - Ok(()) - })); - - daemon - .process_single_toggle(Some(DaemonToggleRequest::default()), None, false) - .unwrap(); + super::super::overlay::tests::with_visible_overlay(None, true, |daemon| { + let hide_started = Instant::now(); + daemon + .process_single_toggle(Some(DaemonToggleRequest::default()), None, false) + .unwrap(); + assert!( + hide_started.elapsed() >= DUPLICATE_SHORTCUT_SUPPRESSION_WINDOW, + "test setup should keep hide slow enough to cross the debounce window" + ); + assert!( + hide_started.elapsed() >= Duration::from_millis(1900), + "forced hide must not shorten the two-second graceful shutdown policy" + ); + assert_eq!(daemon.test_state(), OverlayState::Hidden); + assert!(!daemon.overlay.active_flag().load(Ordering::Acquire)); + assert!(daemon.overlay.active_named_session_file().is_none()); + assert!(daemon.overlay.poll_fd().is_none()); + + let called = Arc::new(AtomicUsize::new(0)); + let called_clone = Arc::clone(&called); + daemon.backend_runner = Some(Arc::new(move |_| { + called_clone.fetch_add(1, AtomicOrdering::SeqCst); + Ok(()) + })); + + daemon + .process_single_toggle(Some(DaemonToggleRequest::default()), None, false) + .unwrap(); - assert_eq!(called.load(AtomicOrdering::SeqCst), 0); - assert_eq!(daemon.test_state(), OverlayState::Hidden); + assert_eq!(called.load(AtomicOrdering::SeqCst), 0); + assert_eq!(daemon.test_state(), OverlayState::Hidden); + }); } #[test] @@ -471,6 +438,55 @@ fn published_v2_runtime_drives_a_typed_request_to_terminal_response() { } } +#[test] +fn a_visible_v2_action_retains_only_the_pending_activation_token() { + super::super::overlay::tests::with_visible_overlay(None, false, |daemon| { + let owner = CommandOwner::open(&daemon.instance_token).unwrap(); + let journal = ActionJournal::open().unwrap(); + let mut runtime = + DaemonRuntimeRecordV2::current(ProtocolToken::generate().unwrap()).unwrap(); + runtime.v2_instance_token = daemon.instance_token.clone(); + super::super::protocol_v2::write_runtime_record_v2( + &crate::paths::daemon_pid_file(), + &runtime, + ) + .unwrap(); + let _daemon_lock = hold_daemon_lock(); + daemon.protocol_mode = DaemonControlProtocolMode::dark_harness(); + daemon.v2_command_owner = Some(owner); + daemon.v2_action_journal = Some(journal); + daemon.queue_overlay_launch( + Some(DaemonToggleRequest { + mode: Some("whiteboard".into()), + no_resume_session: true, + ..Default::default() + }), + Some("v2-retained-token".into()), + ); + + let request = DaemonToggleRequest { + overlay_action: Some(TrayAction::CaptureFull), + mode: Some("blackboard".into()), + no_resume_session: true, + ..Default::default() + }; + // Keep the real command lease open while the daemon authorizes and + // delivers the action. The fixture receives the wake but does not run + // capture UI or provide a terminal application receipt. + let _client = super::super::protocol_v2::ClientCommand::publish( + &super::super::protocol_v2::DaemonRequestV2::from(&request), + &daemon.instance_token, + ) + .unwrap(); + + daemon.process_v2_commands().unwrap(); + + assert_eq!(daemon.test_state(), OverlayState::Visible); + assert!(daemon.overlay.active_flag().load(AtomicOrdering::Acquire)); + super::super::overlay::tests::assert_token_only_pending_launch(daemon, "v2-retained-token"); + }); +} + #[test] fn failed_anonymous_action_admission_does_not_allow_the_tail_to_overtake() { let _env_guard = crate::test_env::lock(); @@ -602,8 +618,12 @@ fn hold_daemon_lock() -> std::fs::File { /// inside the backoff window after a spawn failure, so it attempts no start. fn daemon_in_spawn_backoff() -> Daemon { let mut daemon = Daemon::new(None, false, None, None); - daemon.overlay_spawn_failures = 1; - daemon.overlay_spawn_next_retry = Some(Instant::now() + Duration::from_secs(60)); + + // Use the production cap rather than injecting a private retry deadline. + for _ in 0..6 { + daemon.overlay.record_spawn_failure(); + } + daemon } @@ -653,7 +673,7 @@ fn typed_v2_requests_during_spawn_backoff_fail_without_an_effect() { } assert_eq!(daemon.test_state(), OverlayState::Hidden); - assert!(daemon.pending_toggle_request.is_none()); + assert!(daemon.pending_launch.is_none()); assert!( journal .claim_next(&token_text, |_, _| Ok(true)) @@ -709,25 +729,84 @@ fn legacy_action_during_spawn_backoff_fails_without_queueing_it() { .unwrap_err(); assert!(format!("{error:#}").contains("backing off"), "{error:#}"); - assert!(daemon.pending_toggle_request.is_none()); + assert!(daemon.pending_launch.is_none()); assert!(crate::tray_action::take_pending_actions().is_empty()); }); } +#[test] +fn a_failed_runner_does_not_replay_launch_options_or_leak_resume_override() { + let _env_guard = crate::test_env::lock(); + let previous_override = crate::runtime_session_override(); + crate::set_runtime_session_override(Some(true)); + + let calls = Arc::new(Mutex::new(Vec::new())); + let runner_calls = Arc::clone(&calls); + let runner: Arc = Arc::new(move |mode| { + let mut calls = runner_calls.lock().unwrap(); + let fail = calls.is_empty(); + calls.push((mode, crate::runtime_session_override())); + + if fail { + anyhow::bail!("runner failed before opening overlay"); + } + Ok(()) + }); + let mut daemon = Daemon::with_backend_runner(Some("transparent".into()), runner); + daemon.queue_overlay_launch( + Some(DaemonToggleRequest { + mode: Some("whiteboard".into()), + no_resume_session: true, + ..Default::default() + }), + Some("runner-token".into()), + ); + + let failed = daemon.show_overlay(); + let after_failure = crate::runtime_session_override(); + let token_retained_after_failure = daemon.pending_launch.is_some(); + let next = daemon.show_overlay(); + let after_success = crate::runtime_session_override(); + let token_retained_after_success = daemon.pending_launch.is_some(); + crate::set_runtime_session_override(previous_override); + + assert!(failed.unwrap_err().to_string().contains("runner failed")); + assert_eq!(next.unwrap(), ShowOutcome::Shown); + assert_eq!(daemon.test_state(), OverlayState::Hidden); + assert_eq!(after_failure, Some(true)); + assert_eq!(after_success, Some(true)); + assert!( + token_retained_after_failure, + "failed runner must keep token" + ); + assert!( + token_retained_after_success, + "successful runner must keep token" + ); + assert_eq!( + *calls.lock().unwrap(), + vec![ + (Some("whiteboard".into()), Some(false)), + (Some("transparent".into()), None), + ] + ); +} + #[test] fn a_deferred_overlay_start_does_not_keep_its_request_for_a_later_start() { let mut daemon = daemon_in_spawn_backoff(); - daemon.pending_toggle_request = Some(DaemonToggleRequest { - mode: Some("whiteboard".into()), - session_file: Some(PathBuf::from("/tmp/stale.wayscriber-session")), - ..Default::default() - }); - daemon.pending_activation_token = Some("stale-token".into()); + daemon.queue_overlay_launch( + Some(DaemonToggleRequest { + mode: Some("whiteboard".into()), + session_file: Some(PathBuf::from("/tmp/stale.wayscriber-session")), + ..Default::default() + }), + Some("stale-token".into()), + ); let outcome = daemon.show_overlay().unwrap(); assert!(matches!(outcome, ShowOutcome::BackingOff { .. })); assert_eq!(daemon.test_state(), OverlayState::Hidden); - assert!(daemon.pending_toggle_request.is_none()); - assert!(daemon.pending_activation_token.is_none()); + assert!(daemon.pending_launch.is_none()); } diff --git a/src/daemon/core/toggles.rs b/src/daemon/core/toggles.rs index 516f27bc8..9d4a00411 100644 --- a/src/daemon/core/toggles.rs +++ b/src/daemon/core/toggles.rs @@ -19,12 +19,12 @@ impl Daemon { let Some(requested) = request.and_then(|request| request.session_file.as_ref()) else { return Ok(()); }; - if self.overlay_state != OverlayState::Visible { + if self.overlay.state() != OverlayState::Visible { return Ok(()); } if self - .active_named_session_file - .as_ref() + .overlay + .active_named_session_file() .is_some_and(|active| named_session_paths_match(active, requested)) { return Ok(()); @@ -62,41 +62,38 @@ impl Daemon { return Ok(false); } } - if let Some(action) = request.as_ref().and_then(|request| request.overlay_action) { - self.pending_activation_token = activation_token; - self.pending_toggle_request = request.filter(|request| !request.is_empty()); - if self.overlay_state == OverlayState::Hidden + let action = request.as_ref().and_then(|request| request.overlay_action); + self.queue_overlay_launch( + request.filter(|request| !request.is_empty()), + activation_token, + ); + if let Some(action) = action { + if self.overlay.state() == OverlayState::Hidden && matches!(action, TrayAction::LightDrawOff) { - self.pending_activation_token = None; - self.pending_toggle_request = None; + self.pending_launch = None; return Ok(false); } - let was_hidden = self.overlay_state == OverlayState::Hidden; + let was_hidden = self.overlay.state() == OverlayState::Hidden; if was_hidden && let Some(retry_in) = self.overlay_start_backoff() { // Queued now, the action would run whenever the overlay next // starts, long after the caller was told it had happened. - self.pending_activation_token = None; - self.pending_toggle_request = None; + self.pending_launch = None; return Err(anyhow::anyhow!(overlay_start_backoff_reason(retry_in))); } self.dispatch_overlay_action(action, !suppress_overlay_action_signal)?; - if self.overlay_state == OverlayState::Hidden { + if self.overlay.state() == OverlayState::Hidden { self.show_overlay()?.require_shown()?; return Ok(was_hidden); } else { - self.pending_activation_token = None; - self.pending_toggle_request = None; + self.pending_launch = None; } return Ok(false); } - self.pending_activation_token = activation_token; - self.pending_toggle_request = request.filter(|request| !request.is_empty()); if let Err(err) = self.toggle_overlay() { - self.pending_activation_token = None; - self.pending_toggle_request = None; + self.pending_launch = None; return Err(err); } if plain_visibility_toggle_requested { @@ -142,10 +139,10 @@ impl Daemon { ) -> Result<()> { let action_path = crate::tray_action::queue_action(action)?; - if signal_visible_overlay && self.overlay_state == OverlayState::Visible { + if signal_visible_overlay && self.overlay.state() == OverlayState::Visible { #[cfg(unix)] { - if let Err(error) = self.overlay_child.signal(libc::SIGUSR2) { + if let Err(error) = self.overlay.signal(libc::SIGUSR2) { warn!( "Failed to signal overlay process for action {}: {error:#}", action.as_str(), @@ -167,10 +164,10 @@ impl Daemon { } pub(super) fn signal_overlay_action_ready(&self, action: TrayAction) -> Result<()> { - if self.overlay_state != OverlayState::Visible { + if self.overlay.state() != OverlayState::Visible { return Ok(()); } - self.overlay_child.signal(libc::SIGUSR2).with_context(|| { + self.overlay.signal(libc::SIGUSR2).with_context(|| { format!( "failed to notify overlay about committed v2 action {}", action.as_str() diff --git a/src/daemon/overlay/launch.rs b/src/daemon/overlay/launch.rs new file mode 100644 index 000000000..d3b3d631a --- /dev/null +++ b/src/daemon/overlay/launch.rs @@ -0,0 +1,171 @@ +use std::ffi::OsString; +use std::path::{Path, PathBuf}; + +use crate::daemon::control::DaemonToggleRequest; +use crate::env_vars::{DESKTOP_STARTUP_ID_ENV, NO_DETACH_ENV, XDG_ACTIVATION_TOKEN_ENV}; + +pub(in crate::daemon) struct OverlayLaunchRequest { + mode: Option, + named_session_file: Option, + freeze: bool, + exit_after_capture: bool, + no_exit_after_capture: bool, + session_resume_override: Option, + activation_token: Option, +} + +impl OverlayLaunchRequest { + pub(super) fn new( + request: Option, + activation_token: Option, + mode: Option<&str>, + named_session_file: Option<&Path>, + freeze: bool, + ) -> Self { + let request = request.unwrap_or_default(); + let session_resume_override = request.session_resume_override(); + + Self { + mode: request.mode.or_else(|| mode.map(str::to_owned)), + named_session_file: request + .session_file + .or_else(|| named_session_file.map(Path::to_path_buf)), + freeze: request.freeze || freeze, + exit_after_capture: request.exit_after_capture, + no_exit_after_capture: request.no_exit_after_capture, + session_resume_override, + activation_token, + } + } + + pub(super) fn mode(&self) -> Option<&str> { + self.mode.as_deref() + } + + pub(super) fn named_session_file(&self) -> Option<&Path> { + self.named_session_file.as_deref() + } + + pub(super) fn activation_token(&self) -> Option<&str> { + self.activation_token.as_deref() + } + + pub(super) fn into_activation_token(self) -> Option { + self.activation_token + } + + pub(super) fn session_resume_override(&self, default: Option) -> Option { + self.session_resume_override.or(default) + } +} + +#[derive(Debug, PartialEq, Eq)] +pub(super) struct OverlayLaunch { + pub(super) arguments: Vec, + pub(super) environment: Vec<(OsString, Option)>, +} + +pub(super) fn build_overlay_launch( + request: &OverlayLaunchRequest, + resume_default: Option, + generation: Option<&str>, +) -> OverlayLaunch { + let mut arguments = vec![OsString::from("--active")]; + if request.freeze { + arguments.push("--freeze".into()); + } + if request.exit_after_capture { + arguments.push("--exit-after-capture".into()); + } else if request.no_exit_after_capture { + arguments.push("--no-exit-after-capture".into()); + } + + // Daemon children are already backgrounded and tracked; do not detach again. + let mut environment = vec![(OsString::from(NO_DETACH_ENV), Some("1".into()))]; + if let Some(generation) = generation { + environment.push(( + crate::env_vars::OVERLAY_CHILD_GENERATION_ENV.into(), + Some(generation.into()), + )); + } + if let Some(token) = request.activation_token() { + environment.push((XDG_ACTIVATION_TOKEN_ENV.into(), Some(token.into()))); + environment.push((DESKTOP_STARTUP_ID_ENV.into(), Some(token.into()))); + } else { + environment.push((XDG_ACTIVATION_TOKEN_ENV.into(), None)); + environment.push((DESKTOP_STARTUP_ID_ENV.into(), None)); + } + environment.push(( + crate::RESUME_SESSION_ENV.into(), + request + .session_resume_override(resume_default) + .map(|enabled| if enabled { "on".into() } else { "off".into() }), + )); + + if let Some(mode) = request.mode() { + arguments.push("--mode".into()); + arguments.push(mode.into()); + } + if let Some(path) = request.named_session_file() { + arguments.push("--session-file".into()); + arguments.push(path.as_os_str().into()); + } + + OverlayLaunch { + arguments, + environment, + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn launch_environment_preserves_token_removal_generation_and_resume_precedence() { + for (requested, default, expected_resume) in [ + (None, None, None), + (None, Some(true), Some("on")), + (None, Some(false), Some("off")), + (Some(true), None, Some("on")), + (Some(false), None, Some("off")), + (Some(true), Some(false), Some("on")), + (Some(false), Some(true), Some("off")), + ] { + for token in [None, Some("activation-token")] { + for generation in [None, Some("child-generation")] { + let request = OverlayLaunchRequest::new( + requested.map(|enabled| DaemonToggleRequest { + resume_session: enabled, + no_resume_session: !enabled, + ..Default::default() + }), + token.map(str::to_owned), + None, + None, + false, + ); + + let launch = build_overlay_launch(&request, default, generation); + + let mut expected = vec![(NO_DETACH_ENV.into(), Some("1".into()))]; + if let Some(generation) = generation { + expected.push(( + crate::env_vars::OVERLAY_CHILD_GENERATION_ENV.into(), + Some(generation.into()), + )); + } + expected.extend([ + (XDG_ACTIVATION_TOKEN_ENV.into(), token.map(OsString::from)), + (DESKTOP_STARTUP_ID_ENV.into(), token.map(OsString::from)), + ( + crate::RESUME_SESSION_ENV.into(), + expected_resume.map(OsString::from), + ), + ]); + assert_eq!(launch.environment, expected); + } + } + } + } +} diff --git a/src/daemon/overlay/lifecycle.rs b/src/daemon/overlay/lifecycle.rs new file mode 100644 index 000000000..583680621 --- /dev/null +++ b/src/daemon/overlay/lifecycle.rs @@ -0,0 +1,255 @@ +use std::os::fd::{AsRawFd, BorrowedFd}; +use std::path::{Path, PathBuf}; +use std::sync::Arc; +use std::sync::atomic::{AtomicBool, AtomicU8, Ordering}; +use std::time::{Duration, Instant}; + +use anyhow::{Context, Result}; +use log::{debug, info, warn}; + +use super::super::protocol_v2::OverlayChildOwner; +use super::super::types::{BackendRunner, OverlaySpawnCandidate, OverlayState}; +use super::launch::{OverlayLaunchRequest, build_overlay_launch}; + +mod stop; + +const SPAWN_BACKOFF_BASE: Duration = Duration::from_secs(1); +const SPAWN_BACKOFF_MAX: Duration = Duration::from_secs(30); + +pub(super) enum OverlayStartFailure { + BeforeAttempt(anyhow::Error), + Attempt(anyhow::Error), +} + +pub(in crate::daemon) struct OverlayLifecycle { + state: OverlayState, + child: OverlayChildOwner, + active: Arc, + active_named_session_file: Option, + spawn_failures: u32, + next_spawn_retry: Option, + backoff_logged: bool, +} + +impl Default for OverlayLifecycle { + fn default() -> Self { + Self { + state: OverlayState::Hidden, + child: OverlayChildOwner::default(), + active: Arc::new(AtomicBool::new(false)), + active_named_session_file: None, + spawn_failures: 0, + next_spawn_retry: None, + backoff_logged: false, + } + } +} + +impl OverlayLifecycle { + pub(in crate::daemon) fn state(&self) -> OverlayState { + self.state + } + + pub(in crate::daemon) fn active_flag(&self) -> Arc { + Arc::clone(&self.active) + } + + pub(in crate::daemon) fn poll_fd(&self) -> Option> { + self.child.poll_fd() + } + + pub(in crate::daemon) fn signal(&self, signal: i32) -> Result<()> { + self.child.signal(signal) + } + + pub(in crate::daemon) fn active_named_session_file(&self) -> Option<&Path> { + self.active_named_session_file.as_deref() + } + + pub(super) fn start( + &mut self, + request: &OverlayLaunchRequest, + candidate: &OverlaySpawnCandidate, + resume_override: &AtomicU8, + daemon_token: &str, + ) -> std::result::Result { + self.child + .reserve() + .map_err(OverlayStartFailure::BeforeAttempt)?; + + debug!( + "Attempting overlay spawn via {} ({})", + candidate.source, + candidate.program.to_string_lossy() + ); + + let launch = build_overlay_launch( + request, + crate::decode_session_override(resume_override.load(Ordering::Acquire)), + self.child.generation(), + ); + + let attempt = (|| -> Result { + let daemon_watchdog = super::super::protocol_v2::open_daemon_watchdog()?; + let child = crate::process_broker::current()?.spawn_with_watchdog( + crate::process_broker::HelperKind::Overlay, + crate::process_broker::HelperLifetime::OwnedChild, + &candidate.program, + &launch.arguments, + launch.environment, + daemon_watchdog.as_raw_fd(), + )?; + self.child.start(child)?; + self.child + .wait_until_ready(Duration::from_secs(5), daemon_token)?; + self.mark_shown(request.named_session_file().map(Path::to_path_buf)) + })(); + + match attempt { + Ok(pid) => Ok(pid), + Err(error) => { + self.abort_start(); + Err(OverlayStartFailure::Attempt(error)) + } + } + } + + /// Retry cleanup after exhausted candidates if the last abort could not reap. + pub(super) fn abort_start(&mut self) { + self.child.abort_reservation(); + } + + fn mark_shown(&mut self, named_session_file: Option) -> Result { + let pid = self + .child + .display_pid() + .context("cannot show an overlay without an owned child")?; + + self.active.store(true, Ordering::Release); + self.state = OverlayState::Visible; + self.active_named_session_file = named_session_file; + + Ok(pid) + } + + pub(super) fn mark_hidden(&mut self) { + debug_assert!(self.child.display_pid().is_none()); + self.state = OverlayState::Hidden; + self.active_named_session_file = None; + self.active.store(false, Ordering::Release); + } + + // The existing in-process backend does not own a broker child or advertise + // an active child to the tray/update watcher. Keep that distinction explicit. + pub(super) fn run_backend( + &mut self, + runner: &BackendRunner, + request: &OverlayLaunchRequest, + resume_default: Option, + ) -> Result<()> { + self.state = OverlayState::Visible; + self.active_named_session_file = request.named_session_file().map(Path::to_path_buf); + info!("Overlay state set to Visible"); + let previous_override = crate::runtime_session_override(); + crate::set_runtime_session_override(request.session_resume_override(resume_default)); + + let result = runner(request.mode().map(str::to_owned)); + + crate::set_runtime_session_override(previous_override); + self.state = OverlayState::Hidden; + self.active_named_session_file = None; + info!("Overlay closed, back to daemon mode"); + + result + } + + fn spawn_backoff_duration(&self) -> Duration { + let failures = self.spawn_failures.max(1); + let shift = failures.saturating_sub(1).min(5); + let base = SPAWN_BACKOFF_BASE.as_secs().max(1); + let secs = base.saturating_mul(1_u64 << shift); + Duration::from_secs(secs.min(SPAWN_BACKOFF_MAX.as_secs())) + } + + pub(super) fn spawn_backoff_remaining(&mut self) -> Option { + if let Some(next_retry) = self.next_spawn_retry { + let now = Instant::now(); + if now < next_retry { + let remaining = next_retry.saturating_duration_since(now); + if !self.backoff_logged { + warn!( + "Overlay spawn backoff active (retry in {}s)", + remaining.as_secs().max(1) + ); + self.backoff_logged = true; + } + return Some(remaining); + } + } + + self.backoff_logged = false; + None + } + + pub(in crate::daemon) fn record_spawn_failure(&mut self) -> Duration { + self.spawn_failures = self.spawn_failures.saturating_add(1); + let backoff = self.spawn_backoff_duration(); + self.next_spawn_retry = Some(Instant::now() + backoff); + self.backoff_logged = false; + + backoff + } + + #[cfg(feature = "tray")] + pub(super) fn next_spawn_retry(&self) -> Option { + self.next_spawn_retry + } + + pub(super) fn clear_spawn_backoff(&mut self) { + self.spawn_failures = 0; + self.next_spawn_retry = None; + self.backoff_logged = false; + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn backoff_duration_grows_and_caps() { + let mut lifecycle = OverlayLifecycle::default(); + for expected in [1, 2, 4, 8, 16, 30, 30] { + assert_eq!( + lifecycle.record_spawn_failure(), + Duration::from_secs(expected) + ); + } + + lifecycle.clear_spawn_backoff(); + assert!(lifecycle.spawn_backoff_remaining().is_none()); + assert_eq!(lifecycle.record_spawn_failure(), Duration::from_secs(1)); + } + + #[test] + fn overlay_spawn_backoff_honors_retry_window() { + let mut lifecycle = OverlayLifecycle { + next_spawn_retry: Some(Instant::now() + Duration::from_secs(2)), + ..Default::default() + }; + assert!(lifecycle.spawn_backoff_remaining().is_some()); + assert!(lifecycle.backoff_logged); + + lifecycle.next_spawn_retry = Some(Instant::now() - Duration::from_secs(1)); + assert!(lifecycle.spawn_backoff_remaining().is_none()); + assert!(!lifecycle.backoff_logged); + } + + #[test] + fn showing_requires_an_owned_child() { + let mut lifecycle = OverlayLifecycle::default(); + assert!(lifecycle.mark_shown(None).is_err()); + assert_eq!(lifecycle.state(), OverlayState::Hidden); + assert!(!lifecycle.active_flag().load(Ordering::Acquire)); + } +} diff --git a/src/daemon/overlay/lifecycle/stop.rs b/src/daemon/overlay/lifecycle/stop.rs new file mode 100644 index 000000000..d14976fcc --- /dev/null +++ b/src/daemon/overlay/lifecycle/stop.rs @@ -0,0 +1,109 @@ +use anyhow::{Context, Result}; +use log::{info, warn}; +use std::os::fd::AsFd; +use std::sync::atomic::Ordering; +use std::thread; +use std::time::{Duration, Instant}; + +use super::OverlayLifecycle; +use crate::daemon::protocol_v2::{BootClock, open_overlay_pidfd, wait_for_pidfd_exit}; + +impl OverlayLifecycle { + fn terminate(&mut self) -> Result<()> { + if let Some(pid) = self.child.display_pid() { + let stop_started = Instant::now(); + let timeout = Duration::from_secs(2); + info!( + "Stopping overlay process (pid {}, graceful_timeout={:?})", + pid, timeout + ); + if let Err(err) = self.child.begin_stop() { + warn!("Failed to signal overlay process: {err:#}"); + } + + // Wait on the child's pidfd rather than waking every 50ms to poll + // it. A pidfd becomes readable exactly when its process exits, so + // prompt exits are observed immediately and slow exits cost no + // periodic wakeups. This termination path is still synchronous: + // the daemon event loop remains occupied until the child exits or + // the graceful timeout expires. + let exit_watch = open_overlay_pidfd(pid).ok(); + let deadline = BootClock::now()?.checked_add(timeout)?; + loop { + match self.child.try_wait() { + Ok(Some(status)) => { + info!( + "Overlay process exited with status {:?} after {:?}", + status, + stop_started.elapsed() + ); + break; + } + Ok(None) => { + if BootClock::now()? >= deadline { + warn!( + "Overlay process did not exit after {:?}, sending SIGKILL", + stop_started.elapsed() + ); + let status = self + .child + .force_kill_and_wait() + .context("lost broker ownership while forcing overlay shutdown")?; + warn!( + "Overlay process killed with status {:?} after {:?}", + status, + stop_started.elapsed() + ); + break; + } + // Without a pidfd (the child raced us to exit, or the + // open failed) fall back to the original pacing. + match exit_watch.as_ref() { + Some(fd) => { + let now = BootClock::now()?.as_nanos(); + let remaining = + Duration::from_nanos(deadline.as_nanos().saturating_sub(now)); + let _ = wait_for_pidfd_exit(fd.as_fd(), remaining); + } + None => thread::sleep(Duration::from_millis(50)), + } + } + Err(err) => { + let forced = self.child.force_kill_and_wait(); + return match forced { + Ok(_) => Err(err).context( + "broker ownership failed while querying overlay; child was forced down", + ), + Err(force_error) => Err(anyhow::anyhow!( + "broker ownership failed while querying overlay: {err:#}; \ + forced termination also failed: {force_error:#}" + )), + }; + } + } + } + } + + self.active.store(false, Ordering::Release); + self.active_named_session_file = None; + Ok(()) + } + + pub(in crate::daemon::overlay) fn hide(&mut self) -> Result<()> { + self.terminate()?; + self.mark_hidden(); + Ok(()) + } + + pub(in crate::daemon::overlay) fn poll_exit(&mut self) -> Result<()> { + match self.child.try_wait() { + Ok(Some(status)) => { + info!("Overlay process exited with status {:?}", status); + self.mark_hidden(); + } + Ok(None) => {} + Err(err) => return Err(err).context("lost broker ownership of overlay child"), + } + Ok(()) + } +} diff --git a/src/daemon/overlay/mod.rs b/src/daemon/overlay/mod.rs index 9036ae81e..e6de2b969 100644 --- a/src/daemon/overlay/mod.rs +++ b/src/daemon/overlay/mod.rs @@ -3,13 +3,19 @@ use std::time::Duration; use anyhow::{Result, anyhow}; use log::{debug, info}; -use crate::{runtime_session_override, set_runtime_session_override}; - use super::core::Daemon; -use super::types::OverlayState; +use super::types::{OverlaySpawnCandidate, OverlayState}; +use crate::daemon::control::DaemonToggleRequest; +pub(super) mod launch; +pub(super) mod lifecycle; mod process; mod spawn; +#[cfg(test)] +pub(super) mod tests; + +use launch::OverlayLaunchRequest; +use lifecycle::OverlayStartFailure; /// What [`Daemon::show_overlay`] did with a request to show the overlay. #[must_use] @@ -22,8 +28,7 @@ pub(super) enum ShowOutcome { } impl ShowOutcome { - /// The outcome as a result, for a caller that must report a deferred - /// start to whoever asked for the overlay rather than claim it happened. + /// Turn a deferred start into an error rather than claiming it happened. pub(super) fn require_shown(self) -> Result<()> { match self { Self::Shown => Ok(()), @@ -41,9 +46,9 @@ pub(super) fn overlay_start_backoff_reason(retry_in: Duration) -> String { } impl Daemon { - /// Toggle overlay visibility + /// Toggle overlay visibility. pub(super) fn toggle_overlay(&mut self) -> Result<()> { - match self.overlay_state { + match self.overlay.state() { OverlayState::Hidden => { info!("Showing overlay"); self.show_overlay()?.require_shown()?; @@ -56,92 +61,134 @@ impl Daemon { Ok(()) } - /// Show overlay (create layer surface and enter drawing mode) - /// - /// A start attempt consumes the pending request: once the attempt has - /// started the overlay, been held back by the spawn backoff, or failed, - /// the request is gone, so a later start that nobody asked to shape - /// cannot inherit its mode or session file. + /// Each hidden start consumes one owned request. Backoff and exhausted + /// candidates drop it; preparation errors retain only the activation token. pub(super) fn show_overlay(&mut self) -> Result { - if self.overlay_state == OverlayState::Visible { + if self.overlay.state() == OverlayState::Visible { debug!("Overlay already visible"); return Ok(ShowOutcome::Shown); } + let request = self.take_pending_launch(); if let Some(runner) = self.backend_runner.clone() { - self.overlay_state = OverlayState::Visible; - self.active_named_session_file = self.effective_named_session_file(); - info!("Overlay state set to Visible"); self.clear_overlay_spawn_error(); - let previous_override = runtime_session_override(); - let request_override = self - .pending_toggle_request - .as_ref() - .and_then(|request| request.session_resume_override()); - set_runtime_session_override( - request_override.or_else(|| self.session_resume_override()), - ); - let requested_mode = self - .pending_toggle_request - .as_ref() - .and_then(|request| request.mode.clone()) - .or_else(|| self.initial_mode.clone()); - let result = runner(requested_mode); - set_runtime_session_override(previous_override); - self.pending_toggle_request = None; - self.active_named_session_file = None; - self.overlay_state = OverlayState::Hidden; - info!("Overlay closed, back to daemon mode"); + let resume_default = self.session_resume_override(); + + let result = self + .overlay + .run_backend(runner.as_ref(), &request, resume_default); + + // Preserve the in-process backend's existing token retention, on + // both success and failure, without retaining any launch options. + self.retain_launch_token(request); + return result.map(|()| ShowOutcome::Shown); } - if let Some(retry_in) = self.overlay_spawn_backoff_remaining() { - self.pending_toggle_request = None; - self.pending_activation_token = None; + if let Some(retry_in) = self.overlay.spawn_backoff_remaining() { return Ok(ShowOutcome::BackingOff { retry_in }); } - let spawned = self.spawn_overlay_process(); - self.pending_toggle_request = None; - if let Err(err) = spawned { - self.record_overlay_spawn_failure(err.to_string()); - return Err(err); + let candidates = self.overlay_spawn_candidates(); + self.start_launch(request, &candidates) + } + + fn start_launch( + &mut self, + request: OverlayLaunchRequest, + candidates: &[OverlaySpawnCandidate], + ) -> Result { + match self.spawn_overlay_process(&request, candidates) { + Ok(()) => { + self.clear_overlay_spawn_error(); + Ok(ShowOutcome::Shown) + } + Err(failure) => { + let error = match failure { + OverlayStartFailure::BeforeAttempt(error) => { + self.retain_launch_token(request); + error + } + OverlayStartFailure::Attempt(error) => error, + }; + self.record_overlay_spawn_failure(error.to_string()); + Err(error) + } } + } - self.clear_overlay_spawn_error(); - Ok(ShowOutcome::Shown) + fn overlay_launch_request( + &self, + request: Option, + activation_token: Option, + ) -> OverlayLaunchRequest { + OverlayLaunchRequest::new( + request, + activation_token, + self.initial_mode.as_deref(), + self.initial_named_session_file.as_deref(), + self.freeze_on_show, + ) + } + + pub(super) fn queue_overlay_launch( + &mut self, + request: Option, + activation_token: Option, + ) { + self.pending_launch = Some(self.overlay_launch_request(request, activation_token)); + } + + pub(super) fn replace_pending_launch_request(&mut self, request: DaemonToggleRequest) { + let token = self + .pending_launch + .take() + .and_then(OverlayLaunchRequest::into_activation_token); + self.queue_overlay_launch(Some(request), token); + } + + fn take_pending_launch(&mut self) -> OverlayLaunchRequest { + self.pending_launch + .take() + .unwrap_or_else(|| self.overlay_launch_request(None, None)) + } + + fn retain_launch_token(&mut self, request: OverlayLaunchRequest) { + self.pending_launch = request + .into_activation_token() + .map(|token| self.overlay_launch_request(None, Some(token))); + } + + pub(super) fn discard_pending_launch_options(&mut self) { + if let Some(request) = self.pending_launch.take() { + self.retain_launch_token(request); + } } - /// How long the spawn backoff still holds back a start of the hidden - /// overlay, checked before a request commits to one. An internal runner - /// never backs off. + /// Internal runners never back off; child starts use the lifecycle policy. pub(super) fn overlay_start_backoff(&mut self) -> Option { if self.backend_runner.is_some() { return None; } - self.overlay_spawn_backoff_remaining() + self.overlay.spawn_backoff_remaining() } - /// Hide overlay (destroy layer surface, return to hidden state) + /// Hide overlay (destroy layer surface, return to hidden state). pub(super) fn hide_overlay(&mut self) -> Result<()> { - if self.overlay_state == OverlayState::Hidden { + if self.overlay.state() == OverlayState::Hidden { debug!("Overlay already hidden"); return Ok(()); } if self.backend_runner.is_some() { - // Internal runner does not keep additional state to tear down debug!("Internal backend runner hidden"); - self.pending_toggle_request = None; - self.active_named_session_file = None; - self.overlay_state = OverlayState::Hidden; + self.discard_pending_launch_options(); + self.overlay.mark_hidden(); return Ok(()); } - self.terminate_overlay_process()?; - self.pending_toggle_request = None; - self.active_named_session_file = None; - self.overlay_state = OverlayState::Hidden; + self.overlay.hide()?; + self.discard_pending_launch_options(); Ok(()) } } diff --git a/src/daemon/overlay/process.rs b/src/daemon/overlay/process.rs index 28a2f0831..4e7df083d 100644 --- a/src/daemon/overlay/process.rs +++ b/src/daemon/overlay/process.rs @@ -1,113 +1,13 @@ -use anyhow::{Context, Result}; -use log::{info, warn}; -use std::os::fd::AsFd; -use std::thread; -use std::time::{Duration, Instant}; +use anyhow::Result; use super::super::core::Daemon; -use super::super::types::OverlayState; impl Daemon { - pub(super) fn terminate_overlay_process(&mut self) -> Result<()> { - if let Some(pid) = self.overlay_child.display_pid() { - let stop_started = Instant::now(); - let timeout = Duration::from_secs(2); - info!( - "Stopping overlay process (pid {}, graceful_timeout={:?})", - pid, timeout - ); - if let Err(err) = self.overlay_child.begin_stop() { - warn!("Failed to signal overlay process: {err:#}"); - } - - // Wait on the child's pidfd rather than waking every 50ms to poll - // it. A pidfd becomes readable exactly when its process exits, so - // prompt exits are observed immediately and slow exits cost no - // periodic wakeups. This termination path is still synchronous: - // the daemon event loop remains occupied until the child exits or - // the graceful timeout expires. - let exit_watch = super::super::protocol_v2::open_overlay_pidfd(pid).ok(); - let deadline = super::super::protocol_v2::BootClock::now()?.checked_add(timeout)?; - loop { - match self.overlay_child.try_wait() { - Ok(Some(status)) => { - info!( - "Overlay process exited with status {:?} after {:?}", - status, - stop_started.elapsed() - ); - break; - } - Ok(None) => { - if super::super::protocol_v2::BootClock::now()? >= deadline { - warn!( - "Overlay process did not exit after {:?}, sending SIGKILL", - stop_started.elapsed() - ); - let status = self - .overlay_child - .force_kill_and_wait() - .context("lost broker ownership while forcing overlay shutdown")?; - warn!( - "Overlay process killed with status {:?} after {:?}", - status, - stop_started.elapsed() - ); - break; - } - // Without a pidfd (the child raced us to exit, or the - // open failed) fall back to the original pacing. - match exit_watch.as_ref() { - Some(fd) => { - let now = super::super::protocol_v2::BootClock::now()?.as_nanos(); - let remaining = - Duration::from_nanos(deadline.as_nanos().saturating_sub(now)); - let _ = super::super::protocol_v2::wait_for_pidfd_exit( - fd.as_fd(), - remaining, - ); - } - None => thread::sleep(Duration::from_millis(50)), - } - } - Err(err) => { - let forced = self.overlay_child.force_kill_and_wait(); - return match forced { - Ok(_) => Err(err).context( - "broker ownership failed while querying overlay; child was forced down", - ), - Err(force_error) => Err(anyhow::anyhow!( - "broker ownership failed while querying overlay: {err:#}; forced termination also failed: {force_error:#}" - )), - }; - } - } - } - } - self.overlay_active - .store(false, std::sync::atomic::Ordering::Release); - self.active_named_session_file = None; - Ok(()) - } - pub(in crate::daemon) fn update_overlay_process_state(&mut self) -> Result<()> { if self.backend_runner.is_some() { return Ok(()); } - match self.overlay_child.try_wait() { - Ok(Some(status)) => { - info!("Overlay process exited with status {:?}", status); - self.overlay_state = OverlayState::Hidden; - self.overlay_active - .store(false, std::sync::atomic::Ordering::Release); - self.active_named_session_file = None; - } - Ok(None) => {} - Err(err) => { - return Err(err).context("lost broker ownership of overlay child"); - } - } - Ok(()) + self.overlay.poll_exit() } } diff --git a/src/daemon/overlay/spawn.rs b/src/daemon/overlay/spawn.rs index 6e2f62882..3278e985b 100644 --- a/src/daemon/overlay/spawn.rs +++ b/src/daemon/overlay/spawn.rs @@ -1,63 +1,22 @@ -use anyhow::{Result, anyhow}; -use log::{debug, info, warn}; +use anyhow::anyhow; +use log::{info, warn}; use std::collections::HashSet; use std::env; use std::ffi::OsString; -use std::os::fd::AsRawFd; -use std::time::{Duration, Instant}; -use crate::env_vars::{DESKTOP_STARTUP_ID_ENV, NO_DETACH_ENV, PATH_ENV, XDG_ACTIVATION_TOKEN_ENV}; +use crate::env_vars::PATH_ENV; + +use super::launch::OverlayLaunchRequest; +use super::lifecycle::OverlayStartFailure; use super::super::core::Daemon; +use super::super::types::OverlaySpawnCandidate; #[cfg(feature = "tray")] use super::super::types::OverlaySpawnErrorInfo; -use super::super::types::{OverlaySpawnCandidate, OverlayState}; - -const OVERLAY_SPAWN_BACKOFF_BASE: Duration = Duration::from_secs(1); -const OVERLAY_SPAWN_BACKOFF_MAX: Duration = Duration::from_secs(30); - -#[derive(Debug, PartialEq, Eq)] -struct OverlayLaunch { - arguments: Vec, - environment: Vec<(OsString, Option)>, -} impl Daemon { - fn overlay_spawn_backoff_duration(&self) -> Duration { - let failures = self.overlay_spawn_failures.max(1); - let shift = failures.saturating_sub(1).min(5); - let base = OVERLAY_SPAWN_BACKOFF_BASE.as_secs().max(1); - let secs = base.saturating_mul(1_u64 << shift); - Duration::from_secs(secs.min(OVERLAY_SPAWN_BACKOFF_MAX.as_secs())) - } - - /// The time left before another overlay spawn may be tried, or `None` - /// when no backoff is holding it back. - pub(super) fn overlay_spawn_backoff_remaining(&mut self) -> Option { - if let Some(next_retry) = self.overlay_spawn_next_retry { - let now = Instant::now(); - if now < next_retry { - let remaining = next_retry.saturating_duration_since(now); - if !self.overlay_spawn_backoff_logged { - warn!( - "Overlay spawn backoff active (retry in {}s)", - remaining.as_secs().max(1) - ); - self.overlay_spawn_backoff_logged = true; - } - return Some(remaining); - } - } - self.overlay_spawn_backoff_logged = false; - None - } - pub(super) fn record_overlay_spawn_failure(&mut self, message: String) { - self.overlay_spawn_failures = self.overlay_spawn_failures.saturating_add(1); - let backoff = self.overlay_spawn_backoff_duration(); - let next_retry_at = Instant::now() + backoff; - self.overlay_spawn_next_retry = Some(next_retry_at); - self.overlay_spawn_backoff_logged = false; + let backoff = self.overlay.record_spawn_failure(); warn!( "Failed to spawn overlay process: {} (retry in {}s)", message, @@ -67,19 +26,17 @@ impl Daemon { self.tray_status .set_overlay_error(Some(OverlaySpawnErrorInfo { message, - next_retry_at: Some(next_retry_at), + next_retry_at: self.overlay.next_spawn_retry(), })); } pub(super) fn clear_overlay_spawn_error(&mut self) { - self.overlay_spawn_failures = 0; - self.overlay_spawn_next_retry = None; - self.overlay_spawn_backoff_logged = false; + self.overlay.clear_spawn_backoff(); #[cfg(feature = "tray")] self.tray_status.set_overlay_error(None); } - fn overlay_spawn_candidates(&self) -> Vec { + pub(super) fn overlay_spawn_candidates(&self) -> Vec { let mut candidates = Vec::new(); let mut seen = HashSet::::new(); @@ -133,134 +90,94 @@ impl Daemon { } } - fn build_overlay_launch(&self) -> OverlayLaunch { - let mut arguments = vec![OsString::from("--active")]; - let request = self.pending_toggle_request.as_ref(); - if request.is_some_and(|request| request.freeze) || self.freeze_on_show { - arguments.push("--freeze".into()); - } - if request.is_some_and(|request| request.exit_after_capture) { - arguments.push("--exit-after-capture".into()); - } else if request.is_some_and(|request| request.no_exit_after_capture) { - arguments.push("--no-exit-after-capture".into()); - } - // Overlay children launched by daemon are already backgrounded and tracked. - // Prevent `--active` from spawning another detached grandchild process. - let mut environment = vec![(OsString::from(NO_DETACH_ENV), Some("1".into()))]; - if let Some(generation) = self.overlay_child.generation() { - environment.push(( - crate::env_vars::OVERLAY_CHILD_GENERATION_ENV.into(), - Some(generation.into()), - )); - } - if let Some(token) = self.pending_activation_token.as_deref() { - environment.push((XDG_ACTIVATION_TOKEN_ENV.into(), Some(token.into()))); - environment.push((DESKTOP_STARTUP_ID_ENV.into(), Some(token.into()))); - } else { - environment.push((XDG_ACTIVATION_TOKEN_ENV.into(), None)); - environment.push((DESKTOP_STARTUP_ID_ENV.into(), None)); - } - if let Some(request_override) = - request.and_then(|request| request.session_resume_override()) - { - match request_override { - true => environment.push((crate::RESUME_SESSION_ENV.into(), Some("on".into()))), - false => environment.push((crate::RESUME_SESSION_ENV.into(), Some("off".into()))), - } - } else { - environment.push(( - crate::RESUME_SESSION_ENV.into(), - self.session_resume_override() - .map(|enabled| if enabled { "on".into() } else { "off".into() }), - )); - } - if let Some(mode) = request - .and_then(|request| request.mode.as_ref()) - .or(self.initial_mode.as_ref()) - { - arguments.push("--mode".into()); - arguments.push(mode.into()); - } - if let Some(path) = self.effective_named_session_file() { - arguments.push("--session-file".into()); - arguments.push(path.into_os_string()); - } - OverlayLaunch { - arguments, - environment, - } - } - - pub(super) fn spawn_overlay_process(&mut self) -> Result<()> { - let candidates = self.overlay_spawn_candidates(); + pub(super) fn spawn_overlay_process( + &mut self, + request: &OverlayLaunchRequest, + candidates: &[OverlaySpawnCandidate], + ) -> std::result::Result<(), OverlayStartFailure> { if candidates.is_empty() { - return Err(anyhow!("No overlay spawn candidates available")); + return Err(OverlayStartFailure::BeforeAttempt(anyhow!( + "No overlay spawn candidates available" + ))); } let mut failures = Vec::new(); for candidate in candidates { - self.overlay_child.reserve()?; - let had_activation_token = self.pending_activation_token.is_some(); - debug!( - "Attempting overlay spawn via {} ({})", - candidate.source, - candidate.program.to_string_lossy() - ); - let launch = self.build_overlay_launch(); - let attempt = (|| -> Result { - let daemon_watchdog = super::super::protocol_v2::open_daemon_watchdog()?; - let child = crate::process_broker::current()?.spawn_with_watchdog( - crate::process_broker::HelperKind::Overlay, - crate::process_broker::HelperLifetime::OwnedChild, - &candidate.program, - &launch.arguments, - launch.environment, - daemon_watchdog.as_raw_fd(), - )?; - let pid = child.id(); - self.overlay_child.start(child)?; - self.overlay_child - .wait_until_ready(Duration::from_secs(5), &self.instance_token)?; - Ok(pid) - })(); - match attempt { + match self.overlay.start( + request, + candidate, + &self.session_resume_override, + &self.instance_token, + ) { Ok(pid) => { - self.overlay_active - .store(true, std::sync::atomic::Ordering::Release); - self.overlay_state = OverlayState::Visible; - self.active_named_session_file = self.effective_named_session_file(); - self.pending_activation_token = None; info!( "Overlay process started via {} (pid {pid}, startup_activation_token={})", - candidate.source, had_activation_token + candidate.source, + request.activation_token().is_some() ); return Ok(()); } - Err(error) => { - self.overlay_child.abort_reservation(); + Err(OverlayStartFailure::BeforeAttempt(error)) => { + return Err(OverlayStartFailure::BeforeAttempt(error)); + } + Err(OverlayStartFailure::Attempt(error)) => { failures.push(format!( "{} ({}) -> {error:#}", candidate.source, - candidate.program.to_string_lossy(), + candidate.program.to_string_lossy() )); } } } - self.pending_activation_token = None; - self.overlay_child.abort_reservation(); - warn!("Overlay spawn attempts failed: {}", failures.join("; ")); - Err(anyhow!( + self.overlay.abort_start(); + let failed = SpawnAttemptsFailed { attempts: failures }; + warn!( + "Overlay spawn attempts failed: {}", + failed.attempts.join("; ") + ); + + Err(OverlayStartFailure::Attempt(anyhow::Error::new(failed))) + } +} + +/// Every spawn candidate failed. It displays only the summary that callers +/// report, as `{:#}` too, since it has no source; the per-candidate failures go +/// to the log and stay on the error for inspection. +#[derive(Debug)] +pub(super) struct SpawnAttemptsFailed { + attempts: Vec, +} + +impl SpawnAttemptsFailed { + #[cfg(test)] + pub(super) fn attempts(&self) -> &[String] { + &self.attempts + } +} + +impl std::fmt::Display for SpawnAttemptsFailed { + fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!( + formatter, "Unable to launch overlay process (tried current_exe/argv0/{PATH_ENV})" - )) + ) } } +impl std::error::Error for SpawnAttemptsFailed {} + #[cfg(test)] mod tests { + use super::super::launch::{OverlayLaunch, build_overlay_launch}; use super::*; + fn build_test_launch(daemon: &mut Daemon) -> OverlayLaunch { + let request = daemon.take_pending_launch(); + build_overlay_launch(&request, daemon.session_resume_override(), None) + } + fn launch_args(launch: &OverlayLaunch) -> Vec { launch .arguments @@ -269,55 +186,12 @@ mod tests { .collect() } - #[test] - fn backoff_duration_grows_and_caps() { - let mut daemon = Daemon::new(None, false, None, None); - - daemon.overlay_spawn_failures = 1; - assert_eq!( - daemon.overlay_spawn_backoff_duration(), - Duration::from_secs(1) - ); - - daemon.overlay_spawn_failures = 2; - assert_eq!( - daemon.overlay_spawn_backoff_duration(), - Duration::from_secs(2) - ); - - daemon.overlay_spawn_failures = 5; - assert_eq!( - daemon.overlay_spawn_backoff_duration(), - Duration::from_secs(16) - ); - - daemon.overlay_spawn_failures = 6; - assert_eq!( - daemon.overlay_spawn_backoff_duration(), - Duration::from_secs(30) - ); - } - - #[test] - fn overlay_spawn_backoff_honors_retry_window() { - let mut daemon = Daemon::new(None, false, None, None); - daemon.overlay_spawn_next_retry = Some(Instant::now() + Duration::from_secs(2)); - daemon.overlay_spawn_backoff_logged = false; - - assert!(daemon.overlay_spawn_backoff_remaining().is_some()); - assert!(daemon.overlay_spawn_backoff_logged); - - daemon.overlay_spawn_next_retry = Some(Instant::now() - Duration::from_secs(1)); - assert!(daemon.overlay_spawn_backoff_remaining().is_none()); - assert!(!daemon.overlay_spawn_backoff_logged); - } - #[test] fn build_overlay_command_includes_freeze_when_enabled() { let mut daemon = Daemon::new(Some("whiteboard".into()), false, None, None); daemon.set_freeze_on_show(true); - let launch = daemon.build_overlay_launch(); + let launch = build_test_launch(&mut daemon); assert_eq!( launch_args(&launch), @@ -327,38 +201,47 @@ mod tests { #[test] fn build_overlay_command_uses_toggle_request_args() { - let mut daemon = Daemon::new(Some("whiteboard".into()), false, None, None); - daemon.pending_toggle_request = Some(crate::daemon::DaemonToggleRequest { - mode: Some("transparent".into()), - freeze: true, - exit_after_capture: true, - ..Default::default() - }); - - let launch = daemon.build_overlay_launch(); + for (exit_after_capture, no_exit_after_capture, expected_flag) in [ + (true, false, "--exit-after-capture"), + (false, true, "--no-exit-after-capture"), + ] { + let mut daemon = Daemon::new(Some("whiteboard".into()), false, None, None); + daemon.queue_overlay_launch( + Some(crate::daemon::DaemonToggleRequest { + mode: Some("transparent".into()), + freeze: true, + exit_after_capture, + no_exit_after_capture, + ..Default::default() + }), + None, + ); - assert_eq!( - launch_args(&launch), - vec![ - "--active", - "--freeze", - "--exit-after-capture", - "--mode", - "transparent" - ] - ); + let launch = build_test_launch(&mut daemon); + + assert_eq!( + launch_args(&launch), + vec![ + "--active", + "--freeze", + expected_flag, + "--mode", + "transparent" + ] + ); + } } #[test] fn build_overlay_command_includes_initial_named_session_file() { - let daemon = Daemon::new( + let mut daemon = Daemon::new( Some("whiteboard".into()), false, None, Some(std::path::PathBuf::from("/tmp/lecture.wayscriber-session")), ); - let launch = daemon.build_overlay_launch(); + let launch = build_test_launch(&mut daemon); assert_eq!( launch_args(&launch), @@ -380,14 +263,17 @@ mod tests { None, Some(std::path::PathBuf::from("/tmp/default.wayscriber-session")), ); - daemon.pending_toggle_request = Some(crate::daemon::DaemonToggleRequest { - session_file: Some(std::path::PathBuf::from( - "/tmp/requested.wayscriber-session", - )), - ..Default::default() - }); + daemon.queue_overlay_launch( + Some(crate::daemon::DaemonToggleRequest { + session_file: Some(std::path::PathBuf::from( + "/tmp/requested.wayscriber-session", + )), + ..Default::default() + }), + None, + ); - let launch = daemon.build_overlay_launch(); + let launch = build_test_launch(&mut daemon); assert_eq!( launch_args(&launch), @@ -403,9 +289,9 @@ mod tests { #[test] fn build_overlay_command_omits_freeze_by_default() { - let daemon = Daemon::new(Some("whiteboard".into()), false, None, None); + let mut daemon = Daemon::new(Some("whiteboard".into()), false, None, None); - let launch = daemon.build_overlay_launch(); + let launch = build_test_launch(&mut daemon); assert_eq!( launch_args(&launch), diff --git a/src/daemon/overlay/tests.rs b/src/daemon/overlay/tests.rs new file mode 100644 index 000000000..20d55bc9e --- /dev/null +++ b/src/daemon/overlay/tests.rs @@ -0,0 +1,337 @@ +use super::*; +use std::ffi::OsStr; +use std::fs; +use std::path::{Path, PathBuf}; +use std::sync::atomic::Ordering; +use std::time::Instant; + +mod fake_overlay; + +pub(in crate::daemon) fn with_visible_overlay( + named_file: Option, + ignore_term: bool, + body: impl FnOnce(&mut Daemon), +) { + with_fixture(ignore_term, |daemon, candidate, _| { + daemon.queue_overlay_launch( + named_file.map(|session_file| DaemonToggleRequest { + session_file: Some(session_file), + ..Default::default() + }), + None, + ); + let request = daemon.take_pending_launch(); + daemon + .start_launch(request, &[candidate]) + .unwrap() + .require_shown() + .unwrap(); + + body(daemon); + + daemon.hide_overlay().unwrap(); + }); +} + +pub(in crate::daemon) fn assert_token_only_pending_launch(daemon: &Daemon, token: &str) { + let retained = daemon.pending_launch.as_ref().expect("token must be kept"); + + assert_eq!(retained.activation_token(), Some(token)); + assert_eq!(retained.mode(), Some("transparent")); + assert_eq!(retained.session_resume_override(Some(true)), Some(true)); +} + +/// Runs `body` with a broker whose overlay launches of this test binary act +/// as [`fake_overlay`] children. Receipts land in the runtime root passed to `body`. +fn with_fixture(ignore_term: bool, body: impl FnOnce(&mut Daemon, OverlaySpawnCandidate, &Path)) { + let temp = crate::test_temp::tempdir().unwrap(); + let behavior = if ignore_term { + fake_overlay::IGNORES_TERM + } else { + fake_overlay::STOPS_ON_TERM + }; + + crate::test_env::with_env_vars( + &[ + ( + crate::env_vars::XDG_RUNTIME_DIR_ENV, + Some(temp.path().as_os_str()), + ), + (fake_overlay::FAKE_OVERLAY_ENV, Some(OsStr::new(behavior))), + ], + || { + let _broker = crate::process_broker::start_for_runtime().unwrap(); + let mut daemon = Daemon::new( + Some("transparent".into()), + false, + Some(true), + Some(PathBuf::from("/tmp/home.wayscriber-session")), + ); + + daemon.instance_token = crate::daemon::protocol_v2::ProtocolToken::generate() + .unwrap() + .to_string(); + let candidate = OverlaySpawnCandidate { + program: std::env::current_exe().unwrap().into_os_string(), + source: "test binary", + }; + + body(&mut daemon, candidate, temp.path()); + }, + ); +} + +fn receipt(root: &Path) -> serde_json::Value { + let deadline = Instant::now() + Duration::from_secs(3); + loop { + if let Some(path) = fs::read_dir(root) + .unwrap() + .filter_map(Result::ok) + .map(|entry| entry.path()) + .find(|path| path.extension().is_some_and(|ext| ext == "receipt")) + { + return serde_json::from_slice(&fs::read(path).unwrap()).unwrap(); + } + assert!( + Instant::now() < deadline, + "owned overlay did not publish its launch receipt" + ); + std::thread::sleep(Duration::from_millis(5)); + } +} + +fn assert_retired(daemon: &Daemon, generation: &str) { + assert_eq!(daemon.overlay.state(), OverlayState::Hidden); + assert!(!daemon.overlay.active_flag().load(Ordering::Acquire)); + assert!(daemon.overlay.active_named_session_file().is_none()); + assert!(daemon.overlay.poll_fd().is_none()); + let proofs = crate::daemon::protocol_v2::command_root().join("children"); + for suffix in ["active", "enabled", "ready", "signals"] { + assert!(!proofs.join(format!("{generation}.{suffix}")).exists()); + } +} + +#[test] +fn real_child_hide_restart_and_natural_retirement_clear_target_and_visibility() { + with_fixture(false, |daemon, _, root| { + daemon.queue_overlay_launch( + Some(DaemonToggleRequest { + mode: Some("whiteboard".into()), + session_file: Some(PathBuf::from("/tmp/away.wayscriber-session")), + freeze: true, + no_exit_after_capture: true, + no_resume_session: true, + ..Default::default() + }), + Some("owned-token".into()), + ); + daemon.show_overlay().unwrap().require_shown().unwrap(); + let first = receipt(root); + + assert_eq!( + first["args"], + serde_json::json!([ + "--active", + "--freeze", + "--no-exit-after-capture", + "--mode", + "whiteboard", + "--session-file", + "/tmp/away.wayscriber-session" + ]) + ); + assert_eq!(first["token"], "owned-token"); + assert_eq!(first["startup"], "owned-token"); + assert_eq!(first["resume"], "off"); + assert_eq!(first["detach"], "1"); + assert!(daemon.overlay.active_flag().load(Ordering::Acquire)); + assert_eq!( + daemon.overlay.active_named_session_file(), + Some(Path::new("/tmp/away.wayscriber-session")) + ); + assert!(daemon.pending_launch.is_none()); + + daemon.queue_overlay_launch( + Some(DaemonToggleRequest { + mode: Some("blackboard".into()), + freeze: true, + no_resume_session: true, + ..Default::default() + }), + Some("hide-retained-token".into()), + ); + + daemon.hide_overlay().unwrap(); + assert_retired(daemon, first["generation"].as_str().unwrap()); + fs::remove_file(root.join(format!("{}.receipt", first["generation"].as_str().unwrap()))) + .unwrap(); + daemon.show_overlay().unwrap().require_shown().unwrap(); + let second = receipt(root); + + assert_ne!(first["generation"], second["generation"]); + assert_eq!( + second["args"], + serde_json::json!([ + "--active", + "--mode", + "transparent", + "--session-file", + "/tmp/home.wayscriber-session" + ]) + ); + assert_eq!(second["token"], "hide-retained-token"); + assert_eq!(second["startup"], "hide-retained-token"); + assert_eq!(second["resume"], "on"); + assert!(daemon.pending_launch.is_none()); + assert_eq!( + daemon.overlay.active_named_session_file(), + Some(Path::new("/tmp/home.wayscriber-session")) + ); + + daemon.overlay.signal(libc::SIGTERM).unwrap(); + let deadline = Instant::now() + Duration::from_secs(3); + while daemon.overlay.state() == OverlayState::Visible { + daemon.update_overlay_process_state().unwrap(); + assert!(Instant::now() < deadline, "exited child was not retired"); + std::thread::sleep(Duration::from_millis(5)); + } + assert_retired(daemon, second["generation"].as_str().unwrap()); + }); +} + +#[test] +fn real_spawn_failure_drops_options_and_token_before_retry() { + with_fixture(false, |daemon, candidate, root| { + // A real child that starts and exits before publishing readiness. + let failed_program = root.join(fake_overlay::EXITS_BEFORE_READY); + std::os::unix::fs::symlink(&candidate.program, &failed_program).unwrap(); + daemon.queue_overlay_launch( + Some(DaemonToggleRequest { + mode: Some("whiteboard".into()), + no_resume_session: true, + session_file: Some(PathBuf::from("/tmp/rejected.wayscriber-session")), + ..Default::default() + }), + Some("rejected-token".into()), + ); + let request = daemon.take_pending_launch(); + + let failure = daemon + .start_launch( + request, + &[OverlaySpawnCandidate { + program: failed_program.into_os_string(), + source: "exits before readiness", + }], + ) + .unwrap_err(); + + // Callers report the summary alone, even with `{:#}`; the attempts stay + // on the error for inspection. + assert_eq!( + format!("{failure:#}"), + format!( + "Unable to launch overlay process (tried current_exe/argv0/{})", + crate::env_vars::PATH_ENV + ) + ); + let attempts = failure + .downcast_ref::() + .expect("every candidate failed") + .attempts(); + assert!( + attempts.iter().any(|attempt| { + attempt.contains("overlay child exited before publishing readiness") + }), + "{attempts:?}" + ); + assert!( + fs::read_dir(root) + .unwrap() + .filter_map(Result::ok) + .any(|entry| { + entry + .path() + .extension() + .is_some_and(|extension| extension == fake_overlay::STARTED_THEN_EXITED) + }), + "the failing candidate never started" + ); + assert!(daemon.pending_launch.is_none()); + assert_eq!(daemon.overlay.state(), OverlayState::Hidden); + assert!(!daemon.overlay.active_flag().load(Ordering::Acquire)); + assert!(daemon.overlay.poll_fd().is_none()); + let deadline = Instant::now() + Duration::from_secs(3); + while daemon.overlay_start_backoff().is_some() { + assert!(Instant::now() < deadline); + std::thread::sleep(Duration::from_millis(10)); + } + + let request = daemon.take_pending_launch(); + daemon + .start_launch(request, &[candidate]) + .unwrap() + .require_shown() + .unwrap(); + let actual = receipt(root); + + assert_eq!( + actual["args"], + serde_json::json!([ + "--active", + "--mode", + "transparent", + "--session-file", + "/tmp/home.wayscriber-session" + ]) + ); + assert!(actual["token"].is_null()); + assert!(actual["startup"].is_null()); + assert_eq!(actual["resume"], "on"); + daemon.hide_overlay().unwrap(); + assert_retired(daemon, actual["generation"].as_str().unwrap()); + }); +} + +#[test] +fn preparation_error_keeps_only_the_token_and_does_not_replay_options() { + with_fixture(false, |daemon, candidate, _| { + let commands = crate::daemon::protocol_v2::command_root(); + fs::create_dir_all(&commands).unwrap(); + fs::write(commands.join("children"), b"not a proof directory").unwrap(); + daemon.queue_overlay_launch( + Some(DaemonToggleRequest { + mode: Some("whiteboard".into()), + no_resume_session: true, + ..Default::default() + }), + Some("retained-token".into()), + ); + let request = daemon.take_pending_launch(); + + assert!(daemon.start_launch(request, &[candidate]).is_err()); + let retained = daemon.take_pending_launch(); + let actual = launch::build_overlay_launch(&retained, Some(true), None); + + assert_eq!( + actual.arguments, + [ + "--active", + "--mode", + "transparent", + "--session-file", + "/tmp/home.wayscriber-session" + ] + .map(std::ffi::OsString::from) + ); + assert!(actual.environment.contains(&( + crate::env_vars::XDG_ACTIVATION_TOKEN_ENV.into(), + Some("retained-token".into()) + ))); + assert!( + actual + .environment + .contains(&(crate::RESUME_SESSION_ENV.into(), Some("on".into()))) + ); + }); +} diff --git a/src/daemon/overlay/tests/fake_overlay.rs b/src/daemon/overlay/tests/fake_overlay.rs new file mode 100644 index 000000000..2a8608454 --- /dev/null +++ b/src/daemon/overlay/tests/fake_overlay.rs @@ -0,0 +1,172 @@ +//! The unit-test binary doubles as the overlay process for real-child tests. +//! +//! Spawn candidate discovery tries `current_exe()` first, which under +//! `cargo test` is this test binary. A fixture marks the environment of the +//! broker it starts; when the daemon then launches this binary as an overlay +//! child, the constructor below runs the production child handshake and +//! records how it was launched, before libtest would parse the overlay +//! arguments. Without both the fixture marker and an overlay generation, the +//! constructor returns and the binary runs its tests as usual. + +use std::collections::HashMap; +use std::convert::Infallible; +use std::ffi::OsStr; +use std::os::unix::ffi::OsStrExt; +use std::path::Path; +use std::time::Duration; + +use anyhow::{Context, Result}; + +use crate::daemon::protocol_v2::{ + ActiveGeneration, active_generation_from_environment, publish_ready_from_environment, + publish_signal_ready_from_environment, +}; + +/// Fixture marker inherited by overlay launches from the test's broker. +pub(super) const FAKE_OVERLAY_ENV: &str = "WAYSCRIBER_TEST_FAKE_OVERLAY"; +/// Behave like an overlay that exits on SIGTERM. +pub(super) const STOPS_ON_TERM: &str = "stops-on-term"; +/// Behave like an overlay that ignores SIGTERM and must be force-killed. +pub(super) const IGNORES_TERM: &str = "ignores-term"; +/// Launched under this name, through a link to the test binary, the fake exits +/// before publishing readiness, like an overlay that dies during startup. The +/// marker above applies to every launch from the fixture's broker, so this +/// behavior is chosen per launch instead. +pub(super) const EXITS_BEFORE_READY: &str = "wayscriber-exits-before-ready"; +/// Written to the fixture's runtime directory as that child starts, so a test +/// can tell a child that ran and exited from one that never started. +pub(super) const STARTED_THEN_EXITED: &str = "started-then-exited"; +const EXIT_BEFORE_READY_STATUS: i32 = 7; +/// The fake overlay could not serve; the test that launched it then fails. +const FAILURE_STATUS: i32 = 1; + +// SAFETY: the loader calls each `.init_array` entry once, before `main`, with +// the C calling convention. glibc passes `argc`, `argv`, and `envp`; under the +// C ABI a function that declares no parameters ignores extra arguments, so an +// `extern "C" fn()` is sound to register here. Without both markers the +// function only reads the environment and returns. With them it never returns +// into `main`: it serves or exits, and a panic aborts instead of unwinding +// through the loader because the function is `extern "C"`. +#[used] +#[unsafe(link_section = ".init_array")] +static RUN_AS_FAKE_OVERLAY: extern "C" fn() = run_as_fake_overlay; + +extern "C" fn run_as_fake_overlay() { + let Some(behavior) = std::env::var_os(FAKE_OVERLAY_ENV) else { + return; + }; + if std::env::var_os(crate::env_vars::OVERLAY_CHILD_GENERATION_ENV).is_none() { + return; + } + + if launched_as(EXITS_BEFORE_READY) { + if let Err(error) = record(STARTED_THEN_EXITED, b"") { + eprintln!("fake overlay failed: {error:#}"); + } + std::process::exit(EXIT_BEFORE_READY_STATUS); + } + + let Err(error) = serve(behavior == IGNORES_TERM); + eprintln!("fake overlay failed: {error:#}"); + std::process::exit(FAILURE_STATUS); +} + +fn serve(ignore_term: bool) -> Result { + // SAFETY: installs ignore dispositions only; no handler code runs. + unsafe { + // Action delivery signals the overlay; the fake has no action handler. + libc::signal(libc::SIGUSR2, libc::SIG_IGN); + if ignore_term { + libc::signal(libc::SIGTERM, libc::SIG_IGN); + } + } + + publish_ready_from_environment()?; + publish_signal_ready_from_environment()?; + while !matches!( + active_generation_from_environment()?, + ActiveGeneration::Enabled { .. } + ) { + std::thread::sleep(Duration::from_millis(2)); + } + + write_receipt()?; + + loop { + std::thread::sleep(Duration::from_secs(60)); + } +} + +fn write_receipt() -> Result<()> { + let environment = launch_environment()?; + let launched_with = |name: &str| environment.get(name).cloned(); + let receipt = serde_json::json!({ + "args": launch_arguments()?, + "token": launched_with(crate::env_vars::XDG_ACTIVATION_TOKEN_ENV), + "startup": launched_with(crate::env_vars::DESKTOP_STARTUP_ID_ENV), + "resume": launched_with(crate::RESUME_SESSION_ENV), + "detach": launched_with(crate::env_vars::NO_DETACH_ENV), + "pid": std::process::id(), + "generation": std::env::var(crate::env_vars::OVERLAY_CHILD_GENERATION_ENV)?, + }); + + record("receipt", &serde_json::to_vec(&receipt)?) +} + +/// Writes `.` into the fixture's runtime directory. +fn record(kind: &str, contents: &[u8]) -> Result<()> { + let generation = std::env::var(crate::env_vars::OVERLAY_CHILD_GENERATION_ENV)?; + let root = std::env::var_os(crate::env_vars::XDG_RUNTIME_DIR_ENV) + .context("fake overlay needs the fixture's runtime directory")?; + + crate::durable_io::write_atomic( + &Path::new(&root).join(format!("{generation}.{kind}")), + contents, + crate::durable_io::AtomicWriteOptions::private_runtime_file(), + )?; + + Ok(()) +} + +/// Whether this process was started as `name`: the broker passes the program +/// path it was given as `argv[0]`, which for a link is the link's own path. +fn launched_as(name: &str) -> bool { + nul_separated("/proc/self/cmdline") + .ok() + .and_then(|arguments| arguments.into_iter().next()) + .is_some_and(|program| { + Path::new(OsStr::from_bytes(&program)).file_name() == Some(OsStr::new(name)) + }) +} + +/// Reads the kernel's copy of argv: std's own argument capture is not +/// guaranteed to have run before this constructor. +fn launch_arguments() -> Result> { + nul_separated("/proc/self/cmdline")? + .into_iter() + .skip(1) + .map(|argument| String::from_utf8(argument).context("non-UTF-8 launch argument")) + .collect() +} + +/// Reads the environment as the daemon launched this process. Library +/// constructors that run first consume startup-notification variables such +/// as `XDG_ACTIVATION_TOKEN`, so the live environment no longer shows them. +fn launch_environment() -> Result> { + Ok(nul_separated("/proc/self/environ")? + .into_iter() + .filter_map(|entry| String::from_utf8(entry).ok()) + .filter_map(|entry| { + entry + .split_once('=') + .map(|(name, value)| (name.to_owned(), value.to_owned())) + }) + .collect()) +} + +fn nul_separated(path: &str) -> Result>> { + let raw = std::fs::read(path).with_context(|| format!("failed to read {path}"))?; + let raw = raw.strip_suffix(b"\0").unwrap_or(&raw); + + Ok(raw.split(|byte| *byte == 0).map(<[u8]>::to_vec).collect()) +} diff --git a/src/daemon/protocol_v2/mod.rs b/src/daemon/protocol_v2/mod.rs index 29516d47a..83770e9b6 100644 --- a/src/daemon/protocol_v2/mod.rs +++ b/src/daemon/protocol_v2/mod.rs @@ -8,6 +8,8 @@ mod runtime; mod wire; pub(crate) use child::OverlayChildOwner; +#[cfg(test)] +pub(crate) use child::{ActiveGeneration, active_generation_from_environment}; pub(crate) use child::{ open_daemon_watchdog, recover_stale_child_records, start_daemon_watchdog_from_environment, }; diff --git a/src/daemon/tests.rs b/src/daemon/tests.rs index 5289c87b9..a1a823224 100644 --- a/src/daemon/tests.rs +++ b/src/daemon/tests.rs @@ -67,9 +67,11 @@ fn hide_overlay_is_idempotent() { daemon.hide_overlay().unwrap(); assert_eq!(daemon.test_state(), OverlayState::Hidden); - daemon.overlay_state = OverlayState::Visible; - daemon.toggle_overlay().unwrap(); - assert_eq!(daemon.test_state(), OverlayState::Hidden); + super::overlay::tests::with_visible_overlay(None, false, |daemon| { + daemon.toggle_overlay().unwrap(); + daemon.hide_overlay().unwrap(); + assert_eq!(daemon.test_state(), OverlayState::Hidden); + }); } #[cfg(feature = "tray")] diff --git a/src/test_env.rs b/src/test_env.rs index a69400667..a175b8241 100644 --- a/src/test_env.rs +++ b/src/test_env.rs @@ -30,16 +30,34 @@ pub(crate) fn with_env_var( key: &'static str, value: Option<&std::ffi::OsStr>, body: impl FnOnce() -> T, +) -> T { + with_env_vars(&[(key, value)], body) +} + +/// Sets several variables under one hold of the environment lock, restoring +/// all of them afterwards, including when `body` panics. +#[cfg(test)] +pub(crate) fn with_env_vars( + vars: &[(&'static str, Option<&std::ffi::OsStr>)], + body: impl FnOnce() -> T, ) -> T { let _guard = lock(); - let _saved = SavedEnv(vec![(key, std::env::var_os(key))]); - // SAFETY: serialized by the environment lock held above. - unsafe { - match value { - Some(value) => std::env::set_var(key, value), - None => std::env::remove_var(key), + let _saved = SavedEnv( + vars.iter() + .map(|(key, _)| (*key, std::env::var_os(key))) + .collect(), + ); + + for (key, value) in vars { + // SAFETY: serialized by the environment lock held above. + unsafe { + match value { + Some(value) => std::env::set_var(key, value), + None => std::env::remove_var(key), + } } } + body() } diff --git a/src/toolbar_gtk/view/capture_suppression.rs b/src/toolbar_gtk/view/capture_suppression.rs index 104255b63..e9b9c17bf 100644 --- a/src/toolbar_gtk/view/capture_suppression.rs +++ b/src/toolbar_gtk/view/capture_suppression.rs @@ -489,6 +489,7 @@ fn reject_withdrawn_target( mod tests { mod presentation; mod process; + mod wayland_proxy; use super::*; diff --git a/src/toolbar_gtk/view/capture_suppression/tests/presentation.rs b/src/toolbar_gtk/view/capture_suppression/tests/presentation.rs index 889561824..98ea38062 100644 --- a/src/toolbar_gtk/view/capture_suppression/tests/presentation.rs +++ b/src/toolbar_gtk/view/capture_suppression/tests/presentation.rs @@ -1,3 +1,4 @@ +use super::wayland_proxy::{CONTROL_ENV, ProxyResponse, ProxyStatus, run_in_private_wayland}; use super::*; use std::future::{Future, poll_fn}; use std::io::{BufRead, BufReader, Write}; @@ -115,17 +116,7 @@ fn run_in_private_display(test_name: &str) -> bool { let test_name = test_name .strip_prefix(concat!(env!("CARGO_CRATE_NAME"), "::")) .expect("test name contains the crate prefix"); - let fixture = concat!( - env!("CARGO_MANIFEST_DIR"), - "/tools/test-fixtures/gtk_popup_wayland.py" - ); - let output = std::process::Command::new("python3") - .arg(fixture) - .arg(std::env::current_exe().expect("test binary")) - .arg(test_name) - .env(CHILD_ENV, "1") - .output() - .expect("run private Wayland popup fixture"); + let output = run_in_private_wayland(CHILD_ENV, test_name); std::io::stdout().write_all(&output.stdout).unwrap(); std::io::stderr().write_all(&output.stderr).unwrap(); @@ -250,22 +241,8 @@ async fn wait_for_held_popup_callback() { } } -#[derive(Debug, serde::Deserialize)] -struct ProxyStatus { - held: bool, - popup_commits: u64, - popup_callbacks_delivered: u64, -} - -#[derive(Debug, serde::Deserialize)] -#[serde(untagged)] -enum ProxyResponse { - Status(ProxyStatus), - Error { error: String }, -} - fn control(command: &str) -> ProxyStatus { - let path = std::env::var_os("WAYSCRIBER_GTK_WAYLAND_CONTROL").expect("private proxy control"); + let path = std::env::var_os(CONTROL_ENV).expect("private proxy control"); let mut socket = UnixStream::connect(path).expect("connect proxy control"); socket.set_read_timeout(Some(FIXTURE_WAIT)).unwrap(); diff --git a/src/toolbar_gtk/view/capture_suppression/tests/wayland_proxy.rs b/src/toolbar_gtk/view/capture_suppression/tests/wayland_proxy.rs new file mode 100644 index 000000000..9622421cd --- /dev/null +++ b/src/toolbar_gtk/view/capture_suppression/tests/wayland_proxy.rs @@ -0,0 +1,250 @@ +//! Runs one GTK test behind a private headless Weston with one frame event withheld. +//! +//! All protocol messages and SCM_RIGHTS descriptors pass through unchanged, except +//! the selected popup callback and its `delete_id` event. The protocol tables +//! compiled into `wayland-client` and `wayland-protocols` provide opcodes and +//! new-object types; this fixture neither renders nor invents feedback. + +mod protocol; +mod relay; +mod wire; + +use std::fs::File; +use std::io::Read; +use std::os::unix::fs::FileTypeExt; +use std::os::unix::process::CommandExt; +use std::path::{Path, PathBuf}; +use std::process::{Child, Command, ExitStatus, Output, Stdio}; +use std::thread::JoinHandle; +use std::time::{Duration, Instant}; + +/// Names the control socket for the GTK test running behind the proxy. +pub(super) const CONTROL_ENV: &str = "WAYSCRIBER_GTK_WAYLAND_CONTROL"; + +const UPSTREAM_SOCKET: &str = "upstream"; +const PROXY_SOCKET: &str = "proxy"; +const CONTROL_SOCKET: &str = "control"; +const WESTON_START_TIMEOUT: Duration = Duration::from_secs(10); +const WESTON_STOP_TIMEOUT: Duration = Duration::from_secs(3); +const TEST_TIMEOUT: Duration = Duration::from_secs(20); +const POLL_INTERVAL: Duration = Duration::from_millis(10); + +#[derive(Debug, serde::Serialize, serde::Deserialize)] +pub(super) struct ProxyStatus { + pub(super) held: bool, + pub(super) popup_commits: u64, + pub(super) popup_callbacks_delivered: u64, +} + +#[derive(Debug, serde::Serialize, serde::Deserialize)] +#[serde(untagged)] +pub(super) enum ProxyResponse { + Status(ProxyStatus), + Error { error: String }, +} + +impl ProxyResponse { + fn error(message: impl Into) -> Self { + Self::Error { + error: message.into(), + } + } +} + +/// Runs `test_name` from this test binary in a private D-Bus session whose +/// Wayland display is the proxy, and returns its captured output. +pub(super) fn run_in_private_wayland(child_env: &str, test_name: &str) -> Output { + let runtime = tempfile::Builder::new() + .prefix("wayscriber-popup-test-") + .tempdir() + .expect("private XDG_RUNTIME_DIR"); + let weston = Weston::start(runtime.path()); + let proxy = relay::Proxy::start(runtime.path(), weston.socket.clone(), protocol::schema()) + .expect("listen for proxied Wayland clients"); + + let output = run_test_session(runtime.path(), child_env, test_name); + + // Stop relaying before the compositor, and remove the runtime directory last. + drop(proxy); + drop(weston); + output +} + +fn private_environment<'a>( + command: &'a mut Command, + runtime: &Path, + display: &str, +) -> &'a mut Command { + command + .env("XDG_RUNTIME_DIR", runtime) + .env("WAYLAND_DISPLAY", display) + .env("GDK_BACKEND", "wayland") + .env("GSK_RENDERER", "gl") + .env("LIBGL_ALWAYS_SOFTWARE", "1") + .env("GTK_A11Y", "test") + .env("GDK_DEBUG", "no-portals") + .env_remove("DISPLAY") + .env_remove("WAYLAND_SOCKET") + .env_remove("DBUS_SESSION_BUS_ADDRESS") +} + +struct Weston { + process: Child, + socket: PathBuf, +} + +impl Weston { + fn start(runtime: &Path) -> Self { + let log_path = runtime.join("weston.log"); + let log = File::create(&log_path).expect("create the Weston log"); + let mut command = Command::new("weston"); + command + .args([ + "--backend=headless-backend.so", + "--renderer=pixman", + "--no-config", + ]) + .arg(format!("--socket={UPSTREAM_SOCKET}")) + .arg("--idle-time=0") + .stdout(log.try_clone().expect("share the Weston log")) + .stderr(log); + private_environment(&mut command, runtime, UPSTREAM_SOCKET); + + let mut weston = Self { + process: command.spawn().expect("start the private headless Weston"), + socket: runtime.join(UPSTREAM_SOCKET), + }; + weston.wait_for_socket(&log_path); + weston + } + + fn wait_for_socket(&mut self, log_path: &Path) { + let deadline = Instant::now() + WESTON_START_TIMEOUT; + while !is_socket(&self.socket) { + let exited = self.process.try_wait().expect("poll Weston").is_some(); + if exited || Instant::now() >= deadline { + let log = std::fs::read_to_string(log_path).unwrap_or_default(); + panic!("private Weston did not create its socket:\n{log}"); + } + + std::thread::sleep(POLL_INTERVAL); + } + } +} + +impl Drop for Weston { + fn drop(&mut self) { + if matches!(self.process.try_wait(), Ok(None)) { + // SAFETY: kill only signals the unreaped child this guard owns. + unsafe { libc::kill(self.process.id() as libc::pid_t, libc::SIGTERM) }; + } + if !matches!( + wait_until(&mut self.process, WESTON_STOP_TIMEOUT), + Ok(Some(_)) + ) { + let _ = self.process.kill(); + } + let _ = self.process.wait(); + } +} + +fn is_socket(path: &Path) -> bool { + std::fs::metadata(path).is_ok_and(|metadata| metadata.file_type().is_socket()) +} + +fn run_test_session(runtime: &Path, child_env: &str, test_name: &str) -> Output { + let mut command = Command::new("dbus-run-session"); + command + .arg("--") + .arg(std::env::current_exe().expect("test binary")) + .args([test_name, "--exact", "--test-threads=1", "--nocapture"]); + private_environment(&mut command, runtime, PROXY_SOCKET) + .env(child_env, "1") + .env("G_DEBUG", "fatal-criticals") + .env(CONTROL_ENV, runtime.join(CONTROL_SOCKET)) + // GIO activates gvfsd on the private bus. Its FUSE helper would mount + // inside the runtime directory, and the final SIGKILL can strand that mount. + .env("GVFS_DISABLE_FUSE", "1") + .stdout(Stdio::piped()) + .stderr(Stdio::piped()); + // SAFETY: setsid is async-signal-safe and touches no parent memory. + unsafe { command.pre_exec(start_session) }; + + let mut session = TestSession(Some(command.spawn().expect("start dbus-run-session"))); + let leader = session.leader(); + let stdout = drain(leader.stdout.take()); + let stderr = drain(leader.stderr.take()); + let exited = wait_until(leader, TEST_TIMEOUT).expect("poll the private GTK test session"); + if exited.is_none() { + eprintln!("private GTK test exceeded {TEST_TIMEOUT:?}; killing its session"); + } + let status = session.finish(); + + Output { + status, + stdout: stdout.join().expect("collect test stdout"), + stderr: stderr.join().expect("collect test stderr"), + } +} + +fn start_session() -> std::io::Result<()> { + // SAFETY: setsid has no memory-safety preconditions. + if unsafe { libc::setsid() } < 0 { + return Err(std::io::Error::last_os_error()); + } + + Ok(()) +} + +/// Owns the session leader so the whole session is killed however the run ends. +struct TestSession(Option); + +impl TestSession { + fn leader(&mut self) -> &mut Child { + self.0.as_mut().expect("session leader is still owned") + } + + fn finish(mut self) -> ExitStatus { + let mut leader = self.0.take().expect("session leader is still owned"); + kill_session(&leader); + leader.wait().expect("reap the private GTK test session") + } +} + +impl Drop for TestSession { + fn drop(&mut self) { + if let Some(mut leader) = self.0.take() { + kill_session(&leader); + let _ = leader.wait(); + } + } +} + +/// Kills the whole session, including the private bus and any services it activated. +fn kill_session(leader: &Child) { + // SAFETY: killpg only sends a signal to the group this leader created. + unsafe { libc::killpg(leader.id() as libc::pid_t, libc::SIGKILL) }; +} + +fn drain(pipe: Option) -> JoinHandle> { + let mut pipe = pipe.expect("piped test output"); + std::thread::spawn(move || { + let mut bytes = Vec::new(); + pipe.read_to_end(&mut bytes).expect("read test output"); + bytes + }) +} + +fn wait_until(process: &mut Child, timeout: Duration) -> std::io::Result> { + let deadline = Instant::now() + timeout; + loop { + if let Some(status) = process.try_wait()? { + return Ok(Some(status)); + } + if Instant::now() >= deadline { + return Ok(None); + } + + std::thread::sleep(POLL_INTERVAL); + } +} diff --git a/src/toolbar_gtk/view/capture_suppression/tests/wayland_proxy/protocol.rs b/src/toolbar_gtk/view/capture_suppression/tests/wayland_proxy/protocol.rs new file mode 100644 index 000000000..dd25f4697 --- /dev/null +++ b/src/toolbar_gtk/view/capture_suppression/tests/wayland_proxy/protocol.rs @@ -0,0 +1,260 @@ +//! Message layouts from the protocol tables compiled into `wayland-client` and +//! `wayland-protocols`: core Wayland and stable xdg-shell, as the overlay links them. + +use std::collections::HashMap; + +use anyhow::{Context, Result}; +use wayland_client::Proxy; +use wayland_client::backend::protocol::{ + ANONYMOUS_INTERFACE, ArgumentType, Interface, MessageDesc, +}; +use wayland_client::protocol::{ + wl_compositor::WlCompositor, wl_data_device_manager::WlDataDeviceManager, + wl_display::WlDisplay, wl_fixes::WlFixes, wl_output::WlOutput, wl_seat::WlSeat, + wl_shell::WlShell, wl_shm::WlShm, wl_subcompositor::WlSubcompositor, +}; +use wayland_protocols::xdg::shell::client::xdg_wm_base::XdgWmBase; + +/// Interfaces by name. +pub(super) type Schema = HashMap<&'static str, &'static Interface>; + +#[derive(Clone, Copy, Debug)] +pub(super) enum Direction { + Request, + Event, +} + +impl Direction { + pub(super) fn messages(self, interface: &'static Interface) -> &'static [MessageDesc] { + match self { + Self::Request => interface.requests, + Self::Event => interface.events, + } + } +} + +/// The display, every global a client can bind, and every interface their +/// messages create or name. +pub(super) fn schema() -> Schema { + let mut schema = Schema::new(); + let mut pending = vec![ + WlDisplay::interface(), + WlCompositor::interface(), + WlShm::interface(), + WlDataDeviceManager::interface(), + WlShell::interface(), + WlSeat::interface(), + WlOutput::interface(), + WlSubcompositor::interface(), + WlFixes::interface(), + XdgWmBase::interface(), + ]; + while let Some(interface) = pending.pop() { + if std::ptr::eq(interface, &ANONYMOUS_INTERFACE) + || schema.insert(interface.name, interface).is_some() + { + continue; + } + for message in interface.requests.iter().chain(interface.events) { + pending.extend(message.child_interface); + pending.extend(message.arg_interfaces); + } + } + + schema +} + +/// One-word arguments by signature position, plus the objects a message creates. +pub(super) struct Arguments { + words: Vec>, + pub(super) new_objects: Vec<(u32, String)>, +} + +impl Arguments { + pub(super) fn word(&self, position: usize) -> Result { + self.words + .get(position) + .copied() + .flatten() + .with_context(|| format!("argument {position} is not a one-word value")) + } +} + +pub(super) fn decode(message: &MessageDesc, payload: &[u8]) -> Result { + let mut reader = Payload { + bytes: payload, + offset: 0, + }; + let mut arguments = Arguments { + words: Vec::with_capacity(message.signature.len()), + new_objects: Vec::new(), + }; + // An untyped new_id arrives as the interface name, its version, then the ID. + let mut named_interface = None; + + for kind in message.signature { + let word = match kind { + ArgumentType::Fd => None, + ArgumentType::Str(_) => { + named_interface = Some(reader.text()?); + None + } + ArgumentType::Array => { + reader.skip_array()?; + None + } + ArgumentType::NewId => { + let id = reader.word()?; + let interface = match message.child_interface { + Some(interface) => interface.name.to_owned(), + None => named_interface + .take() + .context("untyped new_id without an interface name")?, + }; + arguments.new_objects.push((id, interface)); + Some(id) + } + ArgumentType::Int + | ArgumentType::Uint + | ArgumentType::Fixed + | ArgumentType::Object(_) => Some(reader.word()?), + }; + arguments.words.push(word); + } + + Ok(arguments) +} + +struct Payload<'a> { + bytes: &'a [u8], + offset: usize, +} + +impl Payload<'_> { + fn word(&mut self) -> Result { + let word = self + .bytes + .get(self.offset..) + .and_then(<[u8]>::first_chunk::<4>) + .context("message payload ends inside an argument")?; + self.offset += 4; + + Ok(u32::from_ne_bytes(*word)) + } + + /// Reads a length-prefixed string. A length past the payload end yields only + /// the bytes present; a later argument read then fails. + fn text(&mut self) -> Result { + let length = self.word()? as usize; + let start = self.offset.min(self.bytes.len()); + let end = (self.offset + length).min(self.bytes.len()); + let raw = &self.bytes[start..end]; + self.offset += padded(length); + + let text_len = raw + .iter() + .rposition(|&byte| byte != 0) + .map_or(0, |last| last + 1); + String::from_utf8(raw[..text_len].to_vec()).context("Wayland string is not UTF-8") + } + + fn skip_array(&mut self) -> Result<()> { + let length = self.word()? as usize; + self.offset += padded(length); + + Ok(()) + } +} + +fn padded(length: usize) -> usize { + length.next_multiple_of(4) +} + +#[test] +fn schema_holds_every_core_and_xdg_shell_interface() { + let mut names: Vec<_> = schema().into_keys().collect(); + names.sort_unstable(); + + assert_eq!( + names, + [ + "wl_buffer", + "wl_callback", + "wl_compositor", + "wl_data_device", + "wl_data_device_manager", + "wl_data_offer", + "wl_data_source", + "wl_display", + "wl_fixes", + "wl_keyboard", + "wl_output", + "wl_pointer", + "wl_region", + "wl_registry", + "wl_seat", + "wl_shell", + "wl_shell_surface", + "wl_shm", + "wl_shm_pool", + "wl_subcompositor", + "wl_subsurface", + "wl_surface", + "wl_touch", + "xdg_popup", + "xdg_positioner", + "xdg_surface", + "xdg_toplevel", + "xdg_wm_base", + ] + ); +} + +#[test] +fn decode_reports_typed_and_bound_objects() { + let schema = schema(); + let words = |values: &[u32]| -> Vec { + values + .iter() + .flat_map(|value| value.to_ne_bytes()) + .collect() + }; + + let get_xdg_surface = message( + &schema, + "xdg_wm_base", + Direction::Request, + "get_xdg_surface", + ); + let arguments = decode(get_xdg_surface, &words(&[7, 5])).unwrap(); + assert_eq!(arguments.new_objects, [(7, "xdg_surface".to_owned())]); + assert_eq!( + (arguments.word(0).unwrap(), arguments.word(1).unwrap()), + (7, 5) + ); + + // bind(name, interface: "wl_compositor\0" padded to 16, version, id) + let mut bind = words(&[3, 14]); + bind.extend(b"wl_compositor\0\0\0"); + bind.extend(words(&[6, 9])); + let registry_bind = message(&schema, "wl_registry", Direction::Request, "bind"); + let arguments = decode(registry_bind, &bind).unwrap(); + assert_eq!(arguments.new_objects, [(9, "wl_compositor".to_owned())]); + assert!( + arguments.word(1).is_err(), + "a string is not a one-word value" + ); +} + +fn message( + schema: &Schema, + interface: &str, + direction: Direction, + name: &str, +) -> &'static MessageDesc { + direction + .messages(schema[interface]) + .iter() + .find(|message| message.name == name) + .expect("message in the schema") +} diff --git a/src/toolbar_gtk/view/capture_suppression/tests/wayland_proxy/relay.rs b/src/toolbar_gtk/view/capture_suppression/tests/wayland_proxy/relay.rs new file mode 100644 index 000000000..3178dcbc4 --- /dev/null +++ b/src/toolbar_gtk/view/capture_suppression/tests/wayland_proxy/relay.rs @@ -0,0 +1,411 @@ +//! Per-client relays between GTK and Weston, and the control socket that arms, +//! inspects, and releases the withheld popup frame callback. + +use std::collections::{HashMap, HashSet}; +use std::io::{self, BufRead, BufReader, Write}; +use std::net::Shutdown; +use std::os::unix::net::{UnixListener, UnixStream}; +use std::path::{Path, PathBuf}; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::{Arc, Mutex, MutexGuard, PoisonError}; +use std::thread::JoinHandle; + +use anyhow::{Context, Result}; + +use super::protocol::{self, Arguments, Direction, Schema}; +use super::wire::{self, Packet}; +use super::{CONTROL_SOCKET, PROXY_SOCKET, ProxyResponse, ProxyStatus}; + +/// Accepts GTK clients on `proxy` and commands on `control` until dropped. +pub(super) struct Proxy { + shared: Arc, + listeners: Vec<(PathBuf, JoinHandle<()>)>, +} + +struct Shared { + upstream: PathBuf, + schema: Arc, + connections: Mutex>>, + forwarders: Mutex>>, + stopping: AtomicBool, +} + +impl Proxy { + pub(super) fn start(directory: &Path, upstream: PathBuf, schema: Schema) -> io::Result { + let shared = Arc::new(Shared { + upstream, + schema: Arc::new(schema), + connections: Mutex::new(Vec::new()), + forwarders: Mutex::new(Vec::new()), + stopping: AtomicBool::new(false), + }); + + let proxy_path = directory.join(PROXY_SOCKET); + let clients = UnixListener::bind(&proxy_path)?; + let control_path = directory.join(CONTROL_SOCKET); + let commands = UnixListener::bind(&control_path)?; + + let accept_shared = Arc::clone(&shared); + let accept = std::thread::spawn(move || accept_shared.accept_clients(clients)); + let control_shared = Arc::clone(&shared); + let control = std::thread::spawn(move || control_shared.answer_commands(commands)); + + Ok(Self { + shared, + listeners: vec![(proxy_path, accept), (control_path, control)], + }) + } +} + +impl Drop for Proxy { + fn drop(&mut self) { + self.shared.stopping.store(true, Ordering::SeqCst); + for (path, listener) in self.listeners.drain(..) { + // A connection wakes the blocking accept so the thread sees `stopping`. + let wake = UnixStream::connect(&path); + let _ = listener.join(); + drop(wake); + } + + // No listener runs now, so the connection list is final. + for connection in lock(&self.shared.connections).iter() { + connection.shutdown(); + } + let forwarders = std::mem::take(&mut *lock(&self.shared.forwarders)); + for forwarder in forwarders { + let _ = forwarder.join(); + } + } +} + +impl Shared { + fn accept_clients(&self, listener: UnixListener) { + for client in listener.incoming() { + if self.stopping.load(Ordering::SeqCst) { + return; + } + + let client = client.expect("accept a proxied Wayland client"); + let server = + UnixStream::connect(&self.upstream).expect("connect to the private Weston"); + let connection = Arc::new(Connection::new(client, server, Arc::clone(&self.schema))); + lock(&self.connections).push(Arc::clone(&connection)); + + for direction in [Direction::Request, Direction::Event] { + let connection = Arc::clone(&connection); + let forwarder = std::thread::spawn(move || connection.forward(direction)); + lock(&self.forwarders).push(forwarder); + } + } + } + + /// Answers one line command per control connection with one JSON line. + fn answer_commands(&self, listener: UnixListener) { + for client in listener.incoming() { + if self.stopping.load(Ordering::SeqCst) { + return; + } + + let client = client.expect("accept a proxy control client"); + let mut command = String::new(); + if BufReader::new(&client).read_line(&mut command).is_err() { + continue; + } + + let response = self.control(command.trim()); + let mut line = serde_json::to_string(&response).expect("serialize proxy response"); + line.push('\n'); + // A client that left without reading its response needs nothing more. + let _ = (&client).write_all(line.as_bytes()); + } + } + + /// Applies a command to the latest connection that has created a popup. + fn control(&self, command: &str) -> ProxyResponse { + let connections = lock(&self.connections).clone(); + let latest = connections + .iter() + .rev() + .find(|connection| connection.has_popups()); + + match latest { + Some(connection) => connection.control(command), + None => ProxyResponse::error("no GTK popup connection"), + } + } +} + +struct Connection { + client: UnixStream, + server: UnixStream, + schema: Arc, + state: Mutex, +} + +#[derive(Default)] +struct State { + objects: HashMap, + xdg_surfaces: HashMap, + popups: HashSet, + /// Pending frame callbacks and the surfaces that requested them. + callbacks: HashMap, + armed: bool, + selected: Option, + held: Vec, + commits: u64, + delivered: u64, + error: Option, +} + +impl Connection { + fn new(client: UnixStream, server: UnixStream, schema: Arc) -> Self { + let mut state = State::default(); + state.objects.insert(1, "wl_display".to_owned()); + + Self { + client, + server, + schema, + state: Mutex::new(state), + } + } + + fn has_popups(&self) -> bool { + !lock(&self.state).popups.is_empty() + } + + fn shutdown(&self) { + let _ = self.client.shutdown(Shutdown::Both); + let _ = self.server.shutdown(Shutdown::Both); + } + + /// The socket a direction reads from, then the one it writes to. + fn endpoints(&self, direction: Direction) -> (&UnixStream, &UnixStream) { + match direction { + Direction::Request => (&self.client, &self.server), + Direction::Event => (&self.server, &self.client), + } + } + + /// Relays one direction until it ends. A source that closes, or a peer that + /// went away, passes the end on by half-closing the destination, so the + /// other direction still drains what is queued, such as the compositor's + /// protocol error before it disconnects, and GTK learns when Weston resets. + /// The half-close is harmless when the destination is the peer that left, as + /// when the test session is killed after a passing test. Any other failure + /// is a fixture or protocol error: the first one is printed and kept for the + /// control socket, and the whole connection closes, so GTK loses its display + /// and the test fails at once instead of stalling. + fn forward(&self, direction: Direction) { + let (_, destination) = self.endpoints(direction); + + match self.relay(direction) { + Ok(()) => { + let _ = destination.shutdown(Shutdown::Write); + } + Err(error) if is_disconnect(&error) => { + let _ = destination.shutdown(Shutdown::Write); + } + Err(error) => { + let mut state = lock(&self.state); + if state.error.is_none() { + let error = format!("{error:#}"); + eprintln!("Wayland proxy stopped relaying {direction:?}s: {error}"); + state.error = Some(error); + } + drop(state); + self.shutdown(); + } + } + } + + /// Passes every message on in order, except the ones `inspect` withholds. + /// The state lock is held while sending so a release cannot interleave. + fn relay(&self, direction: Direction) -> Result<()> { + let (source, destination) = self.endpoints(direction); + + while let Some(packet) = wire::receive(source)? { + let mut state = lock(&self.state); + if state.inspect(&self.schema, &packet, direction)? { + state.held.push(packet); + } else { + wire::send(destination, packet)?; + } + } + + Ok(()) + } + + fn control(&self, command: &str) -> ProxyResponse { + let mut state = lock(&self.state); + if let Some(error) = &state.error { + return ProxyResponse::error(error); + } + + match command { + "arm" => { + state.armed = true; + state.commits = 0; + state.delivered = 0; + } + "release" if state.held.is_empty() => { + return ProxyResponse::error("no popup callback held"); + } + "release" => { + if let Err(error) = self.release(&mut state) { + let error = format!("{error:#}"); + state.error = Some(error.clone()); + return ProxyResponse::error(error); + } + } + "status" => {} + _ => return ProxyResponse::error("unknown control command"), + } + + ProxyResponse::Status(ProxyStatus { + held: !state.held.is_empty(), + popup_commits: state.commits, + popup_callbacks_delivered: state.delivered, + }) + } + + /// Delivers the withheld events in their original order. + fn release(&self, state: &mut State) -> Result<()> { + state.selected = None; + for packet in std::mem::take(&mut state.held) { + state.inspect(&self.schema, &packet, Direction::Event)?; + wire::send(&self.client, packet)?; + } + + Ok(()) + } +} + +impl State { + /// Tracks the message and returns whether it must be withheld. + fn inspect(&mut self, schema: &Schema, packet: &Packet, direction: Direction) -> Result { + let object_id = packet.object_id(); + let Some(interface) = self + .objects + .get(&object_id) + .and_then(|name| schema.get(name.as_str()).copied()) + else { + return Ok(false); + }; + + let message = direction + .messages(interface) + .get(packet.opcode()) + .with_context(|| { + format!( + "{} has no {direction:?} opcode {}", + interface.name, + packet.opcode() + ) + })?; + let arguments = protocol::decode(message, packet.payload())?; + for (id, created) in &arguments.new_objects { + self.objects.insert(*id, created.clone()); + } + + match direction { + Direction::Request => { + self.track_request(interface.name, message.name, object_id, &arguments)?; + Ok(false) + } + Direction::Event => { + self.track_event(interface.name, message.name, object_id, &arguments) + } + } + } + + fn track_request( + &mut self, + interface: &str, + name: &str, + object_id: u32, + arguments: &Arguments, + ) -> Result<()> { + match (interface, name) { + // get_xdg_surface(id: new_id, surface: object) + ("xdg_wm_base", "get_xdg_surface") => { + self.xdg_surfaces + .insert(arguments.word(0)?, arguments.word(1)?); + } + ("xdg_surface", "get_popup") => { + let surface = self + .xdg_surfaces + .get(&object_id) + .with_context(|| format!("get_popup on unknown xdg_surface {object_id}"))?; + self.popups.insert(*surface); + } + // frame(callback: new_id) + ("wl_surface", "frame") => { + let callback = arguments.word(0)?; + self.callbacks.insert(callback, object_id); + if self.armed && self.popups.contains(&object_id) { + self.selected = Some(callback); + self.armed = false; + } + } + ("wl_surface", "commit") if self.popups.contains(&object_id) => { + self.commits += 1; + } + _ => {} + } + + Ok(()) + } + + fn track_event( + &mut self, + interface: &str, + name: &str, + object_id: u32, + arguments: &Arguments, + ) -> Result { + match (interface, name) { + ("wl_callback", "done") => { + if self.selected == Some(object_id) { + return Ok(true); + } + + let surface = self.callbacks.remove(&object_id); + if surface.is_some_and(|surface| self.popups.contains(&surface)) { + self.delivered += 1; + } + } + // delete_id(id: uint) + ("wl_display", "delete_id") => { + let id = arguments.word(0)?; + if self.selected == Some(id) { + return Ok(true); + } + + self.objects.remove(&id); + } + _ => {} + } + + Ok(false) + } +} + +/// Whether a relay stopped because a peer closed its end of the socket. +fn is_disconnect(error: &anyhow::Error) -> bool { + error.downcast_ref::().is_some_and(|error| { + matches!( + error.kind(), + io::ErrorKind::BrokenPipe + | io::ErrorKind::ConnectionAborted + | io::ErrorKind::ConnectionReset + | io::ErrorKind::NotConnected + ) + }) +} + +/// Relay threads report failures through `State::error`, so a poisoned lock +/// carries no extra information and cleanup must still proceed. +fn lock(mutex: &Mutex) -> MutexGuard<'_, T> { + mutex.lock().unwrap_or_else(PoisonError::into_inner) +} diff --git a/src/toolbar_gtk/view/capture_suppression/tests/wayland_proxy/wire.rs b/src/toolbar_gtk/view/capture_suppression/tests/wayland_proxy/wire.rs new file mode 100644 index 000000000..e0e3405b6 --- /dev/null +++ b/src/toolbar_gtk/view/capture_suppression/tests/wayland_proxy/wire.rs @@ -0,0 +1,202 @@ +//! Wayland stream framing. Descriptors stay with the message during whose +//! bytes they arrived and are passed on, then closed, when it is sent. + +use std::io; +use std::os::fd::{AsRawFd, FromRawFd, OwnedFd, RawFd}; +use std::os::unix::net::UnixStream; + +use anyhow::{Result, ensure}; + +const HEADER_LEN: usize = 8; +/// Ancillary space reserved for each read, in descriptors. +const RECEIVE_DESCRIPTORS: usize = 256; +// SAFETY: CMSG_SPACE only performs arithmetic on its argument. +const RECEIVE_CONTROL_LEN: usize = + unsafe { libc::CMSG_SPACE((RECEIVE_DESCRIPTORS * size_of::()) as u32) } as usize; + +pub(super) struct Packet { + data: Vec, + descriptors: Vec, +} + +impl Packet { + pub(super) fn object_id(&self) -> u32 { + word_at(&self.data, 0) + } + + pub(super) fn opcode(&self) -> usize { + (word_at(&self.data, 4) & 0xffff) as usize + } + + pub(super) fn payload(&self) -> &[u8] { + &self.data[HEADER_LEN..] + } +} + +fn word_at(data: &[u8], offset: usize) -> u32 { + let mut word = [0; 4]; + word.copy_from_slice(&data[offset..offset + 4]); + + u32::from_ne_bytes(word) +} + +/// Reads exactly one message, or `None` at end of stream. +pub(super) fn receive(socket: &UnixStream) -> Result> { + let mut data = Vec::with_capacity(HEADER_LEN); + let mut descriptors = Vec::new(); + let mut size = HEADER_LEN; + + while data.len() < size { + let filled = data.len(); + data.resize(size, 0); + let read = receive_chunk(socket, &mut data[filled..], &mut descriptors)?; + if read == 0 { + return Ok(None); + } + data.truncate(filled + read); + + if data.len() == HEADER_LEN { + size = (word_at(&data, 4) >> 16) as usize; + ensure!( + size >= HEADER_LEN && size.is_multiple_of(4), + "invalid Wayland message length" + ); + } + } + + Ok(Some(Packet { data, descriptors })) +} + +fn receive_chunk( + socket: &UnixStream, + buffer: &mut [u8], + descriptors: &mut Vec, +) -> Result { + let mut control = [0_u64; RECEIVE_CONTROL_LEN.div_ceil(size_of::())]; + let mut iovec = libc::iovec { + iov_base: buffer.as_mut_ptr().cast(), + iov_len: buffer.len(), + }; + // SAFETY: zero is a valid initial state for msghdr. + let mut message: libc::msghdr = unsafe { std::mem::zeroed() }; + message.msg_iov = &mut iovec; + message.msg_iovlen = 1; + message.msg_control = control.as_mut_ptr().cast(); + message.msg_controllen = RECEIVE_CONTROL_LEN; + + let read = loop { + // SAFETY: message references the live, writable data and control buffers. + let read = + unsafe { libc::recvmsg(socket.as_raw_fd(), &mut message, libc::MSG_CMSG_CLOEXEC) }; + if read >= 0 { + break read as usize; + } + + let error = io::Error::last_os_error(); + if error.kind() != io::ErrorKind::Interrupted { + return Err(error.into()); + } + }; + + // Own the descriptors first so they close if the read is rejected. + take_descriptors(&message, descriptors)?; + ensure!( + message.msg_flags & libc::MSG_CTRUNC == 0, + "truncated Wayland descriptors" + ); + + Ok(read) +} + +fn take_descriptors(message: &libc::msghdr, descriptors: &mut Vec) -> Result<()> { + // SAFETY: recvmsg filled msg_controllen bytes of the control buffer. + let mut header = unsafe { libc::CMSG_FIRSTHDR(message) }; + while !header.is_null() { + // SAFETY: CMSG_FIRSTHDR/CMSG_NXTHDR return only complete headers. + let (level, kind, length) = unsafe { + ( + (*header).cmsg_level, + (*header).cmsg_type, + (*header).cmsg_len, + ) + }; + + if level == libc::SOL_SOCKET && kind == libc::SCM_RIGHTS { + // SAFETY: CMSG_LEN only performs arithmetic on its argument. + let bytes = length - unsafe { libc::CMSG_LEN(0) } as usize; + // SAFETY: the kernel wrote `bytes` bytes of descriptors after the header. + let data = unsafe { libc::CMSG_DATA(header) }.cast::(); + for index in 0..bytes / size_of::() { + // SAFETY: each received descriptor is open and owned by nobody else. + descriptors.push(unsafe { OwnedFd::from_raw_fd(data.add(index).read_unaligned()) }); + } + ensure!( + bytes.is_multiple_of(size_of::()), + "misaligned Wayland descriptors" + ); + } + + // SAFETY: header is a header of this message's control buffer. + header = unsafe { libc::CMSG_NXTHDR(message, header) }; + } + + Ok(()) +} + +/// Writes the whole message with its descriptors attached to the first chunk. +pub(super) fn send(socket: &UnixStream, packet: Packet) -> Result<()> { + let mut sent = send_chunk(socket, &packet.data, &packet.descriptors)?; + while sent < packet.data.len() { + sent += send_chunk(socket, &packet.data[sent..], &[])?; + } + + Ok(()) +} + +fn send_chunk(socket: &UnixStream, bytes: &[u8], descriptors: &[OwnedFd]) -> Result { + let raw: Vec = descriptors.iter().map(AsRawFd::as_raw_fd).collect(); + // SAFETY: CMSG_SPACE only performs arithmetic on its argument. + let control_len = unsafe { libc::CMSG_SPACE(size_of_val(raw.as_slice()) as u32) } as usize; + let mut control = vec![0_u64; control_len.div_ceil(size_of::())]; + let mut iovec = libc::iovec { + iov_base: bytes.as_ptr().cast_mut().cast(), + iov_len: bytes.len(), + }; + // SAFETY: zero is a valid initial state for msghdr. + let mut message: libc::msghdr = unsafe { std::mem::zeroed() }; + message.msg_iov = &mut iovec; + message.msg_iovlen = 1; + + if !raw.is_empty() { + message.msg_control = control.as_mut_ptr().cast(); + message.msg_controllen = control_len; + // SAFETY: the control buffer holds CMSG_SPACE bytes for these descriptors. + unsafe { + let header = libc::CMSG_FIRSTHDR(&message); + (*header).cmsg_level = libc::SOL_SOCKET; + (*header).cmsg_type = libc::SCM_RIGHTS; + (*header).cmsg_len = libc::CMSG_LEN(size_of_val(raw.as_slice()) as u32) as usize; + std::ptr::copy_nonoverlapping( + raw.as_ptr(), + libc::CMSG_DATA(header).cast::(), + raw.len(), + ); + } + } + + loop { + // SAFETY: message references the live data and control buffers. + let sent = unsafe { libc::sendmsg(socket.as_raw_fd(), &message, libc::MSG_NOSIGNAL) }; + if sent > 0 { + return Ok(sent as usize); + } + + let error = match sent { + 0 => io::Error::from(io::ErrorKind::WriteZero), + _ => io::Error::last_os_error(), + }; + if error.kind() != io::ErrorKind::Interrupted { + return Err(error.into()); + } + } +} diff --git a/tests/AGENTS.md b/tests/AGENTS.md index 9a08d649f..35f7b6674 100644 --- a/tests/AGENTS.md +++ b/tests/AGENTS.md @@ -6,6 +6,7 @@ ## Architecture - `tests/cli.rs` covers CLI behavior. - `tests/ui.rs` covers UI smoke/integration behavior that can run without an ungated visible overlay. +- `tests/repository_guards/` holds the source guards (process sites, config writers, shared dependencies, no Python). They read the checkout, not compiled items, and each guard keeps regression cases showing its forbidden escapes fail: most edit a copy of the checked-out `source::Tree`, the shared-dependency corpus builds small trees from `shared_dependency_fixtures.json`, and `no_python.rs` audits small in-memory file sets. ## Invariants - Tests should not launch visible Wayland overlays, steal focus, or interact with foreground/fullscreen apps by default. @@ -13,6 +14,7 @@ - Keep output assertions specific enough to catch regressions without depending on noisy logs. ## Coupled Changes +- A new process site, config writer, or shared-layer path must satisfy `tests/repository_guards` or update the guard's reviewed list with its reason. - CLI changes may require `tests/cli.rs`, docs, and usage text updates. - UI or rendering smoke changes may require fixtures or focused module tests in `src/`. diff --git a/tests/repository_guards/config_writers.rs b/tests/repository_guards/config_writers.rs new file mode 100644 index 000000000..095aac25b --- /dev/null +++ b/tests/repository_guards/config_writers.rs @@ -0,0 +1,1289 @@ +//! Production code outside the reviewed writers cannot write `config.toml`. +//! +//! `config.toml` is an authored input. It changes only through an explicit +//! user edit action, never as a side effect of running Wayscriber. Two kinds of +//! writer are allowed: the configurator's **Save**, which writes the whole +//! edited draft; and the overlay's **narrow editors** in `src/config/io.rs`, one +//! per explicit gesture, each of which rewrites only its own key and backs the +//! file up first. Everything else reads the file and never writes it. The +//! capability is one `use` away, so this checks for its absence. +//! +//! Six things are enforced: +//! +//! 1. The write primitives are named nowhere outside `src/config/` and the +//! configurator's Save adapter. +//! 2. Each narrow editor's production call sites are pinned by name in +//! [`NARROW_WRITERS`]; a new caller is a new place the file can change from. +//! 3. The write-capable surface of `document.rs` and `io.rs` is pinned. Both are +//! exempt from the primitive scan because they *are* the write, so every +//! exported function that can reach a write, directly or through the file's +//! private helpers, must be listed. In those files and `mod.rs`, a write +//! reachable through a trait fails whatever its name and visibility, and so +//! does a trait declaring a method that can write. +//! 4. `src/config/mod.rs` is a re-export list: a writer's name may appear only +//! inside a `use` item, and no function declared there may reach a writer. +//! 5. Names are pinned. The config module re-exports editors but never a +//! primitive; no production file renames a write-capable name on import or +//! re-export, or casts one with `as`; inside `src/config/` the editors are +//! named only in `io.rs` and `mod.rs`; and the editors' path-taking twins are +//! named in production nowhere. +//! 6. A write may not leave those files as a value: a write-capable name in a +//! `const` or `static` initializer fails, and a `pub use` in `src/config/` +//! fails when it re-exports a `fn`, `const`, or `static` declared in +//! `document.rs` or `io.rs` other than the reviewed editors. Types are +//! deliberately outside this rule. +//! +//! Scope and limits: a name-level guardrail over `src/` and +//! `configurator/src/`, not a proof. It cannot catch a brand-new write built +//! directly on `durable_io::write_text_atomic` under a different name, a macro +//! defined in `src/config/` that expands to a write, a composed identifier, or a +//! writer stored in a value at runtime and consumed within one writer file's +//! private functions. The behavioural proof is the loader immutability fixture +//! in `src/config/tests/immutability.rs` plus the per-flow "only this key +//! changed" tests beside each gesture. Test sources are exempt, and whether a +//! file is one is read from the `#[cfg(test)]` on the `mod` item that brings it +//! in, not from the shape of its path. + +use std::collections::{BTreeMap, BTreeSet, HashMap}; +use std::path::{Path, PathBuf}; + +use super::source::{ + CFG_TEST, Cursor, TestSources, Tree, block_end, is_word_char, item_end, line_of, line_starts, + mask_non_code, within, word_hits, word_index, words, +}; + +const DOCUMENT_SOURCE: &str = "src/config/document.rs"; +const IO_SOURCE: &str = "src/config/io.rs"; +const CONFIG_MODULE: &str = "src/config/mod.rs"; +const CONFIGURATOR_ADAPTER: &str = "configurator/src/app/io.rs"; + +/// The files that implement the single durable write, plus the configurator +/// adapter that performs it. `src/config/document/merge.rs` rewrites a TOML +/// tree in memory and never touches the filesystem, so it is not here. +const WRITE_ALLOWLIST: [&str; 3] = [DOCUMENT_SOURCE, IO_SOURCE, CONFIGURATOR_ADAPTER]; + +/// Every name that reaches the filesystem on the config path. +const WRITE_PRIMITIVES: [&str; 4] = [ + "save_with_backup", + "write_config_text_atomic", + "create_config_backup", + "prepare_config_parent", +]; + +/// The whole of the application's durable config write. What is pinned is +/// reachability, not spelling: anything that reaches `merge_and_write` writes. +const DOCUMENT_WRITE_SURFACE: [&str; 1] = ["save_with_backup"]; +const DOCUMENT_WRITE_STEP: &str = "merge_and_write"; + +/// The three narrow editors, their `#[cfg(test)]` path-taking twins, and the +/// primitives, which `pub(super)` confines to the config module. +const IO_EDITOR_SURFACE: [&str; 6] = [ + "persist_keybinding_edit", + "persist_keybinding_edit_at", + "persist_preset_slot", + "persist_preset_slot_at", + "persist_quick_color", + "persist_quick_color_at", +]; + +/// The overlay's config-edit worker calls every editor, off the dispatch thread. +const EDIT_WORKER: &str = "src/backend/wayland/config_edits.rs"; + +/// One entry per explicit user gesture that may change `config.toml`, mapped +/// to the production files allowed to invoke it. Adding a caller widens where +/// the file can change from; it needs an explicit gesture, an honest +/// in-memory fallback on failure, and an "only this key changed" test. +const NARROW_WRITERS: [(&str, &[&str]); 3] = [ + ("persist_keybinding_edit", &[EDIT_WORKER]), + ("persist_preset_slot", &[EDIT_WORKER]), + ("persist_quick_color", &[EDIT_WORKER]), +]; + +/// Where the editors may be named inside the config module. +const EDITOR_HOME: [&str; 2] = [IO_SOURCE, CONFIG_MODULE]; + +/// Sources whose former write authority this replaces. If the walk stops +/// reaching them, it proves nothing. +const EXPECTED_SCANNED: [&str; 11] = [ + DOCUMENT_SOURCE, + IO_SOURCE, + CONFIGURATOR_ADAPTER, + "src/backend/wayland/state.rs", + "src/daemon/tray/runtime.rs", + "src/backend/wayland/backend/state_init/config.rs", + "configurator/src/app/update/config.rs", + EDIT_WORKER, + "src/backend/wayland/state/keybindings.rs", + "src/backend/wayland/state/toolbar/events/presets.rs", + "src/backend/wayland/state/toolbar/events/quick_colors.rs", +]; + +/// Names left in a `use` item that are not names it imports. +const USE_KEYWORDS: [&str; 7] = ["use", "pub", "crate", "self", "super", "as", "in"]; + +type Spans = Vec<(usize, usize)>; + +fn narrow_writer_names() -> impl Iterator { + NARROW_WRITERS.iter().map(|(name, _)| *name) +} + +fn path_taking_writers() -> BTreeSet { + narrow_writer_names() + .map(|name| format!("{name}_at")) + .collect() +} + +fn io_write_surface() -> BTreeSet<&'static str> { + IO_EDITOR_SURFACE + .into_iter() + .chain(WRITE_PRIMITIVES) + .collect() +} + +fn names(items: impl IntoIterator) -> BTreeSet { + items.into_iter().map(str::to_owned).collect() +} + +/// Ranges covered by inline `#[cfg(test)]` items, and any tracking failure. +fn cfg_test_spans(masked: &str) -> (Spans, Vec) { + let mut spans = Vec::new(); + let mut problems = Vec::new(); + + for (start, _) in masked.match_indices(CFG_TEST) { + let after = start + CFG_TEST.len(); + match masked[after..] + .find(['{', ';']) + .map(|offset| after + offset) + { + None => problems.push("a `#[cfg(test)]` item has neither a body nor a `;`".into()), + Some(index) if masked.as_bytes()[index] == b';' => {} + Some(index) => match block_end(masked, index) { + Some(end) => spans.push((start, end)), + None => problems.push("a `#[cfg(test)]` block never closes".into()), + }, + } + } + + (spans, problems) +} + +struct FunctionItem { + name: String, + visibility: Option, + start: usize, + end: usize, +} + +struct ConstItem { + keyword: String, + name: String, + start: usize, + end: usize, +} + +struct UseItem { + start: usize, + end: usize, + exported: bool, +} + +/// Every function with a body outside `exclude`, at any indentation and +/// visibility, including methods. +fn function_items(masked: &str, exclude: &[(usize, usize)]) -> Vec { + let mut items = Vec::new(); + + for start in line_starts(masked) { + if within(exclude, start) { + continue; + } + + let mut cursor = Cursor::new(masked, start); + cursor.skip_blanks(); + let visibility = cursor.visibility().map(str::to_owned); + cursor.function_qualifiers(); + if !cursor.keyword_then_space("fn") { + continue; + } + let Some(name) = cursor.word() else { + continue; + }; + + let Some(opening) = masked[cursor.at..] + .find(['{', ';']) + .map(|offset| cursor.at + offset) + else { + continue; + }; + if masked.as_bytes()[opening] == b';' { + continue; + } + if let Some(end) = block_end(masked, opening) { + items.push(FunctionItem { + name: name.to_owned(), + visibility, + start, + end, + }); + } + } + + items +} + +/// Every `const` or `static` item outside `exclude`, including associated ones. +fn const_items(masked: &str, exclude: &[(usize, usize)]) -> Vec { + let mut items = Vec::new(); + + for start in line_starts(masked) { + if within(exclude, start) { + continue; + } + + let mut cursor = Cursor::new(masked, start); + cursor.skip_blanks(); + cursor.visibility(); + let keyword = if cursor.keyword_then_space("const") { + "const" + } else if cursor.keyword_then_space("static") { + "static" + } else { + continue; + }; + cursor.keyword_then_space("mut"); + let Some(name) = cursor.word() else { + continue; + }; + cursor.skip_whitespace(); + if !cursor.literal(":") { + continue; + } + + if let Some(end) = item_end(masked, cursor.at) { + items.push(ConstItem { + keyword: keyword.to_owned(), + name: name.to_owned(), + start, + end, + }); + } + } + + items +} + +fn use_items(masked: &str) -> Vec { + let mut items = Vec::new(); + + for start in line_starts(masked) { + let mut cursor = Cursor::new(masked, start); + cursor.skip_blanks(); + let exported = cursor.visibility().is_some(); + if !cursor.keyword("use") { + continue; + } + + if let Some(semicolon) = masked[cursor.at..].find(';') { + items.push(UseItem { + start, + end: cursor.at + semicolon + 1, + exported, + }); + } + } + + items +} + +/// The names a `use` item brings into scope, without their paths. +fn use_leaf_names(text: &str) -> BTreeSet { + words(text) + .filter(|(offset, word)| { + let after = text[offset + word.len()..].trim_start(); + !after.starts_with("::") && !USE_KEYWORDS.contains(word) + }) + .map(|(_, word)| word.to_owned()) + .collect() +} + +/// `name` followed by `as alias` inside `text`. +fn renamed_alias<'a>(text: &'a str, name: &str) -> Option<&'a str> { + word_hits(text, name).find_map(|offset| { + let mut cursor = Cursor::new(text, offset + name.len()); + (cursor.skip_whitespace() > 0 && cursor.keyword_then_space("as")) + .then(|| cursor.word()) + .flatten() + }) +} + +/// Names that reach one of `seeds`, directly or through this file's functions. +fn write_capable_functions( + masked: &str, + items: &[FunctionItem], + seeds: &BTreeSet, +) -> BTreeSet { + let mut references: BTreeMap<&str, BTreeSet<&str>> = BTreeMap::new(); + for item in items { + references.insert( + &item.name, + words(&masked[item.start..item.end]) + .map(|(_, word)| word) + .collect(), + ); + } + + let mut capable: BTreeSet = references + .iter() + .filter(|(_, tokens)| tokens.iter().any(|token| seeds.contains(*token))) + .map(|(name, _)| (*name).to_owned()) + .collect(); + loop { + let reached: Vec = references + .iter() + .filter(|(name, tokens)| { + !capable.contains(**name) && tokens.iter().any(|token| capable.contains(*token)) + }) + .map(|(name, _)| (*name).to_owned()) + .collect(); + if reached.is_empty() { + return capable; + } + capable.extend(reached); + } +} + +/// `impl Trait for Type` headers (not an inherent impl, not `for<'a>`), and +/// `trait` declarations: where a `fn` is callable without a `pub` of its own. +fn trait_spans(masked: &str, exclude: &[(usize, usize)]) -> Spans { + let mut spans = Vec::new(); + + for start in line_starts(masked) { + let mut cursor = Cursor::new(masked, start); + cursor.skip_blanks(); + cursor.keyword_then_space("unsafe"); + if !cursor.keyword("impl") { + continue; + } + let Some((header, opening)) = header_until_brace(masked, cursor.at) else { + continue; + }; + if is_trait_impl(header) + && !within(exclude, start) + && let Some(end) = block_end(masked, opening) + { + spans.push((start, end)); + } + } + + for declaration in trait_declarations(masked, exclude) { + spans.push((declaration.start, declaration.end)); + } + + spans +} + +/// Text up to the first `{`, unless a `;` comes first, like `[^{;]*\{`. +fn header_until_brace(masked: &str, from: usize) -> Option<(&str, usize)> { + let offset = masked[from..].find(['{', ';'])?; + let opening = from + offset; + (masked.as_bytes()[opening] == b'{').then(|| (&masked[from..opening], opening)) +} + +fn is_trait_impl(header: &str) -> bool { + word_hits(header, "for").any(|offset| !header[offset + 3..].trim_start().starts_with('<')) +} + +struct TraitDeclaration { + name: String, + start: usize, + end: usize, + methods: BTreeSet, +} + +fn trait_declarations(masked: &str, exclude: &[(usize, usize)]) -> Vec { + let mut declarations = Vec::new(); + + for start in line_starts(masked) { + if within(exclude, start) { + continue; + } + + let mut cursor = Cursor::new(masked, start); + cursor.skip_blanks(); + cursor.visibility(); + cursor.keyword_then_space("unsafe"); + if !cursor.keyword_then_space("trait") { + continue; + } + let Some(name) = cursor.word() else { + continue; + }; + let Some((_, opening)) = header_until_brace(masked, cursor.at) else { + continue; + }; + let Some(end) = block_end(masked, opening) else { + continue; + }; + + declarations.push(TraitDeclaration { + name: name.to_owned(), + start, + end, + methods: trait_methods(&masked[opening + 1..end]), + }); + } + + declarations +} + +/// Method signatures in a trait body, with or without bodies. +fn trait_methods(body: &str) -> BTreeSet { + line_starts(body) + .filter_map(|start| { + let mut cursor = Cursor::new(body, start); + cursor.skip_blanks(); + cursor.function_qualifiers(); + cursor + .keyword_then_space("fn") + .then(|| cursor.word().map(str::to_owned)) + .flatten() + }) + .collect() +} + +struct Source<'a> { + path: &'a str, + text: &'a str, + masked: String, + test_spans: Spans, +} + +impl<'a> Source<'a> { + fn read(tree: &'a Tree, path: &'a str) -> (Self, Vec) { + let text = tree + .read(Path::new(path)) + .unwrap_or_else(|| panic!("{path} is missing")); + let masked = mask_non_code(text); + let (test_spans, problems) = cfg_test_spans(&masked); + let problems = problems + .into_iter() + .map(|problem| format!("{path}: {problem}")) + .collect(); + + ( + Self { + path, + text, + masked, + test_spans, + }, + problems, + ) + } + + fn line(&self, offset: usize) -> usize { + line_of(self.text, offset) + } +} + +struct Audit<'a> { + tree: &'a Tree, + failures: Vec, + test_sources: TestSources, +} + +impl<'a> Audit<'a> { + fn fail(&mut self, failure: String) { + self.failures.push(failure); + } + + fn is_test(&mut self, path: &Path) -> bool { + self.test_sources.is_test(self.tree, path) + } + + fn sites(&mut self) -> BTreeSet { + let mut scanned = BTreeSet::new(); + let twins = path_taking_writers(); + let editors: Vec = narrow_writer_names() + .map(str::to_owned) + .chain(twins.iter().cloned()) + .collect(); + let mut callers: HashMap> = editors + .iter() + .map(|name| (name.clone(), BTreeSet::new())) + .collect(); + let tracked: Vec<&str> = editors + .iter() + .map(String::as_str) + .chain(WRITE_PRIMITIVES) + .collect(); + let paths: Vec = self + .tree + .paths_under("src") + .chain(self.tree.paths_under("configurator/src")) + .map(Path::to_path_buf) + .collect(); + + for path in paths { + scanned.insert(path.clone()); + let display = path.to_string_lossy().into_owned(); + let text = self.tree.read(&path).expect("listed path"); + let masked = mask_non_code(text); + let (spans, problems) = cfg_test_spans(&masked); + let is_test = self.is_test(&path); + let hits = word_index(&masked, &tracked); + if !is_test { + for problem in problems { + self.fail(format!("{display}: {problem}")); + } + self.renamed_imports(&display, text, &masked, &spans); + self.writer_casts(&display, text, &masked, &spans, &hits); + } + + let production_hits = |name: &str| { + hits.get(name) + .into_iter() + .flatten() + .copied() + .filter(|hit| !within(&spans, *hit)) + }; + let in_config_module = path.starts_with("src/config"); + let in_editor_home = EDITOR_HOME.iter().any(|home| path == Path::new(home)); + for name in &editors { + for hit in production_hits(name) { + if is_test { + continue; + } + if !in_config_module { + callers.get_mut(name).expect("editor").insert(path.clone()); + } else if !in_editor_home { + self.fail(format!( + "{display}:{}: names the narrow config writer `{name}` inside the \ + config module; the editors are declared in io.rs and leave through \ + mod.rs, so a wrapper here would carry the capability out under a \ + name the call-site pins never see", + line_of(text, hit) + )); + } + } + } + + let allowed = WRITE_ALLOWLIST + .iter() + .any(|allowed| path == Path::new(allowed)); + if allowed || is_test { + continue; + } + let lines: Vec<&str> = text.lines().collect(); + for primitive in WRITE_PRIMITIVES { + for hit in production_hits(primitive) { + let number = line_of(text, hit); + let line = lines.get(number - 1).map_or("", |line| line.trim()); + self.fail(format!( + "{display}:{number}: config write capability `{primitive}` outside the \ + reviewed writers: {line}" + )); + } + } + } + + for (name, expected) in NARROW_WRITERS { + let expected: BTreeSet = expected.iter().map(PathBuf::from).collect(); + let found = &callers[name]; + for unexpected in found.difference(&expected) { + self.fail(format!( + "{}: unreviewed caller of the narrow config writer `{name}`; record it in \ + NARROW_WRITERS if this gesture should be able to change config.toml", + unexpected.display() + )); + } + for missing in expected.difference(found) { + self.fail(format!( + "{}: expected to call `{name}` but does not; the pinned call site moved, \ + so this check no longer describes the code", + missing.display() + )); + } + } + for name in &twins { + for caller in &callers[name] { + self.fail(format!( + "{}: production code names `{name}`; the path-taking twins take the file \ + to write from their caller and exist for the suites, so a gesture that \ + needs one is a new writer to review, not an implementation detail", + caller.display() + )); + } + } + + scanned + } + + /// Every `use ... as ...` renaming a name that can write the file: a `pub + /// use` rename hands the capability on under an unpinned name, and a plain + /// rename conceals the call in the file that makes it. + fn renamed_imports(&mut self, display: &str, text: &str, masked: &str, spans: &Spans) { + for item in use_items(masked) { + if within(spans, item.start) { + continue; + } + + let item_text = &masked[item.start..item.end]; + for name in io_write_surface() { + let Some(alias) = renamed_alias(item_text, name) else { + continue; + }; + let verb = if item.exported { + "re-exports" + } else { + "imports" + }; + self.fail(format!( + "{display}:{}: {verb} the config writer `{name}` as `{alias}`; the writers \ + are pinned by name, so they travel under their own or not at all", + line_of(text, item.start) + )); + } + } + } + + /// `writer as T` outside a `use` item, such as a cast to a function pointer, + /// turns the writer into a value that the name pins cannot follow. + fn writer_casts( + &mut self, + display: &str, + text: &str, + masked: &str, + spans: &Spans, + hits: &HashMap<&str, Vec>, + ) { + let uses: Spans = use_items(masked) + .iter() + .map(|item| (item.start, item.end)) + .collect(); + + for name in io_write_surface() { + for &hit in hits.get(name).into_iter().flatten() { + if within(spans, hit) || within(&uses, hit) { + continue; + } + + let mut cursor = Cursor::new(masked, hit + name.len()); + if cursor.skip_whitespace() > 0 && cursor.keyword_then_space("as") { + self.fail(format!( + "{display}:{}: casts the config writer `{name}` with `as`; a writer \ + turned into a value travels under no name, so the writers are only \ + ever called by name", + line_of(text, hit) + )); + } + } + } + } + + /// No write leaves an implementing file as a value. + fn const_initializers(&mut self, source: &Source, capable: &BTreeSet) { + for item in const_items(&source.masked, &source.test_spans) { + let named: BTreeSet<&str> = words(&source.masked[item.start..item.end]) + .map(|(_, word)| word) + .filter(|word| capable.contains(*word)) + .collect(); + for name in named { + self.fail(format!( + "{}:{}: `{} {}` names `{name}`, which can write config.toml; a writer \ + stored as a value declares no function for the surface pins to read, so \ + the writers here stay functions", + source.path, + source.line(item.start), + item.keyword, + item.name + )); + } + } + } + + /// No write leaves an implementing file through a trait. + fn trait_writers( + &mut self, + source: &Source, + items: &[FunctionItem], + capable: &BTreeSet, + ) { + let spans = trait_spans(&source.masked, &source.test_spans); + for item in items { + if capable.contains(&item.name) && within(&spans, item.start) { + self.fail(format!( + "{}:{}: `fn {}` can write config.toml from inside a trait; a trait's \ + methods are callable wherever the trait is and carry no visibility of \ + their own, so the writers here stay inherent or free functions", + source.path, + source.line(item.start), + item.name + )); + } + } + self.trait_declarations(source, capable); + } + + /// A trait declared here may not name a method that can write, even + /// without a body: the offer itself is the capability. + fn trait_declarations(&mut self, source: &Source, capable: &BTreeSet) { + for declaration in trait_declarations(&source.masked, &source.test_spans) { + for method in declaration.methods.intersection(capable) { + self.fail(format!( + "{}:{}: `trait {}` declares `{method}`, which can write config.toml; a \ + trait carries the capability to every caller that can name it, so the \ + writers here are not offered through one", + source.path, + source.line(declaration.start), + declaration.name + )); + } + } + } + + fn io_write_surface(&mut self, source: &Source) { + let items = function_items(&source.masked, &source.test_spans); + if !items + .iter() + .any(|item| item.name == "persist_keybinding_edit") + { + self.fail(format!( + "{IO_SOURCE}: the function scan found no narrow editor; its shape assumption \ + about the file no longer holds" + )); + return; + } + + let primitives = names(WRITE_PRIMITIVES); + let capable = write_capable_functions(&source.masked, &items, &primitives); + self.trait_writers(source, &items, &capable); + self.const_initializers(source, &capable.union(&primitives).cloned().collect()); + + let surface = io_write_surface(); + for item in &items { + let Some(visibility) = &item.visibility else { + continue; + }; + if capable.contains(&item.name) && !surface.contains(item.name.as_str()) { + self.fail(format!( + "{IO_SOURCE}:{}: `{visibility} fn {}` can write config.toml but is not one \ + of the reviewed editors; record it in IO_WRITE_SURFACE if this is a new \ + user gesture", + source.line(item.start), + item.name + )); + } + } + } + + fn document_write_surface(&mut self, source: &Source) { + let items = function_items(&source.masked, &source.test_spans); + let seeds: BTreeSet = names(WRITE_PRIMITIVES) + .into_iter() + .chain([DOCUMENT_WRITE_STEP.to_owned()]) + .collect(); + if !items.iter().any(|item| item.name == "save_with_backup") { + self.fail(format!( + "{DOCUMENT_SOURCE}: the function scan found no document save; its shape \ + assumption about the file no longer holds" + )); + return; + } + if !items.iter().any(|item| seeds.contains(&item.name)) { + self.fail(format!( + "{DOCUMENT_SOURCE}: the function scan found none of the write steps ({}); its \ + shape assumption about the file no longer holds", + seeds.iter().cloned().collect::>().join(", ") + )); + return; + } + + let capable = write_capable_functions(&source.masked, &items, &seeds); + self.trait_writers(source, &items, &capable); + self.const_initializers(source, &capable.union(&seeds).cloned().collect()); + + for item in &items { + let Some(visibility) = &item.visibility else { + continue; + }; + if capable.contains(&item.name) && !DOCUMENT_WRITE_SURFACE.contains(&item.name.as_str()) + { + self.fail(format!( + "{DOCUMENT_SOURCE}:{}: `{visibility} fn {}` can write config.toml but is \ + not the reviewed document save; the application has exactly one durable \ + writer, so record it in DOCUMENT_WRITE_SURFACE only if that changed", + source.line(item.start), + item.name + )); + } + } + } + + /// `src/config/mod.rs` names the writers only where it re-exports them. + fn config_module_surface(&mut self) { + let (source, problems) = Source::read(self.tree, CONFIG_MODULE); + self.failures.extend(problems); + let use_spans: Spans = use_items(&source.masked) + .iter() + .map(|item| (item.start, item.end)) + .collect(); + let items = function_items(&source.masked, &source.test_spans); + let surface: BTreeSet = io_write_surface().into_iter().map(str::to_owned).collect(); + + for name in &surface { + for hit in word_hits(&source.masked, name) { + if within(&source.test_spans, hit) || within(&use_spans, hit) { + continue; + } + let place = enclosing_function(&items, hit).map_or_else( + || "outside any `use` item".to_owned(), + |item| format!("inside `fn {}`", item.name), + ); + self.fail(format!( + "{CONFIG_MODULE}:{}: names the config writer `{name}` {place}; this file \ + re-exports the editors and does nothing else, so anything here that can \ + call one carries the capability out under a name the call-site pins never \ + see", + source.line(hit) + )); + } + } + + let capable = write_capable_functions(&source.masked, &items, &surface); + for item in &items { + if capable.contains(&item.name) { + self.fail(format!( + "{CONFIG_MODULE}:{}: `fn {}` can reach a config writer; the config \ + module's own file declares no functions, so this is an unreviewed writer \ + with a name of its own", + source.line(item.start), + item.name + )); + } + } + self.trait_declarations(&source, &capable); + } + + /// The config module re-exports the editors, never a primitive, and never + /// any other `fn`, `const`, or `static` declared in a writer file. + fn module_reexports(&mut self) { + let values = writer_value_items(self.tree); + let mut paths: Vec = self + .tree + .paths_under("src/config") + .map(Path::to_path_buf) + .collect(); + paths.retain(|path| !self.is_test(path)); + paths.sort_by_key(|path| (path != Path::new(CONFIG_MODULE), path.clone())); + + for path in paths { + let display = path.to_string_lossy().into_owned(); + let (source, _) = Source::read(self.tree, &display); + for item in use_items(&source.masked) { + if within(&source.test_spans, item.start) || !item.exported { + continue; + } + + let item_text = &source.masked[item.start..item.end]; + let line = source.line(item.start); + for primitive in WRITE_PRIMITIVES { + if word_hits(item_text, primitive).next().is_some() { + self.fail(format!( + "{display}:{line}: re-exports the write primitive `{primitive}`; \ + the primitives stay inside the config module and only the narrow \ + editors leave it" + )); + } + } + for name in use_leaf_names(item_text) { + if narrow_writer_names().any(|editor| editor == name) { + continue; + } + let Some((declared, keyword)) = values.get(&name) else { + continue; + }; + self.fail(format!( + "{display}:{line}: re-exports `{name}`, a `{keyword}` declared in \ + {declared}; the write lives in that file, and the surface pins there \ + read functions — so a value leaving it carries whatever it holds past \ + them. Only the reviewed editors leave the module" + )); + } + } + } + } + + /// The implementing files may not widen the capability they own. + fn write_surface(&mut self) { + let (document, problems) = Source::read(self.tree, DOCUMENT_SOURCE); + self.failures.extend(problems); + self.document_write_surface(&document); + if !document.text.contains("pub fn save_with_backup") { + self.fail(format!( + "{DOCUMENT_SOURCE}: `save_with_backup` is gone or renamed; this check no longer \ + describes the code" + )); + } + + let (io, problems) = Source::read(self.tree, IO_SOURCE); + for primitive in [ + "create_config_backup", + "write_config_text_atomic", + "prepare_config_parent", + ] { + if !io.text.contains(&format!("pub(super) fn {primitive}")) { + self.fail(format!( + "{IO_SOURCE}: `{primitive}` is no longer `pub(super)`; the write primitives \ + must stay inside the config module" + )); + } + } + // Each editor builds its update on `document.config()`; basing it on + // `authored_config()` would hand the merge gate every clamped value. + for name in narrow_writer_names() { + if !io.text.contains(&format!("pub fn {name}")) { + self.fail(format!( + "{IO_SOURCE}: narrow config writer `{name}` is gone or no longer declared \ + here; this check no longer describes the code" + )); + } + } + for name in path_taking_writers() { + if !has_cfg_test_twin(io.text, &name) { + self.fail(format!( + "{IO_SOURCE}: `{name}` is no longer a `#[cfg(test)] pub(crate) fn`; the \ + path-taking twins exist for the suites, and an ungated one is a config \ + write at a caller-chosen path available to the whole crate" + )); + } + } + self.failures.extend(problems); + for hit in word_hits(&io.masked, "authored_config") { + if !within(&io.test_spans, hit) { + self.fail(format!( + "{IO_SOURCE}:{}: a narrow writer reads `authored_config()`; the edit base \ + must be `document.config()` so the merge gate writes only the edited key", + io.line(hit) + )); + } + } + + self.io_write_surface(&io); + self.config_module_surface(); + self.module_reexports(); + } +} + +/// `#[cfg(test)]`, whitespace, then `pub(crate) fn name` ending at a word boundary. +fn has_cfg_test_twin(text: &str, name: &str) -> bool { + let declaration = format!("pub(crate) fn {name}"); + + text.match_indices(CFG_TEST).any(|(start, _)| { + let rest = text[start + CFG_TEST.len()..].trim_start(); + rest.starts_with(&declaration) + && !rest[declaration.len()..] + .chars() + .next() + .is_some_and(is_word_char) + }) +} + +fn enclosing_function(items: &[FunctionItem], offset: usize) -> Option<&FunctionItem> { + items + .iter() + .filter(|item| item.start <= offset && offset < item.end) + .max_by_key(|item| item.start) +} + +/// Every `fn`, `const`, and `static` declared in the two writer files: where a +/// name came from and as what, which a value cannot disguise. +fn writer_value_items(tree: &Tree) -> HashMap { + let mut items = HashMap::new(); + + for path in [DOCUMENT_SOURCE, IO_SOURCE] { + let (source, _) = Source::read(tree, path); + for function in function_items(&source.masked, &source.test_spans) { + items + .entry(function.name) + .or_insert((path, "fn".to_owned())); + } + for constant in const_items(&source.masked, &source.test_spans) { + items.insert(constant.name, (path, constant.keyword)); + } + } + + items +} + +fn audit(tree: &Tree) -> Vec { + let mut audit = Audit { + tree, + failures: Vec::new(), + test_sources: TestSources::default(), + }; + + let scanned = audit.sites(); + let missing: Vec<&str> = EXPECTED_SCANNED + .into_iter() + .filter(|path| !scanned.contains(Path::new(path))) + .collect(); + if !missing.is_empty() { + audit.fail(format!( + "the walk missed expected sources, so it proves nothing: {}", + missing.join(", ") + )); + } + audit.write_surface(); + + audit.failures +} + +fn checkout() -> Tree { + Tree::checkout(&["src", "configurator/src"]) +} + +fn assert_fails(tree: Tree, expected: &str) { + let failures = audit(&tree); + + assert!( + failures.iter().any(|failure| failure.contains(expected)), + "expected a failure containing {expected:?}, got {failures:#?}" + ); +} + +#[test] +fn only_the_reviewed_writers_can_write_config_toml() { + let failures = audit(&checkout()); + + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +#[test] +fn a_primitive_outside_the_reviewed_writers_fails() { + assert_fails( + checkout().appending( + "src/daemon/core.rs", + "\nfn leak(document: &ConfigDocument) { document.save_with_backup(config()); }\n", + ), + "config write capability `save_with_backup` outside the reviewed writers", + ); +} + +#[test] +fn only_a_cfg_test_module_makes_a_tests_directory_test_code() { + let tree = |declaration: &str| { + checkout() + .appending("src/lib.rs", "\nmod probe;\n") + .with("src/probe.rs", declaration) + .with( + "src/probe/tests/leak.rs", + "fn f() { create_config_backup(path); }\n", + ) + }; + + assert_fails( + tree("mod tests;\n"), + "src/probe/tests/leak.rs:1: config write capability `create_config_backup`", + ); + let failures = audit(&tree("#[cfg(test)]\nmod tests;\n")); + assert!(failures.is_empty(), "{failures:#?}"); +} + +#[test] +fn editor_calls_are_pinned_to_their_reviewed_callers() { + assert_fails( + checkout().appending( + "src/backend/wayland/state.rs", + "\nfn sneak() { crate::config::persist_quick_color(0, color()); }\n", + ), + "src/backend/wayland/state.rs: unreviewed caller of the narrow config writer \ + `persist_quick_color`", + ); + assert_fails( + checkout() + .replacing(EDIT_WORKER, "\n persist_quick_color,\n", "\n") + .replacing( + EDIT_WORKER, + "persist_quick_color(edit.index, edit.color)", + "unreachable!()", + ), + "expected to call `persist_quick_color` but does not", + ); +} + +#[test] +fn path_taking_twins_stay_out_of_production() { + assert_fails( + checkout().appending( + "src/daemon/core.rs", + "\nfn f() { persist_preset_slot_at(path, 0, None); }\n", + ), + "src/daemon/core.rs: production code names `persist_preset_slot_at`", + ); + assert_fails( + checkout().replacing( + IO_SOURCE, + "#[cfg(test)]\npub(crate) fn persist_quick_color_at(", + "pub(crate) fn persist_quick_color_at(", + ), + "`persist_quick_color_at` is no longer a `#[cfg(test)] pub(crate) fn`", + ); +} + +#[test] +fn editors_are_not_wrapped_elsewhere_in_the_config_module() { + assert_fails( + checkout().appending( + "src/config/types/mod.rs", + "\npub fn wrapped() { super::io::persist_keybinding_edit(action(), &[]); }\n", + ), + "names the narrow config writer `persist_keybinding_edit` inside the config module", + ); +} + +#[test] +fn casting_a_writer_to_a_value_fails() { + // The edit worker may name the editor, but not hand it on as a value. + assert_fails( + checkout().appending( + EDIT_WORKER, + "\nfn leak() -> fn() { persist_quick_color as fn() }\n", + ), + "casts the config writer `persist_quick_color`", + ); +} + +#[test] +fn renaming_a_writer_on_import_or_reexport_fails() { + assert_fails( + checkout().appending( + EDIT_WORKER, + "\nuse crate::config::persist_preset_slot as save;\n", + ), + "imports the config writer `persist_preset_slot` as `save`", + ); + assert_fails( + checkout().appending( + CONFIG_MODULE, + "\npub use io::persist_keybinding_edit as concealed_edit;\n", + ), + "re-exports the config writer `persist_keybinding_edit` as `concealed_edit`", + ); +} + +#[test] +fn a_new_document_entry_that_reaches_the_write_fails() { + assert_fails( + checkout().replacing( + DOCUMENT_SOURCE, + " fn merge_and_write(", + " pub fn persist_any_config(&self, config: &Config) {\n \ + self.merge_and_write(&self.config, config, &mut || {});\n }\n\n \ + fn merge_and_write(", + ), + "`pub fn persist_any_config` can write config.toml but is not the reviewed document save", + ); +} + +#[test] +fn a_new_io_entry_that_reaches_a_primitive_fails() { + assert_fails( + checkout().appending( + IO_SOURCE, + "\nfn helper(path: &Path) -> Result<()> { prepare_config_parent(path) }\n\ + pub fn persist_everything(path: &Path) -> Result<()> { helper(path) }\n", + ), + "`pub fn persist_everything` can write config.toml but is not one of the reviewed editors", + ); +} + +#[test] +fn a_write_reachable_through_a_trait_fails() { + assert_fails( + checkout().appending( + DOCUMENT_SOURCE, + "\nimpl Persist for ConfigDocument {\n fn persist(&self, config: &Config) {\n \ + self.merge_and_write(&self.config, config, &mut || {});\n }\n}\n", + ), + "`fn persist` can write config.toml from inside a trait", + ); + assert_fails( + checkout().appending( + IO_SOURCE, + "\npub trait Concealed {\n fn persist_quick_color(&self);\n}\n", + ), + "`trait Concealed` declares `persist_quick_color`", + ); +} + +#[test] +fn a_writer_stored_as_a_value_fails_on_both_halves() { + assert_fails( + checkout().appending( + DOCUMENT_SOURCE, + "\npub const PERSIST_ANY_CONFIG: fn(&ConfigDocument, Config) -> \ + Result = ConfigDocument::save_with_backup;\n", + ), + "`const PERSIST_ANY_CONFIG` names `save_with_backup`", + ); + assert_fails( + checkout() + .appending(IO_SOURCE, "\npub static WRITE_CONFIG: () = ();\n") + .appending(CONFIG_MODULE, "\npub use io::WRITE_CONFIG;\n"), + "re-exports `WRITE_CONFIG`, a `static` declared in src/config/io.rs", + ); +} + +#[test] +fn the_config_module_stays_a_reexport_list() { + assert_fails( + checkout().appending( + CONFIG_MODULE, + "\npub fn concealed_edit() { io::persist_keybinding_edit(action(), &[]); }\n", + ), + "names the config writer `persist_keybinding_edit` inside `fn concealed_edit`", + ); + assert_fails( + checkout().appending( + CONFIG_MODULE, + "\npub fn indirect() { concealed(); }\n\ + fn concealed() { io::persist_quick_color(0, color()); }\n", + ), + "`fn indirect` can reach a config writer", + ); +} + +#[test] +fn primitives_never_leave_the_config_module() { + assert_fails( + checkout().appending(CONFIG_MODULE, "\npub use io::create_config_backup;\n"), + "re-exports the write primitive `create_config_backup`", + ); + assert_fails( + checkout().replacing( + IO_SOURCE, + "pub(super) fn prepare_config_parent(", + "pub(crate) fn prepare_config_parent(", + ), + "`prepare_config_parent` is no longer `pub(super)`", + ); +} + +#[test] +fn editors_build_on_the_loaded_config() { + assert_fails( + checkout().appending( + IO_SOURCE, + "\nfn base(document: &ConfigDocument) -> &Config { document.authored_config() }\n", + ), + "a narrow writer reads `authored_config()`", + ); +} + +#[test] +fn the_walk_must_reach_every_former_writer() { + assert_fails( + checkout().without("src/daemon/tray/runtime.rs"), + "the walk missed expected sources, so it proves nothing: src/daemon/tray/runtime.rs", + ); +} diff --git a/tests/repository_guards/main.rs b/tests/repository_guards/main.rs new file mode 100644 index 000000000..1342740b7 --- /dev/null +++ b/tests/repository_guards/main.rs @@ -0,0 +1,12 @@ +//! Source-level contracts that keep ownership boundaries reviewable. +//! +//! These read the repository's sources rather than compiled items, so they +//! are guardrails over spelling, not proofs; each module says what its guard +//! can and cannot see. They run with every `cargo test`, so they need no tool +//! beyond Cargo, and each has regression cases showing its escapes fail. + +mod config_writers; +mod no_python; +mod process_sites; +mod shared_dependencies; +mod source; diff --git a/tests/repository_guards/no_python.rs b/tests/repository_guards/no_python.rs new file mode 100644 index 000000000..8333ca386 --- /dev/null +++ b/tests/repository_guards/no_python.rs @@ -0,0 +1,537 @@ +//! The repository carries no Python. +//! +//! Tools are C# or POSIX shell, and tests and fixtures are Rust. This fails on a +//! Python file or a link to one, a Python project or lock file, a Python +//! shebang, or a Python interpreter or package name in the sources it reads: +//! Rust, C#, MSBuild, shell, Nix, TOML, workflow, build, service, desktop-entry, +//! and packaging files. So neither a script embedded in a string literal, an +//! interpreter launched by the tools, nor a declared interpreter dependency can +//! come back. Markdown is not read, so documentation can still say that the +//! repository uses no Python, and a `python` domain label such as +//! `docs.python.org` is not an interpreter. +//! +//! The walk covers every directory and file at the root except `.git`, build +//! output (`target/` and any root directory holding a `CACHEDIR.TAG`), and the +//! local files named by the root `.gitignore`. Inside an ignored directory such +//! as `packaging/`, the files that `.gitignore` re-includes one by one are read. +//! Extensionless files, such as `PKGBUILD` or a script, are read in full. It +//! walks the checkout rather than asking Git, so it also runs in a Nix build, +//! which has no `.git`. +//! +//! It reads spelling, as a tripwire against Python coming back by accident, +//! not as a sandbox. Known limits: an interpreter reached under another name, +//! such as `pypy3`, `PYTHONPATH`, or `buildPythonApplication`, is not seen; a +//! `python` path segment that is not a domain label, as in +//! `github.com/python/cpython`, is reported, so such a link is reworded; and +//! an untracked directory that is not ignored and holds no `CACHEDIR.TAG`, such +//! as `.direnv/` or `node_modules/`, is read like a source directory. + +use std::collections::BTreeMap; +use std::fs; +use std::path::{Path, PathBuf}; + +use crate::source::{files_under, is_word_char, repository_root, words}; + +/// Never walked, whether or not `.gitignore` names them. +const SKIPPED_DIRECTORIES: [&str; 2] = [".git", "target"]; +/// Marks a cache directory, such as a Cargo target directory under any name. +const CACHE_DIRECTORY_TAG: &str = "CACHEDIR.TAG"; +const READ_EXTENSIONS: [&str; 11] = [ + "cs", "desktop", "nix", "props", "rs", "service", "sh", "targets", "toml", "yaml", "yml", +]; +const PYTHON_EXTENSIONS: [&str; 9] = [ + "ipynb", "pxd", "py", "pyc", "pyd", "pyi", "pyo", "pyw", "pyx", +]; +const PYTHON_PROJECT_FILES: [&str; 10] = [ + ".python-version", + "Pipfile", + "Pipfile.lock", + "pdm.lock", + "poetry.lock", + "pyproject.toml", + "requirements.txt", + "setup.cfg", + "tox.ini", + "uv.lock", +]; +/// Top-level domains after which a `python` label names a website. +const DOMAIN_SUFFIXES: [&str; 4] = ["com", "io", "net", "org"]; +/// This guard names the interpreter to reject it. +const THIS_GUARD: &str = "tests/repository_guards/no_python.rs"; + +enum Entry { + /// A file whose contents are read. + Read(Vec), + /// A symbolic link and its target, which is not followed. + Link(PathBuf), +} + +/// One entry of the root `.gitignore` that this guard honors. +#[derive(Debug, PartialEq)] +struct LocalEntry { + path: PathBuf, + /// Written with a trailing `/` or `/**`, so it matches only a directory. + directory_only: bool, +} + +/// The local files and directories the root `.gitignore` names. Only plain +/// names, which match at the root, and anchored paths with a `/` inside are +/// read; globs are left out, so this skips no more than Git ignores. A file Git +/// tracks despite a matching entry is skipped too, unless a `!` line names it. +#[derive(Debug, Default, PartialEq)] +struct LocalFiles { + root_names: Vec, + paths: Vec, + /// Files a `!` line re-includes one by one, such as the packaging recipes. + included: Vec, +} + +impl LocalFiles { + fn parse(gitignore: &str) -> Self { + let mut local = Self::default(); + + for line in gitignore.lines().map(str::trim) { + if let Some(included) = line.strip_prefix('!') { + let path = included.trim_start_matches('/'); + if !path.is_empty() && !path.ends_with('/') && !path.contains(['*', '?', '[', '\\']) + { + local.included.push(PathBuf::from(path)); + } + continue; + } + if line.is_empty() || line.starts_with('#') { + continue; + } + + let (pattern, directory_only) = match line.strip_suffix("/**") { + Some(directory) => (directory, true), + None => (line.trim_end_matches('/'), line.ends_with('/')), + }; + let pattern = pattern.trim_start_matches('/'); + if pattern.is_empty() || pattern.contains(['*', '?', '[', '\\']) { + continue; + } + + let entry = LocalEntry { + path: PathBuf::from(pattern), + directory_only, + }; + if pattern.contains('/') { + local.paths.push(entry); + } else { + local.root_names.push(entry); + } + } + + local + } + + fn contains(&self, path: &Path, is_directory: bool) -> bool { + let matches = |entry: &LocalEntry| { + (path == entry.path && (is_directory || !entry.directory_only)) + || (path != entry.path && path.starts_with(&entry.path)) + }; + let at_root = path.parent() == Some(Path::new("")); + + self.paths.iter().any(matches) || (at_root && self.root_names.iter().any(matches)) + } +} + +fn checkout() -> BTreeMap { + let root = repository_root(); + let gitignore = fs::read_to_string(root.join(".gitignore")).unwrap_or_default(); + let local = LocalFiles::parse(&gitignore); + let mut paths: Vec = local + .included + .iter() + .filter(|path| root.join(path).is_file()) + .cloned() + .collect(); + + for entry in fs::read_dir(&root).expect("read the repository root") { + let path = PathBuf::from(entry.expect("root entry").file_name()); + let is_directory = fs::symlink_metadata(root.join(&path)).is_ok_and(|meta| meta.is_dir()); + let is_cache = is_directory && root.join(&path).join(CACHE_DIRECTORY_TAG).is_file(); + if SKIPPED_DIRECTORIES + .iter() + .any(|skipped| path == Path::new(skipped)) + || is_cache + || local.contains(&path, is_directory) + { + continue; + } + + if is_directory { + let directory = path.to_str().expect("UTF-8 directory name"); + paths.extend( + files_under(directory) + .into_iter() + .filter(|file| !local.contains(file, false)), + ); + } else { + paths.push(path); + } + } + + paths + .into_iter() + .map(|path| { + let entry = read(&root.join(&path)); + (path, entry) + }) + .collect() +} + +fn read(path: &Path) -> Entry { + let metadata = fs::symlink_metadata(path) + .unwrap_or_else(|error| panic!("stat {}: {error}", path.display())); + + if metadata.is_symlink() { + let target = fs::read_link(path) + .unwrap_or_else(|error| panic!("read link {}: {error}", path.display())); + return Entry::Link(target); + } + + Entry::Read(fs::read(path).unwrap_or_else(|error| panic!("read {}: {error}", path.display()))) +} + +fn audit(entries: &BTreeMap) -> Vec { + let mut failures = Vec::new(); + + for (path, entry) in entries { + let shown = path.display(); + if is_python_name(path) { + failures.push(format!("{shown}: Python source, project, or lock file")); + continue; + } + + let bytes = match entry { + Entry::Link(target) => { + if is_python_name(target) || names_python(&target.to_string_lossy()) { + failures.push(format!("{shown}: links to Python at {}", target.display())); + } + continue; + } + Entry::Read(bytes) => bytes, + }; + + let first_line = bytes + .split(|byte| *byte == b'\n') + .next() + .unwrap_or_default(); + if first_line.starts_with(b"#!") && names_python(&String::from_utf8_lossy(first_line)) { + failures.push(format!("{shown}:1: Python shebang")); + continue; + } + if path == Path::new(THIS_GUARD) || !is_read(path) { + continue; + } + + let Ok(text) = std::str::from_utf8(bytes) else { + // An extensionless file may be binary; a source file must be text. + if path.extension().is_some() { + failures.push(format!("{shown}: not UTF-8")); + } + continue; + }; + for (number, line) in text.lines().enumerate() { + if names_python(line) { + failures.push(format!( + "{shown}:{}: names Python: {}", + number + 1, + line.trim() + )); + } + } + } + + failures +} + +/// Sources by extension, and every extensionless file, such as `PKGBUILD`, +/// `.SRCINFO`, `Makefile`, `.envrc`, or a script. +fn is_read(path: &Path) -> bool { + path.extension().is_none() || has_extension(path, &READ_EXTENSIONS) +} + +fn is_python_name(path: &Path) -> bool { + let name = file_name(path); + + has_extension(path, &PYTHON_EXTENSIONS) + || PYTHON_PROJECT_FILES.contains(&name) + || (name.starts_with("requirements") && name.ends_with(".txt")) +} + +fn file_name(path: &Path) -> &str { + path.file_name() + .and_then(|name| name.to_str()) + .unwrap_or_default() +} + +fn has_extension(path: &Path, extensions: &[&str]) -> bool { + path.extension() + .and_then(|extension| extension.to_str()) + .is_some_and(|extension| extensions.contains(&extension)) +} + +/// Whether `text` names a Python interpreter or package, qualified or not: a +/// word that is `python`, a version such as `python3` (and so `python3.12`), +/// or either one followed by a capitalized part, as in the nixpkgs names +/// `python3Packages`, `python311Packages`, and `python3Minimal`. A capital +/// `Python` counts only with a version, so prose stays readable, and a +/// `python` domain label such as `docs.python.org` names a website. +fn names_python(text: &str) -> bool { + words(text).any(|(offset, word)| { + is_python_word(word) && !is_domain_label(&text[offset + word.len()..]) + }) +} + +fn is_python_word(word: &str) -> bool { + let (capitalized, rest) = match (word.strip_prefix("python"), word.strip_prefix("Python")) { + (Some(rest), _) => (false, rest), + (None, Some(rest)) => (true, rest), + (None, None) => return false, + }; + let version_length = rest.bytes().take_while(u8::is_ascii_digit).count(); + let (version, suffix) = rest.split_at(version_length); + + (suffix.is_empty() || suffix.starts_with(|character: char| character.is_ascii_uppercase())) + && (!capitalized || !version.is_empty()) +} + +/// Whether the text after a word continues it into a domain name. +fn is_domain_label(after: &str) -> bool { + let Some(rest) = after.strip_prefix('.') else { + return false; + }; + let label = rest + .split(|character| !is_word_char(character)) + .next() + .unwrap_or_default(); + + DOMAIN_SUFFIXES.contains(&label) +} + +fn read_entry(contents: &str) -> Entry { + Entry::Read(contents.as_bytes().to_vec()) +} + +#[test] +fn the_repository_carries_no_python() { + let failures = audit(&checkout()); + + assert!( + failures.is_empty(), + "Python found:\n{}", + failures.join("\n") + ); +} + +#[test] +fn python_files_links_shebangs_and_words_fail() { + let python_file = "Python source, project, or lock file"; + let cases = [ + ("tools/check.py", read_entry("print('x')\n"), python_file), + ("tests/fixture.pyw", read_entry(""), python_file), + ("docs/notebook.ipynb", read_entry("{}"), python_file), + ("src/speedups.pyx", read_entry(""), python_file), + ("setup.py", read_entry(""), python_file), + ("pyproject.toml", read_entry(""), python_file), + ("poetry.lock", read_entry(""), python_file), + ("requirements-dev.txt", read_entry(""), python_file), + ( + "tools/lib.py", + Entry::Link(PathBuf::from("../vendor/lib")), + python_file, + ), + ( + "helper", + Entry::Link(PathBuf::from("tools/helper.py")), + "links to Python", + ), + ( + "tools/run", + Entry::Link(PathBuf::from("/usr/bin/python3")), + "links to Python", + ), + ( + "bootstrap", + read_entry("#!/usr/bin/env python3\n"), + "Python shebang", + ), + ( + "tools/check", + read_entry("#!/usr/bin/python3.12 -u\n"), + "Python shebang", + ), + ( + "src/overlay/tests.rs", + read_entry( + "const SCRIPT: &str = r#\"import os\"#;\nfn f() { Command::new(\"python3\"); }\n", + ), + "src/overlay/tests.rs:2: names Python", + ), + ( + "tools/csharp/Infrastructure/ToolConstants.cs", + read_entry(" public const string Python = \"python3\";\n"), + "ToolConstants.cs:1: names Python", + ), + ( + "tools/gate.sh", + read_entry("run python -c 1\n"), + "tools/gate.sh:1: names Python", + ), + ( + "makefile", + read_entry("check:\n\tpython3 tools/check\n"), + "makefile:2: names Python", + ), + ( + ".envrc", + read_entry("export PATH=$PWD/.venv/bin:$PATH # python3\n"), + ".envrc:1", + ), + ( + "tools/helper", + read_entry("set -eu\nexec python3 \"$@\"\n"), + "tools/helper:2: names Python", + ), + ( + "packaging/wayscriber.service", + read_entry("ExecStart=/usr/bin/python3 -m wayscriber\n"), + "wayscriber.service:1: names Python", + ), + ( + "packaging/wayscriber.desktop", + read_entry("Exec=python3 /usr/bin/wayscriber\n"), + "wayscriber.desktop:1: names Python", + ), + ( + "tools/Directory.Build.props", + read_entry("\n"), + "Directory.Build.props:1: names Python", + ), + ( + "flake.nix", + read_entry(" nativeCheckInputs = [ python3 ];\n"), + "flake.nix:1: names Python", + ), + ( + "flake.nix", + read_entry(" nativeCheckInputs = [ pkgs.python3 ];\n"), + "flake.nix:1: names Python", + ), + ( + "flake.nix", + read_entry(" nativeCheckInputs = [ pkgs.python3Packages.pytest ];\n"), + "flake.nix:1: names Python", + ), + ( + "flake.nix", + read_entry(" buildInputs = [ python311Packages.requests python3Minimal ];\n"), + "flake.nix:1: names Python", + ), + ( + "packaging/nixpkgs/package.nix", + read_entry(" nativeCheckInputs = [ (pkgs.python312.withPackages (p: [ ])) ];\n"), + "package.nix:1: names Python", + ), + ( + "packaging/PKGBUILD", + read_entry("makedepends=('cargo' 'python')\n"), + "packaging/PKGBUILD:1: names Python", + ), + ( + "src/notes.rs", + read_entry("// Requires Python3 on the build host.\n"), + "src/notes.rs:1: names Python", + ), + ( + ".github/workflows/ci.yml", + read_entry(" - run: python3.12 tools/check\n"), + ".github/workflows/ci.yml:1: names Python", + ), + ]; + + for (path, entry, expected) in cases { + let failures = audit(&BTreeMap::from([(PathBuf::from(path), entry)])); + + assert!( + failures.iter().any(|failure| failure.contains(expected)), + "{path} should fail with {expected:?}, got {failures:?}" + ); + } +} + +#[test] +fn prose_lookalikes_and_unread_files_pass() { + let entries = BTreeMap::from([ + ( + PathBuf::from("tools/README.md"), + read_entry("Uses no python.\n"), + ), + ( + PathBuf::from("src/a.rs"), + read_entry( + "// Python is not used; see docs.python.org.\n// https://www.python.org/\n\ + fn pythonic() {}\nconst python_free: bool = true;\n", + ), + ), + ( + PathBuf::from("tools/run.sh"), + read_entry("#!/usr/bin/env bash\necho python_free\n"), + ), + ( + PathBuf::from("tools/current"), + Entry::Link(PathBuf::from("wayscriber")), + ), + ( + PathBuf::from("assets/a.png"), + Entry::Read(vec![0x89, b'P', b'N', b'G', 0xff]), + ), + // An extensionless binary is skipped rather than reported as not UTF-8. + ( + PathBuf::from(".DS_Store"), + Entry::Read(vec![0, 0, 0, 1, 0xff]), + ), + ]); + + assert_eq!(audit(&entries), Vec::::new()); +} + +#[test] +fn only_plain_ignored_names_and_paths_count_as_local() { + let local = LocalFiles::parse( + "# Local\n/target\n**/*.rs.bk\nCLAUDE.md\nrun.sh\n!/tools/run.sh\ntools/release.sh\n\ + packaging/**\n!packaging/PKGBUILD\n!packaging/icons/*.png\n!packaging/licenses/\nscripts/\ndocs/temp\n", + ); + let entry = |path: &str, directory_only| LocalEntry { + path: PathBuf::from(path), + directory_only, + }; + + assert_eq!( + local, + LocalFiles { + root_names: vec![ + entry("target", false), + entry("CLAUDE.md", false), + entry("run.sh", false), + entry("packaging", true), + entry("scripts", true), + ], + paths: vec![entry("tools/release.sh", false), entry("docs/temp", false)], + included: ["tools/run.sh", "packaging/PKGBUILD"] + .map(PathBuf::from) + .to_vec(), + } + ); + assert!(local.contains(Path::new("run.sh"), false)); + assert!(local.contains(Path::new("docs/temp/draft.py"), false)); + assert!(local.contains(Path::new("packaging"), true)); + assert!(local.contains(Path::new("scripts"), true)); + // A root file named like a directory-only entry is not ignored by Git. + assert!(!local.contains(Path::new("scripts"), false)); + assert!(!local.contains(Path::new("tools/run.sh"), false)); + assert!(!local.contains(Path::new("tools/scripts/run.sh"), false)); +} diff --git a/tests/repository_guards/process_sites.rs b/tests/repository_guards/process_sites.rs new file mode 100644 index 000000000..453a5a27f --- /dev/null +++ b/tests/repository_guards/process_sites.rs @@ -0,0 +1,322 @@ +//! Every process-creation site is in the reviewed ownership map. +//! +//! Production code creates processes only through the process broker; the +//! configurator, a separate process, keeps three reviewed direct sites. The +//! broker's raw-clone child stub runs between `clone` and `execve`, so it may +//! reach only the approved syscalls and nothing that allocates, locks, or logs. +//! Lines are read with `//` comments cut, and strings are read as written, so a +//! shell command spelled inside a literal is still a process site. Integration +//! tests and files compiled only under `cfg(test)` are test code; a `tests` +//! directory in `src/` counts only when its `mod` item says `#[cfg(test)]`. + +use std::path::Path; + +use super::source::{TestSources, Tree, is_word_char}; + +const BROKER_ROOT: &str = "src/process_broker"; +const BROKER_BOOTSTRAP: &str = "src/process_broker/bootstrap.rs"; + +const DIRECT_PRODUCTION_ALLOWLIST: [&str; 3] = [ + // The configurator is a separate process with its own reviewed sites. + "configurator/src/app/session_catalog.rs", + "configurator/src/app/daemon_setup/command.rs", + "configurator/src/app/daemon_setup/service.rs", +]; + +const STUB_START: &str = " if pid == 0 {"; +const STUB_END: &str = " drop(child_socket);"; +const STUB_BANNED: [&str; 11] = [ + "format!(", + "log::", + "panic!(", + ".unwrap(", + ".expect(", + "drop(", + "Command::", + "CString::", + "Vec::", + "String::", + "Box::", +]; +const STUB_LIBC_CALLS: [&str; 2] = ["syscall", "_exit"]; +const STUB_SYSCALLS: [&str; 6] = [ + "fcntl", + "dup3", + "setpgid", + "exit_group", + "close_range", + "execve", +]; + +fn audit_sites(tree: &Tree) -> Vec { + let mut failures = Vec::new(); + let mut test_sources = TestSources::default(); + + for directory in ["src", "configurator/src", "tests"] { + for path in tree.paths_under(directory) { + let allowed = path.starts_with(BROKER_ROOT) + || DIRECT_PRODUCTION_ALLOWLIST + .iter() + .any(|allowed| path == Path::new(allowed)) + || path.starts_with("tests") + || test_sources.is_test(tree, path); + if allowed { + continue; + } + + let source = tree.read(path).expect("listed path"); + for (number, line) in source.lines().enumerate() { + let code = line.split("//").next().unwrap_or_default(); + if is_process_site(code) { + failures.push(format!( + "{}:{}: unclassified process site: {}", + path.display(), + number + 1, + line.trim() + )); + } + } + } + } + + failures +} + +/// Paths that create a process. `Command::new` also covers its +/// `std::process::` spelling. +const PROCESS_PATHS: [&str; 11] = [ + "Command::new", + "std::process::Child", + "libc::fork", + "libc::vfork", + "libc::posix_spawn", + "libc::posix_spawnp", + "libc::pthread_atfork", + "libc::SYS_clone", + "libc::SYS_clone3", + "libc::SYS_fork", + "libc::SYS_vfork", +]; + +fn is_process_site(code: &str) -> bool { + (code.contains("::") && PROCESS_PATHS.iter().any(|path| has_word_path(code, path))) + || (code.contains("-c") && has_shell_command(code)) +} + +/// `path` with word boundaries at both ends, like `\bpath\b`. +fn has_word_path(code: &str, path: &str) -> bool { + code.match_indices(path).any(|(index, _)| { + let before = code[..index].chars().next_back(); + let after = code[index + path.len()..].chars().next(); + !before.is_some_and(is_word_char) && !after.is_some_and(is_word_char) + }) +} + +/// `sh -c`, `bash -c`, or `zsh -c`, like `\b(?:sh|bash|zsh)\s+-c\b`. +fn has_shell_command(code: &str) -> bool { + ["sh", "bash", "zsh"].iter().any(|shell| { + code.match_indices(shell).any(|(index, _)| { + let before = code[..index].chars().next_back(); + let rest = &code[index + shell.len()..]; + let flag = rest.trim_start(); + !before.is_some_and(is_word_char) + && flag.len() < rest.len() + && flag.starts_with("-c") + && !flag[2..].chars().next().is_some_and(is_word_char) + }) + }) +} + +fn audit_child_stub(tree: &Tree) -> Vec { + let source = tree + .read(Path::new(BROKER_BOOTSTRAP)) + .expect("broker bootstrap source"); + let Some(stub) = source + .split_once(STUB_START) + .and_then(|(_, rest)| rest.split_once(STUB_END)) + .map(|(stub, _)| stub) + else { + return vec![format!( + "{BROKER_BOOTSTRAP}: raw-clone child-stub markers changed" + )]; + }; + + let mut failures: Vec = STUB_BANNED + .iter() + .filter(|token| stub.contains(*token)) + .map(|token| format!("{BROKER_BOOTSTRAP}: child stub reaches banned token {token:?}")) + .collect(); + + let mut calls = prefixed_names(stub, "libc::", |rest| rest.trim_start().starts_with('(')); + calls.retain(|call| !STUB_LIBC_CALLS.contains(&call.as_str())); + if !calls.is_empty() { + failures.push(format!( + "{BROKER_BOOTSTRAP}: child stub reaches unapproved libc calls: {}", + calls.join(", ") + )); + } + + let mut syscalls = prefixed_names(stub, "libc::SYS_", |_| true); + syscalls.retain(|syscall| !STUB_SYSCALLS.contains(&syscall.as_str())); + if !syscalls.is_empty() { + failures.push(format!( + "{BROKER_BOOTSTRAP}: child stub reaches unapproved syscalls: {}", + syscalls.join(", ") + )); + } + + failures +} + +/// Sorted, distinct `[A-Za-z0-9_]+` names following `prefix` whose remaining +/// text satisfies `accept`. +fn prefixed_names(text: &str, prefix: &str, accept: impl Fn(&str) -> bool) -> Vec { + let mut names: Vec = text + .match_indices(prefix) + .filter_map(|(index, _)| { + let rest = &text[index + prefix.len()..]; + let length = rest + .find(|character: char| !(character.is_ascii_alphanumeric() || character == '_')) + .unwrap_or(rest.len()); + (length > 0 && accept(&rest[length..])).then(|| rest[..length].to_owned()) + }) + .collect(); + names.sort(); + names.dedup(); + names +} + +fn audit(tree: &Tree) -> Vec { + let mut failures = audit_sites(tree); + failures.extend(audit_child_stub(tree)); + failures +} + +fn checkout() -> Tree { + Tree::checkout(&["src", "configurator/src", "tests"]) +} + +#[test] +fn every_process_site_is_owned() { + let failures = audit(&checkout()); + + assert!(failures.is_empty(), "{}", failures.join("\n")); +} + +#[test] +fn unowned_process_sites_fail() { + let probe = "src/daemon/probe.rs"; + for site in [ + "let child = Command::new(program);", + "let child: std::process::Child = spawn();", + "unsafe { libc::fork() };", + "unsafe { libc::syscall(libc::SYS_clone3, args) };", + "let line = \"bash -c 'exit'\";", + ] { + let failures = audit(&checkout().with(probe, &format!("fn f() {{ {site} }}\n"))); + + assert!( + failures + .iter() + .any(|failure| failure.starts_with(&format!("{probe}:1:"))), + "{site}: {failures:?}" + ); + } +} + +#[test] +fn owned_and_commented_process_sites_pass() { + let site = "fn f() { Command::new(program); }\n"; + let tree = checkout() + .with("src/process_broker/probe.rs", site) + .with("src/daemon/probe/tests.rs", site) + .with("src/daemon/tests/probe.rs", site) + .with("tests/probe.rs", site) + // Off the module chain, a test module's `#[path]` names its file. + .with("src/daemon/fixtures/pathed.rs", site) + .with( + "src/daemon/probe.rs", + "#[cfg(test)]\nmod tests;\n#[cfg(test)]\n#[path = \"fixtures/pathed.rs\"]\nmod pathed;\n\n\ + fn f() {} // Command::new(program)\n", + ); + + let failures = audit(&tree); + + assert!(failures.is_empty(), "{failures:?}"); +} + +#[test] +fn only_a_cfg_test_module_makes_a_file_test_code() { + let site = "fn f() { Command::new(program); }\n"; + let tree = checkout() + .with( + "src/daemon/probe.rs", + "mod tests;\n#[path = \"fixtures/pathed.rs\"]\nmod pathed;\n", + ) + .with("src/daemon/probe/tests.rs", site) + .with("src/daemon/fixtures/pathed.rs", site); + + let failures = audit(&tree); + + for path in ["src/daemon/probe/tests.rs", "src/daemon/fixtures/pathed.rs"] { + assert!( + failures + .iter() + .any(|failure| failure.starts_with(&format!("{path}:1:"))), + "{path}: {failures:?}" + ); + } +} + +#[test] +fn a_test_path_alias_does_not_exempt_production_code() { + let tree = checkout() + .with( + "src/daemon/probe.rs", + "#[cfg(test)]\n#[path = \"core.rs\"]\nmod core_alias;\n", + ) + .appending( + "src/daemon/core.rs", + "\nfn leak() { Command::new(program); }\n", + ); + + let failures = audit(&tree); + + assert!( + failures + .iter() + .any(|failure| failure.starts_with("src/daemon/core.rs:")), + "{failures:?}" + ); +} + +#[test] +fn the_raw_clone_child_stub_stays_minimal() { + for (addition, expected) in [ + ("log::warn!(\"x\");", "banned token \"log::\""), + ("libc::getpid();", "unapproved libc calls: getpid"), + ( + "libc::syscall(libc::SYS_write, 1);", + "unapproved syscalls: write", + ), + ] { + let tree = checkout().replacing( + BROKER_BOOTSTRAP, + STUB_END, + &format!(" {addition}\n{STUB_END}"), + ); + let failures = audit(&tree); + + assert!( + failures.iter().any(|failure| failure.contains(expected)), + "{addition}: {failures:?}" + ); + } + + let failures = audit(&checkout().replacing(BROKER_BOOTSTRAP, STUB_END, " drop(socket);")); + assert!( + failures + .iter() + .any(|failure| failure.contains("markers changed")) + ); +} diff --git a/tests/repository_guards/shared_dependencies.rs b/tests/repository_guards/shared_dependencies.rs new file mode 100644 index 000000000..0a14655f6 --- /dev/null +++ b/tests/repository_guards/shared_dependencies.rs @@ -0,0 +1,201 @@ +//! Shared layers do not reach up into the runtime crates' owners. +//! +//! A partial source guard: it understands rooted and parent-relative paths and +//! grouped `use` trees, and ignores comments and literals. It does not resolve +//! aliases, macro expansion, re-exports, or Rust's complete module graph. The +//! syntax corpus beside this file pins how it reads each form. + +use std::path::Path; + +use super::source::{Tree, is_word_char, mask_non_code}; + +const BOUNDARIES: [(&str, &[&str]); 2] = [ + ( + "src/domain", + &["config", "input", "draw", "backend", "ui", "session"], + ), + ("src/config/validate", &["input", "backend"]), +]; + +/// Public-path compatibility assertions only. +const EXEMPT: &str = "src/domain/tests.rs"; + +fn check(tree: &Tree) -> Vec { + let mut errors = Vec::new(); + + for (directory, forbidden) in BOUNDARIES { + for path in tree.paths_under(directory) { + if path == Path::new(EXEMPT) { + continue; + } + + let source = tree.read(path).expect("listed path"); + if has_upward_path(source, path, forbidden) { + errors.push(format!( + "{}: upward dependency in shared layer", + path.display() + )); + } + } + } + + errors +} + +fn tokens(source: &str) -> Vec { + let code = mask_non_code(source); + let mut tokens = Vec::new(); + let mut characters = code.char_indices().peekable(); + + while let Some((index, character)) = characters.next() { + if character == ':' && code[index + 1..].starts_with(':') { + characters.next(); + tokens.push("::".to_owned()); + } else if "{},;*".contains(character) { + tokens.push(character.to_string()); + } else if character.is_alphabetic() || character == '_' { + let raw = character == 'r' + && code[index + 1..].starts_with('#') + && code[index + 2..] + .chars() + .next() + .is_some_and(|next| next.is_alphabetic() || next == '_'); + let start = if raw { index + 2 } else { index }; + if raw { + characters.next(); + } + + let mut end = start; + for (offset, next) in code[start..].char_indices() { + if !is_word_char(next) { + break; + } + end = start + offset + next.len_utf8(); + } + while characters.peek().is_some_and(|(next, _)| *next < end) { + characters.next(); + } + tokens.push(code[start..end].to_owned()); + } + } + + tokens +} + +/// The module path of a source file below `src/`, `mod.rs` naming its directory. +fn module_path(path: &Path) -> Vec { + let mut parts: Vec = path + .with_extension("") + .iter() + .skip(1) + .map(|part| part.to_string_lossy().into_owned()) + .collect(); + if parts.last().is_some_and(|last| last == "mod") { + parts.pop(); + } + parts +} + +fn has_upward_path(source: &str, path: &Path, forbidden: &[&str]) -> bool { + let tokens = tokens(source); + let module = module_path(path); + + (0..tokens.len().saturating_sub(1)).any(|index| { + matches!(tokens[index].as_str(), "crate" | "super" | "self") + && tokens[index + 1] == "::" + && use_tree(&tokens, index, module.clone(), forbidden).0 + }) +} + +/// Walks one path or grouped tree from `index`, returning whether it names a +/// forbidden top-level module and where it stopped. +fn use_tree( + tokens: &[String], + mut index: usize, + prefix: Vec, + forbidden: &[&str], +) -> (bool, usize) { + let mut path = prefix; + + while index < tokens.len() { + let token = tokens[index].as_str(); + if matches!(token, "," | ";" | "}" | "as") { + break; + } + + if token == "{" { + index += 1; + while index < tokens.len() && tokens[index] != "}" { + let (rejected, next) = use_tree(tokens, index, path.clone(), forbidden); + if rejected { + return (true, next); + } + + index = next; + if tokens.get(index).is_some_and(|token| token == "as") { + index += 2; + } + if tokens.get(index).is_some_and(|token| token == ",") { + index += 1; + } else if tokens.get(index).is_some_and(|token| token != "}") { + break; + } + } + return (false, index + 1); + } + + match token { + "crate" => path.clear(), + "super" => { + path.pop(); + } + "self" | "::" | "*" => {} + name => path.push(name.to_owned()), + } + if path + .first() + .is_some_and(|first| forbidden.contains(&first.as_str())) + { + return (true, index); + } + + index += 1; + if tokens.get(index).is_none_or(|token| token != "::") { + break; + } + index += 1; + } + + (false, index) +} + +#[test] +fn shared_layers_name_no_upward_crate_paths() { + let errors = check(&Tree::checkout(&["src"])); + + assert!(errors.is_empty(), "{}", errors.join("\n")); +} + +#[test] +fn shared_dependency_syntax_corpus_matches_its_expectations() { + let corpus: serde_json::Value = + serde_json::from_str(include_str!("shared_dependency_fixtures.json")) + .expect("fixture JSON"); + let fixtures = corpus.as_array().expect("fixture array"); + + assert!(!fixtures.is_empty()); + for fixture in fixtures { + let name = fixture["name"].as_str().expect("name"); + let tree = Tree::empty().with( + fixture["path"].as_str().expect("path"), + fixture["source"].as_str().expect("source"), + ); + let rejected = !check(&tree).is_empty(); + + assert_eq!( + rejected, + fixture["reject"].as_bool().expect("reject"), + "{name}" + ); + } +} diff --git a/tools/shared-dependency-fixtures.json b/tests/repository_guards/shared_dependency_fixtures.json similarity index 100% rename from tools/shared-dependency-fixtures.json rename to tests/repository_guards/shared_dependency_fixtures.json diff --git a/tests/repository_guards/source.rs b/tests/repository_guards/source.rs new file mode 100644 index 000000000..a11a1998c --- /dev/null +++ b/tests/repository_guards/source.rs @@ -0,0 +1,787 @@ +//! Reading the repository's Rust sources for the guards beside this module. +//! +//! The guards read spelling, not compiled items. The Rust-source guards work on +//! a [`Tree`]: the checked-out files, or a copy with an edit applied, which is +//! how their regression cases show that each forbidden escape fails. A corpus +//! of small sources can also be checked as its own tree. + +use std::collections::{BTreeMap, BTreeSet, HashMap}; +use std::fs; +use std::path::{Path, PathBuf}; +use std::sync::{LazyLock, Mutex, PoisonError}; + +/// Rust sources keyed by repository-relative path. +#[derive(Clone)] +pub struct Tree { + files: BTreeMap, +} + +/// Checkouts this test binary has read, by directory list. The files do not +/// change while the guards run, so each test starts from a copy. +static CHECKOUTS: LazyLock, Tree>>> = LazyLock::new(Mutex::default); + +impl Tree { + /// Every `.rs` file under `directories`, read from this checkout. + pub fn checkout(directories: &[&str]) -> Self { + let key = directories + .iter() + .map(|directory| (*directory).to_owned()) + .collect(); + let mut checkouts = CHECKOUTS.lock().unwrap_or_else(PoisonError::into_inner); + + checkouts + .entry(key) + .or_insert_with(|| Self::read_checkout(directories)) + .clone() + } + + fn read_checkout(directories: &[&str]) -> Self { + let root = repository_root(); + let mut files = BTreeMap::new(); + + for path in directories + .iter() + .flat_map(|directory| files_under(directory)) + { + if path.extension().is_some_and(|extension| extension == "rs") { + let source = fs::read_to_string(root.join(&path)) + .unwrap_or_else(|error| panic!("read {}: {error}", path.display())); + files.insert(path, source); + } + } + + Self { files } + } + + pub fn empty() -> Self { + Self { + files: BTreeMap::new(), + } + } + + pub fn with(mut self, path: &str, source: &str) -> Self { + self.files.insert(PathBuf::from(path), source.to_owned()); + self + } + + pub fn without(mut self, path: &str) -> Self { + assert!( + self.files.remove(Path::new(path)).is_some(), + "{path} is not in the tree" + ); + self + } + + /// Replaces the only occurrence of `from`, so an edit cannot silently miss. + pub fn replacing(mut self, path: &str, from: &str, to: &str) -> Self { + let source = self.files.get_mut(Path::new(path)).expect("edited file"); + assert_eq!(source.matches(from).count(), 1, "{path}: `{from}`"); + *source = source.replacen(from, to, 1); + self + } + + pub fn appending(mut self, path: &str, addition: &str) -> Self { + self.files + .get_mut(Path::new(path)) + .expect("edited file") + .push_str(addition); + self + } + + pub fn read(&self, path: &Path) -> Option<&str> { + self.files.get(path).map(String::as_str) + } + + pub fn contains(&self, path: &Path) -> bool { + self.files.contains_key(path) + } + + /// Every path, in sorted order. + pub fn paths(&self) -> impl Iterator { + self.files.keys().map(PathBuf::as_path) + } + + /// Paths under `prefix`, in sorted order. + pub fn paths_under<'a>(&'a self, prefix: &'a str) -> impl Iterator + 'a { + self.files + .keys() + .map(PathBuf::as_path) + .filter(move |path| path.starts_with(prefix)) + } +} + +pub fn repository_root() -> PathBuf { + PathBuf::from(env!("CARGO_MANIFEST_DIR")) +} + +/// Repository-relative paths of every file under `directory`, sorted. A +/// symbolic link is listed as a file and never followed. +pub fn files_under(directory: &str) -> Vec { + let root = repository_root(); + let mut files = Vec::new(); + + collect_files(&root, &root.join(directory), &mut files); + files +} + +fn collect_files(root: &Path, directory: &Path, files: &mut Vec) { + let mut entries: Vec<_> = fs::read_dir(directory) + .unwrap_or_else(|error| panic!("read {}: {error}", directory.display())) + .map(|entry| entry.expect("directory entry").path()) + .collect(); + entries.sort(); + + for path in entries { + let kind = fs::symlink_metadata(&path) + .unwrap_or_else(|error| panic!("stat {}: {error}", path.display())) + .file_type(); + if kind.is_dir() { + collect_files(root, &path, files); + } else { + files.push(path.strip_prefix(root).expect("inside the root").to_owned()); + } + } +} + +/// Blanks comments, string and byte-string literals (raw or not), and char +/// literals, keeping every other byte, every newline, and every offset in +/// place. Block comments nest as they do in Rust, and a lifetime is not a +/// char literal. +pub fn mask_non_code(source: &str) -> String { + let bytes = source.as_bytes(); + let mut masked = bytes.to_vec(); + let mut index = 0; + + while index < bytes.len() { + let end = match bytes[index] { + b'/' if bytes.get(index + 1) == Some(&b'/') => line_comment_end(bytes, index), + b'/' if bytes.get(index + 1) == Some(&b'*') => block_comment_end(bytes, index), + b'r' | b'b' if !follows_identifier(bytes, index) => raw_string_end(bytes, index), + b'"' => Some(string_end(bytes, index)), + b'\'' => char_literal_end(source, index), + _ => None, + }; + + match end { + Some(end) => { + for byte in &mut masked[index..end] { + if *byte != b'\n' { + *byte = b' '; + } + } + index = end; + } + None => index += 1, + } + } + + String::from_utf8(masked).expect("whole UTF-8 sequences are blanked") +} + +fn follows_identifier(bytes: &[u8], index: usize) -> bool { + index > 0 && is_identifier_byte(bytes[index - 1]) +} + +fn is_identifier_byte(byte: u8) -> bool { + byte.is_ascii_alphanumeric() || byte == b'_' || byte >= 0x80 +} + +fn line_comment_end(bytes: &[u8], start: usize) -> Option { + Some( + bytes[start..] + .iter() + .position(|byte| *byte == b'\n') + .map_or(bytes.len(), |offset| start + offset), + ) +} + +fn block_comment_end(bytes: &[u8], start: usize) -> Option { + let mut depth = 0; + let mut index = start; + + while index + 1 < bytes.len() { + match (bytes[index], bytes[index + 1]) { + (b'/', b'*') => { + depth += 1; + index += 2; + } + (b'*', b'/') => { + depth -= 1; + index += 2; + if depth == 0 { + return Some(index); + } + } + _ => index += 1, + } + } + + Some(bytes.len()) +} + +/// `r"…"`, `r#"…"#`, `br##"…"##`; `r#name` is a raw identifier, not a string. +fn raw_string_end(bytes: &[u8], start: usize) -> Option { + let mut index = start; + if bytes[index] == b'b' { + if bytes.get(index + 1) == Some(&b'"') { + return Some(string_end(bytes, index + 1)); + } + index += 1; + } + if bytes.get(index) != Some(&b'r') { + return None; + } + index += 1; + + let hashes = bytes[index..] + .iter() + .take_while(|byte| **byte == b'#') + .count(); + index += hashes; + if bytes.get(index) != Some(&b'"') { + return None; + } + index += 1; + + while index < bytes.len() { + if bytes[index] == b'"' + && bytes[index + 1..] + .iter() + .take(hashes) + .all(|byte| *byte == b'#') + { + let end = index + 1 + hashes; + if end <= bytes.len() { + return Some(end); + } + } + index += 1; + } + + Some(bytes.len()) +} + +fn string_end(bytes: &[u8], start: usize) -> usize { + let mut index = start + 1; + + while index < bytes.len() { + match bytes[index] { + b'\\' => index += 2, + b'"' => return index + 1, + _ => index += 1, + } + } + + bytes.len() +} + +/// `'x'`, `'\n'`, `'\u{1F600}'`; anything else starting with `'` is a lifetime. +fn char_literal_end(source: &str, start: usize) -> Option { + let rest = &source[start + 1..]; + let mut characters = rest.char_indices(); + let (_, first) = characters.next()?; + + if first == '\\' { + let escaped = rest[1..].chars().next()?; + let from = 1 + escaped.len_utf8(); + let closing = from + rest[from..].find(['\'', '\n'])?; + return (rest.as_bytes()[closing] == b'\'').then_some(start + 1 + closing + 1); + } + + let (offset, second) = characters.next()?; + (second == '\'' && first != '\n').then_some(start + 1 + offset + 1) +} + +/// One-based line of `offset`. +pub fn line_of(source: &str, offset: usize) -> usize { + source[..offset].matches('\n').count() + 1 +} + +/// Offset just past the `}` that closes the block opening at `opening`. +pub fn block_end(masked: &str, opening: usize) -> Option { + let mut depth = 0usize; + + for (index, byte) in masked.bytes().enumerate().skip(opening) { + match byte { + b'{' => depth += 1, + b'}' => { + depth = depth.checked_sub(1)?; + if depth == 0 { + return Some(index + 1); + } + } + _ => {} + } + } + + None +} + +/// Offset just past the top-level `;` that ends the item continuing at `from`. +pub fn item_end(masked: &str, from: usize) -> Option { + let mut depth = 0i64; + + for (index, byte) in masked.bytes().enumerate().skip(from) { + match byte { + b'(' | b'[' | b'{' => depth += 1, + b')' | b']' | b'}' => depth -= 1, + b';' if depth == 0 => return Some(index + 1), + _ => {} + } + } + + None +} + +/// A character of an identifier, as `\w` reads one. +pub fn is_word_char(character: char) -> bool { + character.is_alphanumeric() || character == '_' +} + +/// Every maximal run of word characters, with its offset. +pub fn words(text: &str) -> impl Iterator { + let mut start = None; + let mut found = Vec::new(); + + for (index, character) in text.char_indices() { + let word = is_word_char(character); + match (word, start) { + (true, None) => start = Some(index), + (false, Some(from)) => { + found.push((from, &text[from..index])); + start = None; + } + _ => {} + } + } + if let Some(from) = start { + found.push((from, &text[from..])); + } + + found.into_iter() +} + +/// Offsets where `name` occurs as a whole word. +pub fn word_hits<'a>(text: &'a str, name: &'a str) -> impl Iterator + 'a { + words(text) + .filter(move |(_, word)| *word == name) + .map(|(offset, _)| offset) +} + +/// [`word_hits`] for each of `names`, from one pass over `text`. +pub fn word_index<'n>(text: &str, names: &[&'n str]) -> HashMap<&'n str, Vec> { + let mut index: HashMap<&'n str, Vec> = HashMap::new(); + + for (offset, word) in words(text) { + if let Some(name) = names.iter().find(|name| **name == word) { + index.entry(name).or_default().push(offset); + } + } + + index +} + +pub fn within(spans: &[(usize, usize)], offset: usize) -> bool { + spans + .iter() + .any(|(start, end)| *start <= offset && offset < *end) +} + +/// A cursor over masked source for recognising item headers. +pub struct Cursor<'a> { + pub text: &'a str, + pub at: usize, +} + +impl<'a> Cursor<'a> { + pub fn new(text: &'a str, at: usize) -> Self { + Self { text, at } + } + + fn rest(&self) -> &'a str { + &self.text[self.at..] + } + + /// Spaces and tabs only, like `[ \t]*`. + pub fn skip_blanks(&mut self) { + let skipped = self.rest().len() - self.rest().trim_start_matches([' ', '\t']).len(); + self.at += skipped; + } + + /// Any whitespace, like `\s*`; returns how much was skipped. + pub fn skip_whitespace(&mut self) -> usize { + let skipped = self.rest().len() - self.rest().trim_start().len(); + self.at += skipped; + skipped + } + + /// `keyword` as a whole word. + pub fn keyword(&mut self, keyword: &str) -> bool { + let rest = self.rest(); + let matched = rest.starts_with(keyword) + && !rest[keyword.len()..] + .chars() + .next() + .is_some_and(is_word_char); + if matched { + self.at += keyword.len(); + } + matched + } + + /// `keyword` followed by at least one whitespace character, like `kw\s+`. + pub fn keyword_then_space(&mut self, keyword: &str) -> bool { + let saved = self.at; + if self.keyword(keyword) && self.skip_whitespace() > 0 { + return true; + } + self.at = saved; + false + } + + pub fn literal(&mut self, literal: &str) -> bool { + let matched = self.rest().starts_with(literal); + if matched { + self.at += literal.len(); + } + matched + } + + /// A run of word characters, like `\w+`. + pub fn word(&mut self) -> Option<&'a str> { + let rest = self.rest(); + let length = rest + .char_indices() + .find(|(_, character)| !is_word_char(*character)) + .map_or(rest.len(), |(index, _)| index); + (length > 0).then(|| { + self.at += length; + &rest[..length] + }) + } + + /// `pub`, optionally restricted, then whitespace, like + /// `pub(?:\s*\([^)]*\))?\s+`. Returns the visibility text. + pub fn visibility(&mut self) -> Option<&'a str> { + let start = self.at; + if !self.keyword("pub") { + return None; + } + + let after_pub = self.at; + self.skip_whitespace(); + if self.literal("(") { + match self.rest().find(')') { + Some(close) => self.at += close + 1, + None => self.at = after_pub, + } + } else { + self.at = after_pub; + } + + let visibility_end = self.at; + if self.skip_whitespace() == 0 { + self.at = start; + return None; + } + Some(&self.text[start..visibility_end]) + } + + /// Function qualifiers, like `(?:const\s+|async\s+|unsafe\s+|extern\s+"[^"]*"\s+)*`. + pub fn function_qualifiers(&mut self) { + loop { + if self.keyword_then_space("const") + || self.keyword_then_space("async") + || self.keyword_then_space("unsafe") + { + continue; + } + + let saved = self.at; + if self.keyword_then_space("extern") + && self.literal("\"") + && let Some(close) = self.rest().find('"') + { + self.at += close + 1; + if self.skip_whitespace() > 0 { + continue; + } + } + self.at = saved; + return; + } + } +} + +/// Offsets of every line start: 0 and each offset after a newline. +pub fn line_starts(text: &str) -> impl Iterator + '_ { + std::iter::once(0).chain(text.match_indices('\n').map(|(index, _)| index + 1)) +} + +/// The attribute that compiles an item only for tests. +pub const CFG_TEST: &str = "#[cfg(test)]"; + +/// Which files Rust compiles only under `cfg(test)`. A directory called `tests` +/// inside `src/` is production code unless the `mod` item that brings it in +/// says otherwise, so the declarations on the module chain decide. A +/// `#[cfg(test)]` module's `#[path]` makes the file it names test code too, +/// unless ordinary `mod` items also reach that file, so an alias cannot exempt +/// production code. The `#[path]` must follow `#[cfg(test)]` and is read +/// relative to the declaring file's directory, as Rust reads it outside inline +/// modules. +/// +/// This catches accidents, not deliberate evasion. Known limits: declarations +/// inside an inline `mod` block are read as the file's own; a +/// `#[cfg(not(test))]` and `#[cfg(test)]` pair of modules with one name counts +/// as test code; and an alias of a file that only a production `#[path]` +/// reaches exempts that file. +#[derive(Default)] +pub struct TestSources { + /// Each declaring file's child modules, read once. + declarations: HashMap, + /// Files named by a `cfg(test)` module's `#[path]`, found on first use. + pathed: Option>, +} + +/// The child modules one file declares. +struct Declarations { + /// Declared under `#[cfg(test)]`. + test: BTreeSet, + /// Declared without it. + production: BTreeSet, +} + +impl TestSources { + pub fn is_test(&mut self, tree: &Tree, path: &Path) -> bool { + let chain = module_chain(tree, path); + + self.chain_has_test_link(tree, &chain) + || (self.pathed(tree).contains(path) && !self.chain_is_production(tree, &chain)) + } + + fn pathed(&mut self, tree: &Tree) -> &BTreeSet { + self.pathed.get_or_insert_with(|| { + let mut pathed = BTreeSet::new(); + for file in tree.paths() { + let text = tree.read(file).expect("listed path"); + if !text.contains("#[path") { + continue; + } + + let directory = file.parent().unwrap_or(Path::new("")); + for module in test_modules(text, &mask_non_code(text)) { + pathed.extend(module.path.map(|relative| directory.join(relative))); + } + } + pathed + }) + } + + /// Whether some module on the chain is declared under `#[cfg(test)]`. + fn chain_has_test_link(&mut self, tree: &Tree, chain: &[(Option, String)]) -> bool { + for (declaring, segment) in chain { + let Some(file) = declaring else { + return false; + }; + if self.declarations(tree, file).test.contains(segment) { + return true; + } + } + + false + } + + /// Whether ordinary `mod` items declare every module on the chain. + fn chain_is_production(&mut self, tree: &Tree, chain: &[(Option, String)]) -> bool { + for (declaring, segment) in chain { + let Some(file) = declaring else { + return false; + }; + if !self.declarations(tree, file).production.contains(segment) { + return false; + } + } + + !chain.is_empty() + } + + fn declarations(&mut self, tree: &Tree, file: &Path) -> &Declarations { + self.declarations.entry(file.to_owned()).or_insert_with(|| { + let text = tree.read(file).expect("declaring file"); + let masked = mask_non_code(text); + let test: BTreeSet = test_modules(text, &masked) + .into_iter() + .map(|module| module.name) + .collect(); + let production = declared_modules(&masked) + .into_iter() + .filter(|name| !test.contains(name)) + .collect(); + + Declarations { test, production } + }) + } +} + +/// The crate's module chain down to `path`: each module name with the file +/// that would declare it, or `None` once no file can. +fn module_chain(tree: &Tree, path: &Path) -> Vec<(Option, String)> { + let Some(root) = crate_source_root(tree, path) else { + return Vec::new(); + }; + let base = root + .parent() + .expect("crate root has a directory") + .to_owned(); + let mut segments: Vec = path + .strip_prefix(&base) + .expect("under its crate root") + .iter() + .map(|part| part.to_string_lossy().into_owned()) + .collect(); + if segments.last().is_some_and(|last| last == "mod.rs") { + segments.pop(); + } else if let Some(last) = segments.last_mut() { + *last = last.trim_end_matches(".rs").to_owned(); + } + + let mut chain = Vec::with_capacity(segments.len()); + let mut declaring = Some(root); + let mut module = base; + for segment in segments { + module = module.join(&segment); + let next = module_file(tree, &module); + chain.push((declaring, segment)); + declaring = next; + } + + chain +} + +/// Every module a file declares with `mod name;` or `mod name { ... }`. +fn declared_modules(masked: &str) -> BTreeSet { + let mut modules = BTreeSet::new(); + + for (offset, word) in words(masked) { + if word != "mod" { + continue; + } + + let mut cursor = Cursor::new(masked, offset + word.len()); + if cursor.skip_whitespace() == 0 { + continue; + } + let Some(name) = cursor.word() else { + continue; + }; + cursor.skip_whitespace(); + if masked[cursor.at..].starts_with([';', '{']) { + modules.insert(name.to_owned()); + } + } + + modules +} + +/// A child module declared under `#[cfg(test)]`, with its `#[path]` if any. +struct TestModule { + name: String, + path: Option, +} + +/// Child modules compiled only under `cfg(test)`: `#[cfg(test)]`, any further +/// attributes, an optional visibility, then `mod name` and `;` or `{`. +/// `masked` is `text` with comments and literals blanked, at the same offsets. +fn test_modules(text: &str, masked: &str) -> Vec { + let mut modules = Vec::new(); + + for (start, _) in masked.match_indices(CFG_TEST) { + let mut cursor = Cursor::new(masked, start + CFG_TEST.len()); + let mut path = None; + cursor.skip_whitespace(); + while cursor.literal("#[") { + let Some(close) = masked[cursor.at..].find(']') else { + break; + }; + path = path.or_else(|| path_attribute(&text[cursor.at..cursor.at + close])); + cursor.at += close + 1; + cursor.skip_whitespace(); + } + cursor.visibility(); + + if !cursor.keyword_then_space("mod") { + continue; + } + let Some(name) = cursor.word() else { + continue; + }; + cursor.skip_whitespace(); + if masked[cursor.at..].starts_with([';', '{']) { + modules.push(TestModule { + name: name.to_owned(), + path, + }); + } + } + + modules +} + +/// The file in `path = "..."`, the inside of a `#[path]` attribute. +fn path_attribute(attribute: &str) -> Option { + let value = attribute + .trim_start() + .strip_prefix("path")? + .trim_start() + .strip_prefix('=')? + .trim(); + + value + .strip_prefix('"')? + .strip_suffix('"') + .map(str::to_owned) +} + +/// `lib.rs`, else `main.rs`, of the crate whose sources hold `path`. +fn crate_source_root(tree: &Tree, path: &Path) -> Option { + let base = if path.starts_with("src") { + Path::new("src") + } else if path.starts_with("configurator/src") { + Path::new("configurator/src") + } else { + return None; + }; + + ["lib.rs", "main.rs"] + .iter() + .map(|name| base.join(name)) + .find(|candidate| tree.contains(candidate)) +} + +/// The file holding `module`'s own items: `foo.rs` or `foo/mod.rs`. +fn module_file(tree: &Tree, module: &Path) -> Option { + [module.with_extension("rs"), module.join("mod.rs")] + .into_iter() + .find(|candidate| tree.contains(candidate)) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn masking_keeps_offsets_and_hides_only_non_code() { + let source = "a // b {\nc /* d /* e */ f */ g\n\"h { \\\" i\" r##\"j \"# k\"## \ + b\"l\" br#\"m\"# 'n' '\\'' '\\u{7B}' 'o: r#p {"; + let masked = mask_non_code(source); + + assert_eq!(masked.len(), source.len()); + assert_eq!(masked.matches('\n').count(), source.matches('\n').count()); + for hidden in ["b {", "d", "e", "f", "h {", "i", "j", "k", "l", "m", "n"] { + assert!(!masked.contains(hidden), "{hidden:?} in {masked:?}"); + } + for kept in ["a ", "c ", " g", "'o: r#p {"] { + assert!(masked.contains(kept), "{kept:?} missing from {masked:?}"); + } + } +} diff --git a/tools/AGENTS.md b/tools/AGENTS.md index af1bcd5b2..d8d9dd1b3 100644 --- a/tools/AGENTS.md +++ b/tools/AGENTS.md @@ -7,11 +7,12 @@ - `wayscriber.cs` is the C# entry point used by CI and exposes parity commands for local development, installation, packaging, and releases. - Keep its entry point small, command modules under `csharp/Commands/`, infrastructure under `csharp/Infrastructure/`, and module lists explicit in `csharp/includes.cs`. - Build the C# file apps once, then invoke them with `dotnet run tools/.cs --no-build -- ...`. -- Existing shell and Python tools remain standalone fallbacks for contributors without .NET. Production C# commands must not invoke them. The C# parity test app may execute the retained standalone release contracts so both implementations stay covered in CI. -- Scripts support build, install, lint/test, versioning, packaging, release tags, package repository generation, daemon reload, and dependency fetching. +- Each repository check has one implementation. Source ownership invariants (shared dependencies, process sites, config writers) are Rust tests in `tests/repository_guards` and run in every `cargo test`. Release, packaging, build-metadata, and source-coverage checks are C# commands. Do not add Python (`tests/repository_guards/no_python.rs` enforces this), and do not add another copy of a check. +- The remaining shell tools stay usable without .NET. Production C# commands must not invoke them. The C# test app may execute the retained shell contracts (`test-package-repo-layout.sh`, `test-release-packaging.sh`) and a copy of `lint-and-test.sh` against fake `dotnet` and `cargo` commands. +- Scripts support build, install, lint/test, packaging, package repository generation, daemon reload, and dependency fetching. Versioning and release tags are C# commands only (`version bump`, `version check`, `release create-tag`, `release publish-tag`). - Scripts should resolve the repository root and work from any starting directory. -- The shell and Python fallbacks must remain usable without .NET and must not redirect to `wayscriber.cs`. -- Version-bump regressions live in `test-release-packaging.sh` and `wayscriber.tests.cs`. +- Shell tools must not redirect to `wayscriber.cs`, except `lint-and-test.sh`: it is the complete gate and runs the C# checks, so it requires .NET. +- Version check, bump, and release-tag regressions live in the C# tests (`csharp-tests/VersionConsistencyTests.cs`, `VersionReleaseCommandTests.cs`). - C# helper formatting follows `tools/.editorconfig`: spaced parentheses and braced guards, with LF endings required by `.gitattributes`. - Separate logical steps in C# functions with a blank line. Keep closely related validation, setup, execution, state checks, and result mapping together, and add spacing when the purpose changes. - Assign or deserialize a value first, then use a separate braced null guard. Do not embed `throw` in an assignment, return, or conditional expression. @@ -21,12 +22,12 @@ - Keep cyclomatic complexity at or below 20 per method. `CA1502` is an error, with the threshold in `CodeMetricsConfig.txt` wired through `Directory.Build.props`. ## Invariants -- Keep C# commands and standalone script behavior aligned. Add parity fixtures to `wayscriber.tests.cs` when either implementation changes. +- Keep C# commands and any retained shell script behavior aligned. Add fixtures to `wayscriber.tests.cs` when either changes. - Invoke external programs from C# with `ProcessStartInfo.ArgumentList`; do not invoke a shell interpreter or assemble shell command strings. - Preserve release/version/package semantics, including packaging-only hotfix behavior. - Keep `tools/lint-and-test.sh` aligned with CI. - The canonical gate serializes Rust test harnesses as a native-font race workaround and runs the ignored context-menu and board-picker render regressions separately under both feature configurations. Preserve their assertions and document the workaround separately from any native-library fix. -- Keep `check-rust-source-coverage.py` aligned with the workspace's all-feature and +- Keep `check rust-source-coverage` aligned with the workspace's all-feature and no-default-feature target matrix; intentional exceptions must be narrow and documented. - Avoid platform-specific assumptions unless the script is explicitly platform-specific. diff --git a/tools/README.md b/tools/README.md index 270a19d82..657f73976 100644 --- a/tools/README.md +++ b/tools/README.md @@ -4,10 +4,11 @@ Helper scripts for development, installation, packaging, and release workflows. ## C# automation -CI uses the file-based app at `tools/wayscriber.cs`. It also provides C# versions -of the local commands below, while the standalone shell and Python tools remain -available for contributors who do not have .NET installed. Production C# commands -never call those scripts. +CI uses the file-based app at `tools/wayscriber.cs`. Some commands below exist +only there (the repository checks, the code-health report, and the version and +release-tag commands); the shell tools listed remain available for contributors +who do not have .NET installed. Production C# commands never call those scripts. +The repository uses no Python. The exact SDK is pinned by `global.json` and installed by GitHub Actions through `actions/setup-dotnet`. Nix does not provide .NET. Local users who choose the C# @@ -41,19 +42,24 @@ available when .NET is not installed. Use `./tools/install.cs help` to show the C# installation commands; `--help` is reserved by `dotnet run` when the file is launched through its shebang. -The regression suite also executes the retained package-repository, release-packaging, -and AUR desktop-asset shell contracts. Production C# commands do not invoke those -fallback scripts. - -`./tools/lint-and-test.sh` always runs those three retained shell contracts. When -the pinned .NET SDK is available, it also builds and runs the C# regression suite; -without .NET, it reports that optional local check as skipped and continues with -the standalone checks. - -Common equivalents are `dev build`, `dev test`, `dev fetch`, `ci lint-and-test`, -`ci gtk-widgets`, `install app`, `install configurator`, `version bump`, -`package build`, `release publish-tag`, and `aur update`. Run `--help` for the -complete command list and options. +The regression suite also executes the retained package-repository and +release-packaging shell contracts, and a copy of `lint-and-test.sh` against fake +`dotnet` and `cargo` commands. Production C# commands do not invoke those shell +scripts. + +`./tools/lint-and-test.sh` is the complete local gate and needs the .NET SDK pinned +by `global.json`; without it the gate stops before running anything. It builds the +C# apps, then runs the same steps as `ci lint-and-test`: the C# repository checks, +C# formatting, the C# regression suite (with the two retained shell contracts), then +the Cargo format, lint, build, and test steps. +Without .NET, `cargo test --workspace --all-features` still runs the Rust source +guards in `tests/repository_guards`, but it is not the complete gate. + +C# equivalents of the shell tools include `dev build`, `dev test`, `dev fetch`, +`ci gtk-widgets`, `install app`, `install configurator`, `package build`, and +`aur update`. `ci lint-and-test` is what `tools/lint-and-test.sh` runs after building +the C# apps. `version bump`, `version check`, and the `release` tag commands have no +shell version. Run `--help` for the complete command list and options. ## Development @@ -69,22 +75,23 @@ complete command list and options. - Runs `cargo test --workspace` - Usage: `./tools/test.sh` -- **code-health-report.sh** - Report local maintainability metrics +- **report code-health** - Report local maintainability metrics - Reports Rust files over 500 lines, functions over 120 lines, production unwrap/expect/panic/unsafe markers, selected allowances, and direct `fs::write` usage - Does not fail on reported findings; intended for baseline visibility before adding quality gates - - Usage: `./tools/code-health-report.sh` + - Usage: `dotnet run tools/wayscriber.cs --no-build -- report code-health` -- **check-rust-source-coverage.py** - Reject Rust sources outside the supported Cargo module graph +- **check rust-source-coverage** - Reject Rust sources outside the supported Cargo module graph - Uses current rustc dep-info from all-target/all-feature and no-default-feature checks - Runs as a hard gate in local and GitHub CI - - Usage: `./tools/check-rust-source-coverage.py` + - Usage: `dotnet run tools/wayscriber.cs --no-build -- check rust-source-coverage` -- **check-config-writers.py** - Reject config-write capability outside the configurator's Save - - Scans `src/` and `configurator/src/` for the `config.toml` write primitives, exempting test sources and inline `#[cfg(test)]` items - - Allows only `src/config/document.rs`, `src/config/io.rs`, and `configurator/src/app/io.rs` - - Also checks those files keep the capability narrow (one public save, `pub(super)` primitives) - - Runs as a hard gate in `tools/lint-and-test.sh` - - Usage: `./tools/check-config-writers.py` +- **tests/repository_guards** - Rust source guards that run in every `cargo test` + - `config_writers.rs` rejects `config.toml` write capability outside the configurator's Save and the overlay's pinned narrow editors + - `process_sites.rs` keeps process creation inside the process broker and audits the broker's post-fork child stub + - `shared_dependencies.rs` keeps the shared domain and config validation layers free of upward crate paths, using the syntax corpus beside it + - `no_python.rs` rejects Python files and links to them, Python project and lock files, Python shebangs, and Python interpreter or package names in the Rust, C#, MSBuild, shell, Nix, TOML, workflow, build, service, desktop-entry, packaging, and extensionless files it reads; its known limits are listed at the top of the file + - Each guard carries regression fixtures for the escapes it forbids + - Usage: `cargo test --test repository_guards` - **reload-daemon.sh** - Restart running daemon - Kills and restarts the daemon to pick up config/code changes @@ -111,8 +118,8 @@ complete command list and options. ## Version & Release -- **bump-version.sh** - Bump version numbers - - Checks offline dependency resolution before changing version files; run `./tools/fetch-all-deps.sh` if the cache is incomplete. +- **version bump** - Bump version numbers + - Checks offline dependency resolution before changing version files; run `dev fetch` (or `./tools/fetch-all-deps.sh`) if the cache is incomplete. - Updates Cargo.toml, configurator/Cargo.toml, the workspace Cargo.lock, PKGBUILD, and .SRCINFO - Updates only workspace packages in the lockfile, offline; existing dependency versions stay locked - flake.nix package version follows Cargo.toml automatically @@ -120,13 +127,14 @@ complete command list and options. - Use this in the same change as a user-visible overlay/settings/config toggle, or immediately before tagging that release, so `--version` is not identical to the last shipped crate - Supports MAJOR.MINOR.PATCH.HOTFIX for packaging-only hotfix releases - - Usage: `./tools/bump-version.sh [--dry-run] [new_version]` + - Rolls every version file back if the result fails `version check` + - Usage: `dotnet run tools/wayscriber.cs --no-build -- version bump [--dry-run] [X.Y.Z[.N]]` -- **check-version-consistency.sh** - Check release metadata alignment +- **version check** - Check release metadata alignment - Verifies Cargo manifests, the workspace lockfile, packaging metadata, flake version sourcing, and that the flake compares the selected Rust toolchain to Cargo.toml rust-version - Keeps the configurator's libadwaita 1.4 floor aligned across Cargo, deb, rpm, PKGBUILD, and `.SRCINFO` - With `--release-version X.Y.Z[.N]`, rejects tags that do not match Cargo or an explicit packaging hotfix of Cargo - - Usage: `bash tools/check-version-consistency.sh [--release-version X.Y.Z[.N]]` + - Usage: `dotnet run tools/wayscriber.cs --no-build -- version check [--release-version X.Y.Z[.N]]` Packaging-only hotfix policy: - Normal releases use one version everywhere: Cargo, package metadata, Git tags, and artifacts all use `X.Y.Z`. @@ -134,17 +142,17 @@ Packaging-only hotfix policy: - Repo `packaging/PKGBUILD` and `packaging/.SRCINFO` are templates and keep `sha256sums=('SKIP')` because the final GitHub tag archive checksum can only be computed after the tag exists. AUR automation writes the real checksum into external AUR metadata. - Release builds set `WAYSCRIBER_RELEASE_VERSION`, so packaged binaries report the release artifact version. Nix builds follow Cargo and report `X.Y.Z` unless the Cargo version itself is bumped. -- **create-release-tag.sh** - Create git tag (local only) +- **release create-tag** - Create git tag (local only) - Creates annotated tag `v` without pushing - Requires clean working tree - Runs version consistency checks before tagging - - Usage: `./tools/create-release-tag.sh ` (X.Y.Z or X.Y.Z.N) + - Usage: `dotnet run tools/wayscriber.cs --no-build -- release create-tag X.Y.Z[.N]` -- **publish-release-tag.sh** - Create and push git tag +- **release publish-tag** - Create and push git tag - Creates annotated tag and pushes to origin - Auto-detects version from Cargo.toml if not specified - Runs version consistency checks before tagging - - Usage: `./tools/publish-release-tag.sh [--version X.Y.Z[.N]] [--dry-run]` + - Usage: `dotnet run tools/wayscriber.cs --no-build -- release publish-tag [--version X.Y.Z[.N]] [--dry-run]` See [Releasing](../docs/RELEASING.md) for validation, website release notes, and the final update-manifest publication step. Pushing a tag does not update the website notice. @@ -185,24 +193,19 @@ automatically by the nixpkgs-update bot. The bot only rewrites the version and hashes, so build-level changes still need a pull request from us. See `packaging/nixpkgs/README.md`. -- **check-nixpkgs-recipe.py** - Check the nixpkgs build declares what the default features need - - Uses locked Cargo metadata, so it works with Python 3.10 without an extra TOML parser +- **check nixpkgs-recipe** - Check the nixpkgs build declares what the default features need + - Uses locked Cargo metadata - Maps every direct normal Cargo dependency, including target-specific dependencies, to the nixpkgs system packages it links - Keeps required native inputs, including the GTK application wrapper, aligned between the recipe and flake - Fails when a Linux default-feature dependency is missing from `packaging/nixpkgs/package.nix` or `flake.nix` - Fails on any new direct normal dependency until its system requirements are declared - Runs as a hard gate in GitHub CI and before release packaging - - Usage: `./tools/check-nixpkgs-recipe.py` + - Usage: `dotnet run tools/wayscriber.cs --no-build -- check nixpkgs-recipe` ## AUR (Arch User Repository) -- **aur-desktop-assets.sh** / **aur-desktop-assets.py** - Standalone asset recipe generator - - Reads and validates the package manifests without .NET. - - Usage: `bash tools/aur-desktop-assets.sh REPO_ROOT`. - -- **wayscriber assets emit** - C# asset recipe generator used by CI - - Reads both package YAML manifests and validates asset paths and integer permissions. - - Standalone shell tools keep their own implementation and do not require .NET. +- **wayscriber assets emit** - Desktop asset recipe generator used by CI + - Reads both package YAML manifests, validates asset paths, and requires mode 0644 as a plain YAML integer. - **update-aur.sh** - Interactive AUR update - Updates PKGBUILD, tests build locally, pushes to AUR @@ -228,7 +231,7 @@ All scripts work from any location in the project. ### Potential Overlaps -The release/tag scripts have overlapping functionality: -- `create-release-tag.sh` + push = `publish-release-tag.sh` +The release tag commands have overlapping functionality: +- `release create-tag` + push = `release publish-tag` -Use the individual scripts in sequence for full releases when you need explicit control over each step. +Use the individual commands in sequence for full releases when you need explicit control over each step. diff --git a/tools/aur-desktop-assets.py b/tools/aur-desktop-assets.py deleted file mode 100755 index 34e64cbcd..000000000 --- a/tools/aur-desktop-assets.py +++ /dev/null @@ -1,177 +0,0 @@ -#!/usr/bin/env python3 -"""Standalone counterpart of the C# AUR desktop-asset recipe command.""" - -import json -import pathlib -import re -import sys - - -class ManifestError(Exception): - pass - - -ASSET_PREFIXES = ( - "/usr/share/applications/", - "/usr/share/icons/", - "/usr/share/pixmaps/", -) - - -def parse_contents(root: pathlib.Path, package: str, filename: str): - path = root / "packaging" / filename - lines = path.read_text(encoding="utf-8").splitlines() - content_headers = [index for index, line in enumerate(lines) if line == "contents:"] - if len(content_headers) != 1: - raise ManifestError(f"{filename}: expected one package contents sequence") - - entries = [] - current = None - in_contents = False - for line in lines[content_headers[0] + 1 :]: - if line and not line.startswith(" "): - break - if line.startswith(" - "): - if current is not None: - entries.append(current) - current = {} - in_contents = True - parse_field(current, line[4:], filename) - elif current is not None and line.startswith(" "): - stripped = line[4:] - if stripped == "file_info:": - current["file_info"] = True - elif line.startswith(" "): - parse_field(current, line[6:], filename) - elif stripped and not stripped.startswith("#"): - parse_field(current, stripped, filename) - elif line.strip() and in_contents: - raise ManifestError(f"{filename}: invalid content entry") - if current is not None: - entries.append(current) - if not in_contents: - raise ManifestError(f"{filename}: expected one package contents sequence") - - assets = [] - destinations = set() - for entry in entries: - destination = scalar(entry, "dst") - if not destination.startswith(ASSET_PREFIXES): - continue - source = scalar(entry, "src") - validate_asset(root, filename, entry, source, destination) - if destination in destinations: - raise ManifestError(f"{filename}: duplicate desktop destination {destination}") - destinations.add(destination) - assets.append((source, destination)) - - if ( - f"/usr/share/applications/{package}.desktop" not in destinations - or not any(item.startswith("/usr/share/icons/") for item in destinations) - ): - raise ManifestError(f"{filename}: launcher and icons are required") - return assets - - -def parse_field(entry, text, filename): - if ":" not in text: - raise ManifestError(f"{filename}: invalid content entry") - key, value = text.split(":", 1) - entry[key.strip()] = value.strip() - - -def scalar(entry, key): - value = entry.get(key) - if not value: - raise ManifestError(f"Expected scalar {key}") - return value - - -def validate_asset(root, filename, entry, source, destination): - if ( - not re.fullmatch(r"packaging/[A-Za-z0-9_./-]+", source) - or not re.fullmatch(r"/usr/share/[A-Za-z0-9_./-]+", destination) - or ".." in source.split("/") - or ".." in destination.split("/") - ): - raise ManifestError(f"{filename}: unsupported asset path {source} -> {destination}") - if not entry.get("file_info"): - raise ManifestError( - f"{filename}: desktop asset {destination} has no file_info mapping" - ) - mode = entry.get("mode", "") - try: - if not re.fullmatch(r"(?:0o[0-7]+|0[0-7]+|[1-9][0-9]*|0)", mode): - raise ValueError - number = int(mode[2:], 8) if mode.lower().startswith("0o") else int(mode, 8 if mode.startswith("0") else 10) - except ValueError: - number = -1 - if number != 0o644: - raise ManifestError( - f"{filename}: desktop asset {destination} must have mode 0644" - ) - if not (root / source).is_file(): - raise ManifestError(f"{filename}: missing desktop asset {source}") - - -def recipe(label, title, binary, package, assets, from_archive): - lines = [] - destinations = [] - for source, destination in assets: - input_path = f'"${{srcdir_tmp}}{destination}"' if from_archive else source - lines.append(f' install -Dm644 {input_path} "$pkgdir{destination}"') - destinations.append(destination) - return { - "label": label, - "marker": f"# {title} desktop integration", - "end_marker": f"# End {title} desktop integration", - "anchor": f' install -Dm755 "{binary}" "$pkgdir/usr/bin/{package}"', - "lines": lines, - "destinations": destinations, - } - - -def main(): - if len(sys.argv) != 2: - raise ManifestError("Usage: aur-desktop-assets.py REPO_ROOT") - root = pathlib.Path(sys.argv[1]).resolve() - main_assets = parse_contents(root, "wayscriber", "package.wayscriber.yaml") - configurator_assets = parse_contents( - root, "wayscriber-configurator", "package.configurator.yaml" - ) - result = { - "desktop_path_pattern": "/usr/share/applications/|/usr/share/icons/|/usr/share/pixmaps/", - "source": recipe( - "wayscriber source", - "Wayscriber", - "target/release/wayscriber", - "wayscriber", - main_assets, - False, - ), - "bin": recipe( - "wayscriber bin", - "Wayscriber", - "${srcdir_tmp}/usr/bin/wayscriber", - "wayscriber", - main_assets, - True, - ), - "configurator": recipe( - "wayscriber-configurator", - "Wayscriber configurator", - "target/release/wayscriber-configurator", - "wayscriber-configurator", - configurator_assets, - False, - ), - } - output = json.dumps(result, separators=(",", ":")).replace('\\"', "\\u0022") - print(output) - - -try: - main() -except Exception as error: - print(f"Desktop asset manifest error: {error}", file=sys.stderr) - sys.exit(1) diff --git a/tools/aur-desktop-assets.sh b/tools/aur-desktop-assets.sh deleted file mode 100755 index 9d58dfa29..000000000 --- a/tools/aur-desktop-assets.sh +++ /dev/null @@ -1,19 +0,0 @@ -#!/usr/bin/env bash -# Standalone asset recipe generator for contributors who do not use .NET. -set -euo pipefail - -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)" - -if ! command -v python3 >/dev/null 2>&1; then - echo 'AUR packaging checks require Python 3; see CONTRIBUTING.md.' >&2 - exit 1 -fi -if [[ "${1:-}" == --check && "$#" -eq 1 ]]; then - exit 0 -fi -if [[ "$#" -ne 1 ]]; then - echo 'Usage: bash tools/aur-desktop-assets.sh REPO_ROOT | --check' >&2 - exit 1 -fi - -exec python3 "$SCRIPT_DIR/aur-desktop-assets.py" "$1" diff --git a/tools/bump-version.sh b/tools/bump-version.sh deleted file mode 100755 index 268fe46fe..000000000 --- a/tools/bump-version.sh +++ /dev/null @@ -1,172 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -usage() { - cat <<'EOF' -Usage: tools/bump-version.sh [--dry-run] [new_version] - -- If new_version is omitted, bumps the patch version (e.g., 0.9.2 -> 0.9.3). -- new_version can be MAJOR.MINOR.PATCH or MAJOR.MINOR.PATCH.HOTFIX. -- HOTFIX versions are packaging-only: Cargo and flake.nix stay on MAJOR.MINOR.PATCH; - packaging/PKGBUILD, packaging/.SRCINFO, and release artifacts use the HOTFIX version. -- Updates: - * Cargo.toml (wayscriber) - * configurator/Cargo.toml - * Cargo.lock (workspace versions only; preserves locked dependencies) - * flake.nix package version follows Cargo.toml automatically - * packaging/PKGBUILD pkgver and template sha256sums=('SKIP') - * packaging/.SRCINFO (via makepkg --printsrcinfo) - -The repo PKGBUILD is a template: release/AUR automation computes the real -source archive checksum after the tag exists. - -Requires: cargo, jq, makepkg, sed, perl, python3 or python pointing to Python 3. -EOF -} - -require_bin() { - if ! command -v "$1" >/dev/null 2>&1; then - echo "error: $1 is required" >&2 - exit 1 - fi -} - -if [[ "${1-}" == "-h" || "${1-}" == "--help" ]]; then - usage - exit 0 -fi - -require_bin cargo -require_bin jq -require_bin makepkg -require_bin sed -require_bin perl -if ! { command -v python3 >/dev/null 2>&1 && python3 -c 'import sys; raise SystemExit(0 if sys.version_info[0] == 3 else 1)' >/dev/null 2>&1; } \ - && ! { command -v python >/dev/null 2>&1 && python -c 'import sys; raise SystemExit(0 if sys.version_info[0] == 3 else 1)' >/dev/null 2>&1; }; then - echo "error: python3 or python pointing to Python 3 is required" >&2 - exit 1 -fi - -REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" -cd "$REPO_ROOT" - -current_version="$(cargo metadata --no-deps --format-version 1 | jq -r '.packages[] | select(.name=="wayscriber") | .version')" -if [[ -z "$current_version" ]]; then - echo "error: unable to detect current wayscriber version" >&2 - exit 1 -fi - -DRY_RUN=false -while [[ $# -gt 0 ]]; do - case "$1" in - --dry-run) - DRY_RUN=true - shift - ;; - -h|--help) - usage - exit 0 - ;; - --) - shift - break - ;; - -*) - usage - exit 1 - ;; - *) - break - ;; - esac -done - -if [[ $# -gt 1 ]]; then - usage - exit 1 -fi - -if [[ $# -eq 1 ]]; then - next_version="$1" -else - IFS='.' read -r major minor patch <<<"$current_version" - patch=$((patch + 1)) - next_version="${major}.${minor}.${patch}" -fi - -if ! [[ "$next_version" =~ ^[0-9]+\.[0-9]+\.[0-9]+(\.[0-9]+)?$ ]]; then - echo "error: invalid version format: $next_version (expected MAJOR.MINOR.PATCH[.HOTFIX])" >&2 - exit 1 -fi - -IFS='.' read -r major minor patch hotfix extra <<<"$next_version" -if [[ -n "${extra:-}" ]]; then - echo "error: invalid version format: $next_version (too many segments)" >&2 - exit 1 -fi - -cargo_version="${major}.${minor}.${patch}" -package_version="${next_version}" -if [[ -z "${hotfix:-}" ]]; then - cargo_version="${next_version}" -fi - -echo "Current version: $current_version" -echo "Bumping to: $next_version" -if [[ "$cargo_version" != "$next_version" ]]; then - echo "Cargo version: $cargo_version (release version has hotfix)" -fi - -update_version_field() { - local file="$1" - if ! [[ -f "$file" ]]; then - echo "warn: $file not found, skipping" >&2 - return - fi - if $DRY_RUN; then - echo "dry-run: would update version in $file" - else - NEXT_VERSION="$cargo_version" perl -0777 -pi -e 's/(\[package\][^\[]*?\nversion\s*=\s*")\K[^"]+/$ENV{NEXT_VERSION}/s' "$file" - fi -} - -# Resolve against the existing manifests without writing the lockfile. An empty -# registry cache must fail before either manifest or any package metadata changes. -if [[ -f Cargo.lock ]]; then - if ! cargo update --workspace --offline --dry-run >/dev/null; then - echo "error: cannot resolve locked dependencies offline; run ./tools/fetch-all-deps.sh before bumping the version. No version files changed." >&2 - exit 1 - fi -fi - -update_version_field "Cargo.toml" -update_version_field "configurator/Cargo.toml" - -if [[ -f Cargo.lock ]]; then - if $DRY_RUN; then - echo "dry-run: would update workspace versions in Cargo.lock, preserving dependencies" - else - cargo update --workspace --offline >/dev/null - fi -else - echo "warn: Cargo.lock not found, skipping lockfile update" >&2 -fi - -if [[ -f packaging/PKGBUILD ]]; then - if $DRY_RUN; then - echo "dry-run: would set pkgver=${package_version}, reset sha256sums=SKIP, and regenerate .SRCINFO" - else - sed -i "s/^pkgver=.*/pkgver=${package_version}/" packaging/PKGBUILD - sed -i "s/^sha256sums=.*/sha256sums=('SKIP')/" packaging/PKGBUILD - (cd packaging && makepkg --printsrcinfo > .SRCINFO) - fi -else - echo "warn: packaging/PKGBUILD not found, skipping PKGBUILD/.SRCINFO" >&2 -fi - -if $DRY_RUN; then - echo "Dry run complete (no changes made)" -else - bash tools/check-version-consistency.sh --release-version "$package_version" - echo "Updated versions to $next_version" -fi diff --git a/tools/check-config-writers.py b/tools/check-config-writers.py deleted file mode 100755 index dbe6d8efa..000000000 --- a/tools/check-config-writers.py +++ /dev/null @@ -1,1191 +0,0 @@ -#!/usr/bin/env python3 -"""Fail when production code outside the reviewed writers can write `config.toml`. - -`config.toml` is an authored input. It changes only through an explicit user -edit action, never as a side effect of running Wayscriber. Two kinds of writer -are allowed: - -* the configurator's **Save**, which writes the whole edited draft; and -* the overlay's **narrow editors** in `src/config/io.rs`, one per explicit - gesture, each of which rewrites only its own key and backs the file up first. - -Everything else — startup, shutdown, the daemon, the tray, validation, migration -preview, and every incidental preference toggle — reads the file and never -writes it. The capability is one `use` away, so this checks for its absence -instead of remembering it. - -Six things are enforced. First, the write primitives are named nowhere outside -`src/config/` and the configurator's Save adapter. Second, each narrow editor's -production call sites are pinned by name: a new caller of one of them is a new -place `config.toml` can change, which is a review decision rather than an -implementation detail, so it has to be recorded in `NARROW_WRITERS` below. -Third, the write-capable surface of the two implementing files — -`src/config/document.rs` and `src/config/io.rs` — is pinned the same way. Both -are exempt from the primitive scan because they *are* the write, so their own -exported functions are enumerated instead: every one that can reach a write — -directly or through the file's own private helpers — has to be recorded in -`DOCUMENT_WRITE_SURFACE` or `IO_WRITE_SURFACE`. The walk is what makes the pin -about the capability rather than about the spelling: a `pub fn -persist_any_config(..) { self.merge_and_write(..) }` in `document.rs` reaches -the file exactly as far as `save_with_backup` does, and is caught by name and -line even though nothing about its name says "save". - -Visibility is not the whole of the surface either, so it is not the whole of -that pin. A method inside `impl SomeTrait for ConfigDocument` carries no `pub` -of its own, and neither does a trait's default method: the trait's visibility is -what makes them callable, and the trait is one `pub use` away from the rest of -the crate. In the three implementing files — `document.rs`, `io.rs`, and -`mod.rs` — a write reachable through a trait therefore fails whatever it is -called and whatever visibility it was given, and so does a `trait` declared -there whose methods reach a write, including the bodiless signature whose impl -sits elsewhere in the file. The reviewed writers are inherent functions and free -functions, which is the only shape the surfaces above enumerate. - -Fourth, `src/config/mod.rs` is held to being a re-export list. It is the other -file exempt from caller collection, because it is where the editors leave the -module, so a writer's name may appear there inside a `use` or `pub use` item and -nowhere else, and no function declared there may reach a writer — directly, -through `crate::config::io::`, or through another wrapper in the same file. The -same write-capability walk `io.rs` gets runs over it, so `pub fn -concealed_edit(..) { io::persist_keybinding_edit(..) }` fails by name and line -instead of being invisible to every other check and callable from anywhere. - -Fifth, because everything above matches call sites by *name*, the names -themselves are pinned. Any file in `src/config/` may re-export the editors but -never a primitive. No production file anywhere may re-export or import a -write-capable name under a different one: both `pub use -io::persist_keybinding_edit as concealed_edit;` and, in the calling file, -`use crate::config::persist_preset_slot as save;` would leave a call site the -pins cannot recognise, so the rename itself fails. Inside `src/config/` the -editors are named only where they are declared (`io.rs`) and where they leave -(`mod.rs`), so a wrapper in a third file cannot re-offer the capability under a -name of its own. And the editors' path-taking twins, which exist for the suites, -are named in production nowhere at all. - -Sixth, a write may not leave one of those files as a *value*. Everything above -reads functions — the surfaces enumerate them, and the capability walk follows -calls between them — and a `pub const PERSIST_ANY_CONFIG: fn(..) = -ConfigDocument::save_with_backup;` declares no function at all. It is the same -capability with a type in place of a body, exported, callable by anyone who can -name it, and it used to walk out through a matching `pub use document:: -PERSIST_ANY_CONFIG;` with every pin here reading past both halves: not a -primitive, not a function, not a rename. Both halves now fail. In the three -implementing files a write-capable name inside a `const` or `static` -initializer fails by name and line; and a `pub use` anywhere in `src/config/` -fails when it re-exports a `fn`, `const`, or `static` *declared* in -`document.rs` or `io.rs` and not among the three reviewed editors. The second -half asks what a name is and where it came from rather than whether it is on a -list of known writers, which is what makes it hold for a value nothing here has -heard of. Types are deliberately outside it: `ConfigDocument` and -`ConfigEditOutcome` are most of what the module re-exports, they carry no -capability, and the methods on them are already pinned by the surfaces. - -Scope and limits: this is a name-level guardrail over `src/` and -`configurator/src/`, not the proof. It catches a new caller of the config write -primitives; it cannot catch a brand-new write built directly on -`durable_io::write_text_atomic` under a different name. Renaming is checked at -`use` items, which is where a rename has to be written; a call reached some -other way still names the function and is caught by the pins (a module alias, -`use crate::config::io as elsewhere;` followed by `elsewhere::persist_quick_color -(..)`, spells the writer at the call site). Three known holes remain: a macro -defined inside `src/config/` that expands to a write, whose invocation names only -the macro; any path that composes the identifier rather than writing it, such -as `concat_idents!`; and a writer put into a value at *runtime* rather than in a -`const` — a struct field assigned `save_with_backup` inside some function body, -handed out later through that struct. The last one is narrower than it sounds: -the assignment spells the writer's name inside a function, which is exactly what -the capability walk reads, so the enclosing function becomes write-capable and -the surface pins fail it if it is visible outside the file. What is genuinely -uncovered is a value built and consumed entirely within one writer file's -private functions, which reaches no further than those functions already do. The behavioural proof is the loader immutability fixture in -`src/config/tests/immutability.rs` plus the per-flow "only this key changed" -tests beside each gesture that queues a write. Test sources are exempt, and -whether a file is one is read from the `#[cfg(test)]` on the `mod` item that -brings it in rather than from the shape of its path — a directory called -`tests` under `src/` is production code unless the compiler is told otherwise. -`src/daemon/tests.rs` keeps its own -in-tree version of this check for the daemon subtree, which stays useful because -it runs under `cargo test` rather than only in the full lint gate. -""" - -from __future__ import annotations - -import re -import sys -from pathlib import Path - - -ROOT = Path(__file__).resolve().parent.parent - -DOCUMENT_SOURCE = Path("src/config/document.rs") -IO_SOURCE = Path("src/config/io.rs") -CONFIG_MODULE = Path("src/config/mod.rs") -CONFIGURATOR_ADAPTER = Path("configurator/src/app/io.rs") - -# The files that implement the single durable write, plus the configurator -# adapter that performs it. `src/config/document/merge.rs` is deliberately -# absent: it rewrites a TOML tree in memory and never touches the filesystem, so -# it names none of the primitives below. -WRITE_ALLOWLIST = { - DOCUMENT_SOURCE, # ConfigDocument::save_with_backup, the only durable write - IO_SOURCE, # the atomic write, the timestamped .bak copy, the parent mkdir - CONFIGURATOR_ADAPTER, # the configurator's explicit Save -} - -# Every name that reaches the filesystem on the config path. -WRITE_PRIMITIVES = ( - "save_with_backup", - "write_config_text_atomic", - "create_config_backup", - "prepare_config_parent", -) - -WRITE_PATTERNS = tuple( - re.compile(rf"\b{re.escape(name)}\b") for name in WRITE_PRIMITIVES -) - -# The write-capable surface `src/config/document.rs` is allowed to expose. -# -# One entry, and it is the whole of the application's durable config write. -# The file is exempt from the primitive scan below — it owns the merge, the -# backup, and the rename — so a second exported entry point here would be a -# second way for `config.toml` to change with nothing else in this check to -# notice it. Matching on the *name* is not enough: an entry point called -# anything at all reaches the file if it can reach `merge_and_write`, so what is -# pinned is the reachability. -DOCUMENT_WRITE_SURFACE = { - "save_with_backup", -} - -# What "can write" means inside `document.rs`: the merge-and-rename step every -# save funnels through, plus the primitives it calls. The walk is a fixpoint, so -# naming the step is belt and braces — a helper that called the primitives -# directly would be caught by them — but it keeps the pin meaningful if the -# primitives ever move behind another name inside this file. -DOCUMENT_WRITE_SEEDS = { - "merge_and_write", - *WRITE_PRIMITIVES, -} - -# The write-capable surface `src/config/io.rs` is allowed to expose, by name. -# -# The file is exempt from the primitive scan above — it *is* the primitive — so -# without this a new `pub fn` there could wrap the atomic write under any name, -# be re-exported, and be called from anywhere with nothing to notice it. Each -# entry is a reviewed capability: -# -# * the three narrow editors, one per explicit user gesture; -# * their path-taking twins, the same gesture without the process environment, -# which the suites drive and which are `#[cfg(test)]`-gated so a production -# build has no such function at all; and -# * the three primitives themselves, which `pub(super)` already confines to the -# config module and which `document.rs` is the only caller of. -# -# Anything else in `io.rs` that can reach a primitive — directly or through the -# file's own private helpers — is a new way for `config.toml` to change, which -# is a review decision rather than an implementation detail. -IO_WRITE_SURFACE = { - "persist_keybinding_edit", - "persist_keybinding_edit_at", - "persist_preset_slot", - "persist_preset_slot_at", - "persist_quick_color", - "persist_quick_color_at", - *WRITE_PRIMITIVES, -} - -# The one file that calls the editors. The writes run on a worker thread rather -# than on the overlay's dispatch thread — a parse, a file copy, a rename, and two -# fsyncs are not work for the thread that reads input and paints — so the three -# gestures hand a typed edit to `config_edits.rs` and route its completion when -# it comes back. That makes this module the caller of record for all three. -EDIT_WORKER = Path("src/backend/wayland/config_edits.rs") - -# Where each gesture is still decided, and where its "only this key changed" -# test lives. Not writers any more, but named so the walk below can assert it -# still reaches them: a check that stopped seeing these files would stop proving -# anything about the gestures. -SHORTCUT_CALL_SITE = Path("src/backend/wayland/state/keybindings.rs") -PRESET_CALL_SITE = Path("src/backend/wayland/state/toolbar/events/presets.rs") -QUICK_COLOR_CALL_SITE = Path("src/backend/wayland/state/toolbar/events/quick_colors.rs") - -# The overlay's narrow editors: one entry per explicit user gesture that may -# change `config.toml`, mapped to the production files allowed to invoke it. -# Each is declared in `src/config/io.rs`, writes exactly its own key on top of -# `ConfigDocument::config()`, and backs the file up through `save_with_backup`. -# -# Adding a caller here widens where the file can change from. Do it only with -# the same scrutiny the gestures themselves got: the edit must be explicit, the -# failure must degrade to an in-memory change with honest wording, and there -# must be an "only this key changed" test beside the gesture it belongs to. -NARROW_WRITERS = { - "persist_keybinding_edit": {EDIT_WORKER}, - "persist_preset_slot": {EDIT_WORKER}, - "persist_quick_color": {EDIT_WORKER}, -} - -# The editors' path-taking twins: the same gesture against an explicit file, -# which the suites drive and production has no use for. They are `#[cfg(test)] -# pub(crate)`, so a production build has no such function to call at all; this -# scan is the second line, and says *why* rather than reporting an unresolved -# name. No production file may name one — it would be a config write at a path -# nobody reviewed, and the word boundary that pins the editors above ends before -# the `_at`, so `NARROW_WRITERS` would never match the call. -PATH_TAKING_WRITERS = {f"{name}_at" for name in NARROW_WRITERS} - -# The only value items the config module hands out of the two writer files. The -# three editors, which are the reviewed capability; everything else those files -# export is a type, and a type is not a way to call anything the pins here have -# not already read. -REEXPORTABLE_WRITER_VALUES = set(NARROW_WRITERS) - -# The gate that keeps that first line in place. Dropping the `#[cfg(test)]` would -# hand every module in the crate a config write at a caller-chosen path again. -CFG_TEST_TWIN = { - name: re.compile(rf"\#\[cfg\(test\)\]\s*pub\(crate\) fn {re.escape(name)}\b") - for name in sorted(PATH_TAKING_WRITERS) -} - -# Where the editors may be named inside the config module: declared in `io.rs`, -# re-exported from `mod.rs`. The module is exempt from caller collection because -# it owns the writers, so a wrapper anywhere else in it would be an unreviewed -# writer with a name of its own. Both files earn the exemption by being pinned -# on their own terms instead — `audit_io_write_surface` for the declarations and -# `audit_config_module_surface` for the re-exports. -EDITOR_HOME = {IO_SOURCE, CONFIG_MODULE} - -# Subtrees whose former write authority this check replaces. If the walk stops -# reaching them, it proves nothing, so their presence is asserted rather than -# assumed. -EXPECTED_SCANNED = { - DOCUMENT_SOURCE, - IO_SOURCE, - CONFIGURATOR_ADAPTER, - Path("src/backend/wayland/state.rs"), # the overlay that owned the writer - Path("src/daemon/tray/runtime.rs"), # the tray that owned the resume toggle - Path("src/backend/wayland/backend/state_init/config.rs"), # startup load - Path("configurator/src/app/update/config.rs"), # Save's message handling - EDIT_WORKER, # the overlay's off-dispatch config-edit worker - SHORTCUT_CALL_SITE, # the overlay's shortcut editor - PRESET_CALL_SITE, # the overlay's preset slots - QUICK_COLOR_CALL_SITE, # the overlay's quick-color palette -} - -# String bodies, char literals, and comments carry no capability, and a `{` in -# one would desynchronise the `#[cfg(test)]` block tracking below. Blanked in -# place so byte offsets and line numbers keep pointing at the real source. -MASKED_SPANS = re.compile( - r""" - //[^\n]* # line comment - | /\*.*?\*/ # block comment - | (?\#*)" # raw string, byte or not - .*? - "(?P=hashes) - | b?"(?:\\.|[^"\\])*" # string, byte or not - | '(?:\\.|[^'\\])' # char literal, never a lifetime - """, - re.VERBOSE | re.DOTALL, -) - -CFG_TEST = re.compile(r"\#\[cfg\(test\)\]") - -# `#[cfg(test)] mod foo;`, the declaration that makes a whole file test-only. -# Any further attributes and the visibility sit between the two. An inline -# `#[cfg(test)] mod tests { .. }` counts the same way: its own children are files -# in the directory beside it, and they are just as test-only. -CFG_TEST_MOD = re.compile( - r"\#\[cfg\(test\)\]\s*(?:\#\[[^\]]*\]\s*)*" - r"(?:pub(?:\s*\([^)]*\))?\s+)?mod\s+(?P\w+)\s*[;{]" -) - -# A function item, at any indentation and any visibility. Methods inside `impl` -# blocks match too: a `pub fn` on `Config` reaches as far as a free one. -FUNCTION_ITEM = re.compile( - r"(?m)^[ \t]*(?Ppub(?:\s*\([^)]*\))?\s+)?" - r"(?:const\s+|async\s+|unsafe\s+|extern\s+\"[^\"]*\"\s+)*" - r"fn\s+(?P\w+)" -) - -# A `const` or `static` item, at any indentation and any visibility, including -# an associated one inside an `impl`. The `: ` is what tells it from a `const -# fn`, whose name is followed by its parameter list. -CONST_ITEM = re.compile( - r"(?m)^[ \t]*(?Ppub(?:\s*\([^)]*\))?\s+)?" - r"(?Pconst|static)\s+(?:mut\s+)?(?P\w+)\s*:" -) - -USE_ITEM = re.compile(r"(?m)^[ \t]*(?:pub(?:\s*\([^)]*\))?\s+)?use\b[^;]*;", re.DOTALL) - -# A path prefix inside a `use` item. Masking these leaves the names the item -# actually brings into scope: `pub use document::{ConfigDocument, X};` names two -# things, and `document` is not one of them — it is the module they came from, -# and it collides with method names all over the writer files. -USE_PATH_SEGMENT = re.compile(r"\b\w+\s*::") - -# What is left in a `use` item after that masking and is not a name it imports. -USE_KEYWORDS = frozenset({"use", "pub", "crate", "self", "super", "as", "in"}) - -# An `impl` block header, up to the `{` that opens its body. What tells a trait -# impl from an inherent one is the `for`; the `for<'a>` of a higher-ranked bound -# is not it, hence the lookahead. -IMPL_BLOCK = re.compile(r"(?m)^[ \t]*(?:unsafe\s+)?impl\b(?P
[^{;]*)\{") - -IMPL_FOR = re.compile(r"\bfor\b(?!\s*<)") - -# A `trait` declaration: the surface it hands out, and the bodies of any default -# methods that travel with it. -TRAIT_BLOCK = re.compile( - r"(?m)^[ \t]*(?Ppub(?:\s*\([^)]*\))?\s+)?(?:unsafe\s+)?" - r"trait\s+(?P\w+)[^{;]*\{" -) - -# A method signature inside a trait declaration, body or no body. The bodiless -# ones are why this exists: `function_items` skips them, and they are exactly -# how a trait offers a write whose implementation sits elsewhere in the file. -TRAIT_METHOD = re.compile( - r"(?m)^[ \t]*(?:const\s+|async\s+|unsafe\s+|extern\s+\"[^\"]*\"\s+)*fn\s+(?P\w+)" -) - -# ` as `, for every name that can write once it is in scope. The -# rest of this check recognises a write by the name at the call site, so a -# rename is the one way to put a call beyond it — whether the rename hands the -# capability on (`pub use`) or only conceals the call in the file that makes it -# (a plain `use`). The rename is what fails; the alias is named in the message -# so the reader can find the call it was hiding. -RENAME_PATTERNS = { - name: re.compile(rf"\b{re.escape(name)}\s+as\s+(?P\w+)") - for name in sorted(IO_WRITE_SURFACE) -} - -IDENTIFIER = re.compile(r"\b\w+\b") - - -def blank(match: re.Match[str]) -> str: - return "".join("\n" if character == "\n" else " " for character in match.group(0)) - - -def mask_source(source: str) -> str: - return MASKED_SPANS.sub(blank, source) - - -def block_end(masked: str, opening: int) -> int | None: - """Offset just past the `}` closing the block that starts at `opening`.""" - depth = 0 - for index in range(opening, len(masked)): - if masked[index] == "{": - depth += 1 - elif masked[index] == "}": - depth -= 1 - if depth == 0: - return index + 1 - return None - - -def cfg_test_spans(masked: str) -> tuple[list[tuple[int, int]], list[str]]: - """Offset ranges covered by `#[cfg(test)]` items, and any tracking failure. - - Inline test modules live in production files, so exempting whole files by - name is not enough; the attributed item is what has to be exempt. - """ - spans: list[tuple[int, int]] = [] - problems: list[str] = [] - for attribute in CFG_TEST.finditer(masked): - index = attribute.end() - while index < len(masked) and masked[index] not in "{;": - index += 1 - if index >= len(masked): - problems.append("a `#[cfg(test)]` item has neither a body nor a `;`") - continue - if masked[index] == ";": - # `#[cfg(test)] mod tests;` and `#[cfg(test)] use ...;` bring in no - # inline code; the module file is exempt by its own path. - continue - end = block_end(masked, index) - if end is None: - problems.append("a `#[cfg(test)]` block never closes") - continue - spans.append((attribute.start(), end)) - return spans, problems - - -def crate_source_root(relative: Path) -> Path | None: - """The crate root file whose `mod` items begin `relative`'s module chain.""" - if relative.parts[:1] == ("src",): - base = Path("src") - elif relative.parts[:2] == ("configurator", "src"): - base = Path("configurator/src") - else: - return None - for name in ("lib.rs", "main.rs"): - candidate = base / name - if (ROOT / candidate).is_file(): - return candidate - return None - - -def module_file(module: Path) -> Path | None: - """The file that holds `module`'s own items, `foo.rs` or `foo/mod.rs`.""" - for candidate in (module.with_suffix(".rs"), module / "mod.rs"): - if (ROOT / candidate).is_file(): - return candidate - return None - - -_CFG_TEST_MODULES: dict[Path, frozenset[str]] = {} - - -def cfg_test_modules(declaring: Path) -> frozenset[str]: - """Child modules `declaring` compiles only under `cfg(test)`.""" - cached = _CFG_TEST_MODULES.get(declaring) - if cached is not None: - return cached - masked = mask_source((ROOT / declaring).read_text()) - names = frozenset(match.group("name") for match in CFG_TEST_MOD.finditer(masked)) - _CFG_TEST_MODULES[declaring] = names - return names - - -def is_test_source(relative: Path) -> bool: - """Whether Rust compiles this file only under `cfg(test)`. - - The shape of the path is not evidence. A directory called `tests` inside - `src/` is ordinary production code unless the `mod` item that brings it in - says otherwise, and a file dropped into one would otherwise be exempt from - every check here — a config write nobody reviewed, in a file that only looks - like a test. What the compiler reads is the declaration, so that is what - this reads: any module on the chain gated with `#[cfg(test)]` makes the file - below it test-only, which is exactly how `mod tests;` earns its exemption. - """ - root = crate_source_root(relative) - if root is None: - return False - base = root.parent - segments = list(relative.relative_to(base).parts) - if segments[-1] == "mod.rs": - segments.pop() - else: - segments[-1] = segments[-1].removesuffix(".rs") - - declaring: Path | None = root - module = base - for segment in segments: - if declaring is None: - return False - if segment in cfg_test_modules(declaring): - return True - module = module / segment - declaring = module_file(module) - return False - - -def rust_sources() -> list[Path]: - roots = (ROOT / "src", ROOT / "configurator" / "src") - return sorted(path for root in roots for path in root.rglob("*.rs")) - - -def line_of(source: str, offset: int) -> int: - return source.count("\n", 0, offset) + 1 - - -def audit_sites() -> tuple[list[str], set[Path]]: - failures: list[str] = [] - scanned: set[Path] = set() - editor_names = (*NARROW_WRITERS, *sorted(PATH_TAKING_WRITERS)) - editor_callers: dict[str, set[Path]] = {name: set() for name in editor_names} - editor_patterns = { - name: re.compile(rf"\b{re.escape(name)}\b") for name in editor_names - } - for absolute in rust_sources(): - relative = absolute.relative_to(ROOT) - scanned.add(relative) - source = absolute.read_text() - masked = mask_source(source) - spans, problems = cfg_test_spans(masked) - is_test = is_test_source(relative) - if not is_test: - failures.extend(f"{relative}: {problem}" for problem in problems) - failures.extend(renamed_imports(relative, source, masked, spans)) - - # Narrow-editor call sites are pinned outside the config module, which - # declares and re-exports them. A rogue write inside `src/config/` is - # still caught: only `io.rs` and `document.rs` are allowlisted for the - # primitives, so every other file there is scanned for them below, and - # the editors themselves are confined to `EDITOR_HOME` here. - in_config_module = relative.parts[:2] == ("src", "config") - if not is_test and not in_config_module: - for name, pattern in editor_patterns.items(): - for hit in pattern.finditer(masked): - if any(start <= hit.start() < end for start, end in spans): - continue - editor_callers[name].add(relative) - elif not is_test and relative not in EDITOR_HOME: - for name, pattern in editor_patterns.items(): - for hit in pattern.finditer(masked): - if any(start <= hit.start() < end for start, end in spans): - continue - failures.append( - f"{relative}:{line_of(source, hit.start())}: names the " - f"narrow config writer `{name}` inside the config module; " - "the editors are declared in io.rs and leave through " - "mod.rs, so a wrapper here would carry the capability out " - "under a name the call-site pins never see" - ) - - if relative in WRITE_ALLOWLIST or is_test: - continue - lines = source.splitlines() - for pattern in WRITE_PATTERNS: - for hit in pattern.finditer(masked): - if any(start <= hit.start() < end for start, end in spans): - continue - number = line_of(source, hit.start()) - text = lines[number - 1].strip() if number <= len(lines) else "" - failures.append( - f"{relative}:{number}: config write capability " - f"`{hit.group(0)}` outside the reviewed writers: {text}" - ) - - for name, expected in NARROW_WRITERS.items(): - found = editor_callers[name] - for unexpected in sorted(found - expected): - failures.append( - f"{unexpected}: unreviewed caller of the narrow config writer " - f"`{name}`; record it in NARROW_WRITERS if this gesture should " - "be able to change config.toml" - ) - for missing in sorted(expected - found): - failures.append( - f"{missing}: expected to call `{name}` but does not; the pinned " - "call site moved, so this check no longer describes the code" - ) - for name in sorted(PATH_TAKING_WRITERS): - for caller in sorted(editor_callers[name]): - failures.append( - f"{caller}: production code names `{name}`; the path-taking " - "twins take the file to write from their caller and exist for " - "the suites, so a gesture that needs one is a new writer to " - "review, not an implementation detail" - ) - return failures, scanned - - -def renamed_imports( - relative: Path, source: str, masked: str, spans: list[tuple[int, int]] -) -> list[str]: - """Every `use ... as ...` that renames a name capable of writing the file. - - Both shapes matter and neither is legitimate here. A `pub use` rename hands - the capability to the whole crate under a name nothing pins, and a plain - `use` rename conceals the call in the file that makes it — the call-site - pinning above matches `persist_preset_slot(..)`, never `save(..)`. - """ - failures: list[str] = [] - for item in USE_ITEM.finditer(masked): - if any(start <= item.start() < end for start, end in spans): - continue - exported = item.group(0).lstrip().startswith("pub") - for name, pattern in RENAME_PATTERNS.items(): - rename = pattern.search(item.group(0)) - if rename is None: - continue - failures.append( - f"{relative}:{line_of(source, item.start())}: " - f"{'re-exports' if exported else 'imports'} the config writer " - f"`{name}` as `{rename.group('alias')}`; the writers are pinned " - "by name, so they travel under their own or not at all" - ) - return failures - - -class FunctionItem: - """One `fn` in a file, with the source span of its body.""" - - def __init__(self, name: str, visibility: str | None, start: int, end: int) -> None: - self.name = name - self.visibility = visibility - self.start = start - self.end = end - - @property - def is_exported(self) -> bool: - return self.visibility is not None - - -class ConstItem: - """One `const` or `static` in a file, with the span of its declaration.""" - - def __init__(self, keyword: str, name: str, start: int, end: int) -> None: - self.keyword = keyword - self.name = name - self.start = start - self.end = end - - -def item_end(masked: str, opening: int) -> int | None: - """Offset just past the `;` that ends the item starting at `opening`.""" - depth = 0 - for index in range(opening, len(masked)): - character = masked[index] - if character in "([{": - depth += 1 - elif character in ")]}": - depth -= 1 - elif character == ";" and depth == 0: - return index + 1 - return None - - -def const_items(masked: str, exclude: list[tuple[int, int]]) -> list[ConstItem]: - """Every `const` or `static` defined outside `exclude`, in source order.""" - items: list[ConstItem] = [] - for match in CONST_ITEM.finditer(masked): - if any(start <= match.start() < end for start, end in exclude): - continue - end = item_end(masked, match.end()) - if end is None: - continue - items.append( - ConstItem(match.group("keyword"), match.group("name"), match.start(), end) - ) - return items - - -def audit_const_initializers( - relative: Path, - source: str, - masked: str, - capable: set[str], - exclude: list[tuple[int, int]], -) -> list[str]: - """No write leaves an implementing file as a value. - - The surface pins read `fn` items and walk the calls between them, so what - they see is functions. A `pub const PERSIST_ANY_CONFIG: fn(..) = - ConfigDocument::save_with_backup;` declares no function at all: it is the - same capability with a type instead of a body, exported, callable by anyone - who can name it, and invisible to every walk here. The initializer is where - the writer's name has to appear for that to work, so that is what this - reads. - """ - failures: list[str] = [] - for item in const_items(masked, exclude): - body = masked[item.start : item.end] - for name in sorted({token for token in IDENTIFIER.findall(body)} & capable): - failures.append( - f"{relative}:{line_of(source, item.start)}: `{item.keyword} " - f"{item.name}` names `{name}`, which can write config.toml; a " - "writer stored as a value declares no function for the surface " - "pins to read, so the writers here stay functions" - ) - return failures - - -def function_items(masked: str, exclude: list[tuple[int, int]]) -> list[FunctionItem]: - """Every function defined outside `exclude`, in source order.""" - items: list[FunctionItem] = [] - for match in FUNCTION_ITEM.finditer(masked): - if any(start <= match.start() < end for start, end in exclude): - continue - index = match.end() - while index < len(masked) and masked[index] not in "{;": - index += 1 - if index >= len(masked) or masked[index] == ";": - # A signature without a body: a trait item or an `extern` block. - continue - end = block_end(masked, index) - if end is None: - continue - visibility = match.group("visibility") - items.append( - FunctionItem( - match.group("name"), - visibility.strip() if visibility else None, - match.start(), - end, - ) - ) - return items - - -def write_capable_functions( - masked: str, items: list[FunctionItem], seeds: set[str] | tuple[str, ...] -) -> set[str]: - """Names that reach one of `seeds`, directly or through this file. - - Best-effort by design: it follows plain name references inside each body, - which is what a wrapper in this file looks like. It cannot see through a - function pointer stored in a struct, and it does not need to — the point is - that a *new* write path has to be named here to be reviewed. - """ - bodies = {item.name: masked[item.start : item.end] for item in items} - references = { - name: {token for token in IDENTIFIER.findall(body)} - for name, body in bodies.items() - } - capable = {name for name, tokens in references.items() if tokens & set(seeds)} - # Fixpoint: a caller of a write-capable function is write-capable too. - changed = True - while changed: - changed = False - for name, tokens in references.items(): - if name in capable: - continue - if tokens & capable: - capable.add(name) - changed = True - return capable - - -def trait_spans(masked: str, exclude: list[tuple[int, int]]) -> list[tuple[int, int]]: - """Ranges where a `fn` is callable without a `pub` of its own. - - A method in `impl Trait for Type` is reachable wherever the trait is, and a - trait's own default method travels with the trait. Neither carries a - visibility — the trait's is what counts — so `is_exported` reads `False` for - both, and the surface pins would let a write walk out of the file behind one. - """ - spans: list[tuple[int, int]] = [] - blocks = [ - match - for match in IMPL_BLOCK.finditer(masked) - if IMPL_FOR.search(match.group("header")) is not None - ] - # An inherent `impl Type` is deliberately not here: its methods carry their - # own visibility, and the surface pins already read it. - blocks.extend(TRAIT_BLOCK.finditer(masked)) - for match in blocks: - if any(start <= match.start() < end for start, end in exclude): - continue - end = block_end(masked, match.end() - 1) - if end is not None: - spans.append((match.start(), end)) - return spans - - -def trait_declarations( - masked: str, exclude: list[tuple[int, int]] -) -> list[tuple[str, int, set[str]]]: - """Each `trait` block outside `exclude`: name, offset, and declared methods.""" - declarations: list[tuple[str, int, set[str]]] = [] - for match in TRAIT_BLOCK.finditer(masked): - if any(start <= match.start() < end for start, end in exclude): - continue - end = block_end(masked, match.end() - 1) - if end is None: - continue - body = masked[match.end() : end] - methods = {item.group("name") for item in TRAIT_METHOD.finditer(body)} - declarations.append((match.group("name"), match.start(), methods)) - return declarations - - -def audit_trait_writers( - relative: Path, - source: str, - masked: str, - items: list[FunctionItem], - capable: set[str], - exclude: list[tuple[int, int]], -) -> list[str]: - """No write leaves an implementing file through a trait. - - Everything else here reads the `pub` an item carries, and a trait method - carries none. An `impl SomeTrait for ConfigDocument` whose method calls the - merge step is callable from anywhere the trait is in scope, and the trait - itself is one `pub use` away — so the file would have a second durable - writer, exported, under a name no pin here ever sees. The surfaces above - enumerate inherent and free functions; a write reachable through a trait is - out of bounds whatever it is called and whatever visibility it was given. - """ - failures: list[str] = [] - spans = trait_spans(masked, exclude) - for item in items: - if item.name not in capable: - continue - if not any(start <= item.start < end for start, end in spans): - continue - failures.append( - f"{relative}:{line_of(source, item.start)}: `fn {item.name}` can write " - "config.toml from inside a trait; a trait's methods are callable " - "wherever the trait is and carry no visibility of their own, so the " - "writers here stay inherent or free functions" - ) - failures.extend(audit_trait_declarations(relative, source, masked, capable, exclude)) - return failures - - -def audit_trait_declarations( - relative: Path, - source: str, - masked: str, - capable: set[str], - exclude: list[tuple[int, int]], -) -> list[str]: - """A trait declared here may not name a method that can write. - - Separate from the walk above because a trait's method signatures need no - bodies: `fn persist_any_config(&self);` is the whole export, and the impl - that fills it in is an ordinary block elsewhere in the file. What is pinned - is the offer — a trait whose surface can write is a write capability handed - to every caller that can name the trait. - """ - failures: list[str] = [] - for name, offset, methods in trait_declarations(masked, exclude): - for method in sorted(methods & capable): - failures.append( - f"{relative}:{line_of(source, offset)}: `trait {name}` declares " - f"`{method}`, which can write config.toml; a trait carries the " - "capability to every caller that can name it, so the writers " - "here are not offered through one" - ) - return failures - - -def audit_io_write_surface(io_source: str, masked_io: str, test_spans) -> list[str]: - """`io.rs` is exempt from the primitive scan, so its own surface is pinned. - - Every function here that can reach a write primitive and is visible outside - the file has to be one of the reviewed editors. A new `pub fn` wrapping the - atomic write would otherwise be re-exportable and callable from anywhere, - with the rest of this check none the wiser. - """ - failures: list[str] = [] - items = function_items(masked_io, test_spans) - if not any(item.name == "persist_keybinding_edit" for item in items): - return [ - f"{IO_SOURCE}: the function scan found no narrow editor; its shape " - "assumption about the file no longer holds" - ] - - capable = write_capable_functions(masked_io, items, WRITE_PRIMITIVES) - failures.extend( - audit_trait_writers(IO_SOURCE, io_source, masked_io, items, capable, test_spans) - ) - failures.extend( - audit_const_initializers( - IO_SOURCE, - io_source, - masked_io, - capable | set(WRITE_PRIMITIVES), - test_spans, - ) - ) - for item in items: - if not item.is_exported or item.name not in capable: - continue - if item.name in IO_WRITE_SURFACE: - continue - failures.append( - f"{IO_SOURCE}:{line_of(io_source, item.start)}: `{item.visibility} fn " - f"{item.name}` can write config.toml but is not one of the reviewed " - "editors; record it in IO_WRITE_SURFACE if this is a new user gesture" - ) - return failures - - -def audit_document_write_surface( - document_source: str, masked_document: str, test_spans -) -> list[str]: - """`document.rs` is exempt from the primitive scan, so its surface is pinned. - - The same walk `io.rs` gets, for the same reason: this file owns the merge, - the backup, and the rename, so matching on a name — anything containing - "save", say — pins the spelling rather than the capability. A `pub fn - persist_any_config(..) { self.merge_and_write(..) }` writes `config.toml` - just as `save_with_backup` does, and would leave the application with two - durable writers, one of them unreviewed and callable from anywhere the - document type is. - """ - failures: list[str] = [] - items = function_items(masked_document, test_spans) - if not any(item.name == "save_with_backup" for item in items): - return [ - f"{DOCUMENT_SOURCE}: the function scan found no document save; its " - "shape assumption about the file no longer holds" - ] - if not any(item.name in DOCUMENT_WRITE_SEEDS for item in items): - return [ - f"{DOCUMENT_SOURCE}: the function scan found none of the write steps " - f"({', '.join(sorted(DOCUMENT_WRITE_SEEDS))}); its shape assumption " - "about the file no longer holds" - ] - - capable = write_capable_functions(masked_document, items, DOCUMENT_WRITE_SEEDS) - failures.extend( - audit_trait_writers( - DOCUMENT_SOURCE, document_source, masked_document, items, capable, test_spans - ) - ) - failures.extend( - audit_const_initializers( - DOCUMENT_SOURCE, - document_source, - masked_document, - capable | set(DOCUMENT_WRITE_SEEDS), - test_spans, - ) - ) - for item in items: - if not item.is_exported or item.name not in capable: - continue - if item.name in DOCUMENT_WRITE_SURFACE: - continue - failures.append( - f"{DOCUMENT_SOURCE}:{line_of(document_source, item.start)}: " - f"`{item.visibility} fn {item.name}` can write config.toml but is not " - "the reviewed document save; the application has exactly one durable " - "writer, so record it in DOCUMENT_WRITE_SURFACE only if that changed" - ) - return failures - - -def enclosing_function(items: list[FunctionItem], offset: int) -> FunctionItem | None: - """The innermost function whose body contains `offset`.""" - innermost: FunctionItem | None = None - for item in items: - if item.start <= offset < item.end and ( - innermost is None or item.start > innermost.start - ): - innermost = item - return innermost - - -def writer_value_items() -> dict[str, tuple[Path, str]]: - """Every `fn`, `const`, and `static` declared in the two writer files. - - Names, not capabilities. The surface pins above answer what can *write*, by - walking calls between functions, and they are the right tool for a function. - They are the wrong tool for everything else a name can be: a `const` holding - a function pointer declares no function, so no walk reaches it, and it - leaves the module under a name nothing here has ever heard of. What can be - said about such a name honestly is where it was declared — and a value - declared in a file that owns the durable write is not something the config - module hands out without a reason on the record. - - Types are deliberately absent. `ConfigDocument` and `ConfigEditOutcome` are - most of what the module re-exports, they carry no capability of their own, - and the methods on them are pinned by the surface audits. - """ - items: dict[str, tuple[Path, str]] = {} - for relative in (DOCUMENT_SOURCE, IO_SOURCE): - masked = mask_source((ROOT / relative).read_text()) - spans, _ = cfg_test_spans(masked) - for function in function_items(masked, spans): - items.setdefault(function.name, (relative, "fn")) - for constant in const_items(masked, spans): - items[constant.name] = (relative, constant.keyword) - return items - - -def use_leaf_names(item: str) -> set[str]: - """The names a `use` item brings into scope, without their paths.""" - leaves = USE_PATH_SEGMENT.sub(lambda match: " " * len(match.group(0)), item) - return { - token for token in IDENTIFIER.findall(leaves) if token not in USE_KEYWORDS - } - - -def audit_config_module_surface() -> list[str]: - """`src/config/mod.rs` may name the writers only where it re-exports them. - - Everything else here recognises a write by the name at the call site, and - this file is exempt from that collection because it is where the editors - leave the module. The write-capability walk reads `io.rs` and nothing else, - so without this a `pub fn concealed_edit(..) { io::persist_keybinding_edit(..) }` - sitting here would be invisible to every check and callable from anywhere in - the crate. - - The tightest honest rule is the one the file already lives by: it is a - re-export list. A writer's name may appear inside a `use` or `pub use` item - and nowhere else, and no function declared here may reach a writer at all — - directly, through `crate::config::io::`, or through another wrapper in this - file. Either failure names the function and the line. - """ - source = (ROOT / CONFIG_MODULE).read_text() - masked = mask_source(source) - spans, problems = cfg_test_spans(masked) - failures = [f"{CONFIG_MODULE}: {problem}" for problem in problems] - use_spans = [(item.start(), item.end()) for item in USE_ITEM.finditer(masked)] - items = function_items(masked, spans) - - for name in sorted(IO_WRITE_SURFACE): - for hit in re.finditer(rf"\b{re.escape(name)}\b", masked): - if any(start <= hit.start() < end for start, end in spans): - continue - if any(start <= hit.start() < end for start, end in use_spans): - continue - enclosing = enclosing_function(items, hit.start()) - where = ( - f"inside `fn {enclosing.name}`" - if enclosing is not None - else "outside any `use` item" - ) - failures.append( - f"{CONFIG_MODULE}:{line_of(source, hit.start())}: names the config " - f"writer `{name}` {where}; this file re-exports the editors and " - "does nothing else, so anything here that can call one carries the " - "capability out under a name the call-site pins never see" - ) - - capable = write_capable_functions(masked, items, IO_WRITE_SURFACE) - for item in items: - if item.name not in capable: - continue - failures.append( - f"{CONFIG_MODULE}:{line_of(source, item.start)}: `fn {item.name}` can " - "reach a config writer; the config module's own file declares no " - "functions, so this is an unreviewed writer with a name of its own" - ) - # The loop above reads every function, whatever visibility it carries, so a - # trait's default method is already in it. What is left is the offer with no - # body: a `trait` here declaring a method something else in the file - # implements. - failures.extend( - audit_trait_declarations(CONFIG_MODULE, source, masked, capable, spans) - ) - return failures - - -def audit_module_reexports() -> list[str]: - """The config module may re-export the editors, never the primitives. - - A `pub use` of the atomic write, the backup copy, or the document save would - hand the capability to the whole crate under a path this check's call-site - pinning says nothing about. - - Naming the primitives is not enough on its own, because a re-export need not - name a writer to carry one. `pub use document::PERSIST_ANY_CONFIG;` names a - `const` whose initializer is the document save, and every pin here reads - past it: it is not a primitive, not a function, and not a rename. So the - second half of this asks a question the shape of the export cannot dodge — - which file declared the name, and as what. A `fn`, `const`, or `static` from - one of the two files that own the durable write leaves the module only if - the review that let the editors out covers it too; a type is not part of - this, and types are most of what the module re-exports. - - Every production file in the module is scanned, not just `mod.rs`: a `pub - use` in a submodule travels exactly as far, since the submodule is itself - re-exported. Renaming is caught for all of `src/` by `renamed_imports`. - """ - failures: list[str] = [] - values = writer_value_items() - for relative in config_module_sources(): - source = (ROOT / relative).read_text() - masked = mask_source(source) - spans, _ = cfg_test_spans(masked) - for item in USE_ITEM.finditer(masked): - if any(start <= item.start() < end for start, end in spans): - continue - if not item.group(0).lstrip().startswith("pub"): - continue - for primitive in WRITE_PRIMITIVES: - if re.search(rf"\b{re.escape(primitive)}\b", item.group(0)): - failures.append( - f"{relative}:{line_of(source, item.start())}: re-exports the " - f"write primitive `{primitive}`; the primitives stay inside the " - "config module and only the narrow editors leave it" - ) - for name in sorted(use_leaf_names(item.group(0))): - if name in REEXPORTABLE_WRITER_VALUES or name not in values: - continue - declared, keyword = values[name] - failures.append( - f"{relative}:{line_of(source, item.start())}: re-exports " - f"`{name}`, a `{keyword}` declared in {declared}; the write " - "lives in that file, and the surface pins there read " - "functions — so a value leaving it carries whatever it holds " - "past them. Only the reviewed editors leave the module" - ) - return failures - - -def config_module_sources() -> list[Path]: - """Production files in `src/config/`, `mod.rs` first for readable output.""" - root = ROOT / "src" / "config" - paths = [ - path.relative_to(ROOT) - for path in root.rglob("*.rs") - if not is_test_source(path.relative_to(ROOT)) - ] - return sorted(paths, key=lambda path: (path != CONFIG_MODULE, path)) - - -def audit_write_surface() -> list[str]: - """The implementing files may not widen the capability they own.""" - failures: list[str] = [] - - document = (ROOT / DOCUMENT_SOURCE).read_text() - masked_document = mask_source(document) - document_test_spans, document_problems = cfg_test_spans(masked_document) - failures.extend(f"{DOCUMENT_SOURCE}: {problem}" for problem in document_problems) - failures.extend( - audit_document_write_surface(document, masked_document, document_test_spans) - ) - if "pub fn save_with_backup" not in document: - failures.append( - f"{DOCUMENT_SOURCE}: `save_with_backup` is gone or renamed; " - "this check no longer describes the code" - ) - - io_source = (ROOT / IO_SOURCE).read_text() - for primitive in ("create_config_backup", "write_config_text_atomic", "prepare_config_parent"): - if f"pub(super) fn {primitive}" not in io_source: - failures.append( - f"{IO_SOURCE}: `{primitive}` is no longer `pub(super)`; the write " - "primitives must stay inside the config module" - ) - - # Each narrow editor must still be declared here, and must still build its - # `updated` config on `document.config()` — the base that makes the merge - # gate write one key. Basing it on `authored_config()` would hand the gate - # every value the loader clamped or resolved as if the user had typed it. - for name in NARROW_WRITERS: - if f"pub fn {name}" not in io_source: - failures.append( - f"{IO_SOURCE}: narrow config writer `{name}` is gone or no longer " - "declared here; this check no longer describes the code" - ) - for name, pattern in CFG_TEST_TWIN.items(): - if pattern.search(io_source) is None: - failures.append( - f"{IO_SOURCE}: `{name}` is no longer a `#[cfg(test)] pub(crate) fn`; " - "the path-taking twins exist for the suites, and an ungated one is a " - "config write at a caller-chosen path available to the whole crate" - ) - # Production code only: the inline suite reads `authored_config()` on - # purpose, to prove the loader had something to repair in its fixture. - masked_io = mask_source(io_source) - test_spans, problems = cfg_test_spans(masked_io) - failures.extend(f"{IO_SOURCE}: {problem}" for problem in problems) - for hit in re.finditer(r"\bauthored_config\b", masked_io): - if any(start <= hit.start() < end for start, end in test_spans): - continue - failures.append( - f"{IO_SOURCE}:{line_of(io_source, hit.start())}: a narrow writer reads " - "`authored_config()`; the edit base must be `document.config()` so the " - "merge gate writes only the edited key" - ) - failures.extend(audit_io_write_surface(io_source, masked_io, test_spans)) - failures.extend(audit_config_module_surface()) - failures.extend(audit_module_reexports()) - return failures - - -def main() -> int: - failures, scanned = audit_sites() - missing = sorted(str(path) for path in EXPECTED_SCANNED - scanned) - if missing: - failures.append("the walk missed expected sources, so it proves nothing: " + ", ".join(missing)) - failures.extend(audit_write_surface()) - - if failures: - print("config-writer audit failed:", file=sys.stderr) - for failure in failures: - print(f" {failure}", file=sys.stderr) - return 1 - print(f"config-writer audit passed ({len(scanned)} sources)") - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/tools/check-nixpkgs-recipe.py b/tools/check-nixpkgs-recipe.py deleted file mode 100755 index 4ff626cf1..000000000 --- a/tools/check-nixpkgs-recipe.py +++ /dev/null @@ -1,282 +0,0 @@ -#!/usr/bin/env python3 -"""Fail when a default Cargo feature needs a system library the Nix builds omit. - -The nixpkgs recipe (`packaging/nixpkgs/package.nix`, mirrored from -`pkgs/by-name/wa/wayscriber/package.nix`) is bumped automatically by the -nixpkgs-update bot, which only rewrites the version and hashes. When a release -enables a dependency that links a C library, the bot's next pull request fails -to build unless a human has already added that library. This check catches the -mismatch here instead. - -Every direct normal dependency in Cargo.toml must appear in SYSTEM_LIBRARIES, -mapped to the nixpkgs attributes it needs (an empty tuple for pure Rust crates). -A new normal dependency therefore fails this check until its system requirements -are stated. -""" - -from __future__ import annotations - -import json -import re -import subprocess -import sys -from pathlib import Path - - -REPO_ROOT = Path(__file__).resolve().parent.parent -CARGO_TOML = Path("Cargo.toml") -RECIPE = Path("packaging/nixpkgs/package.nix") -FLAKE = Path("flake.nix") -FLAKE_PACKAGE_MARKER = "wayscriber = rustPlatform.buildRustPackage" -LINUX_TARGET = "x86_64-unknown-linux-gnu" - -# Native tools/hooks required by the default GTK-enabled package. -NATIVE_BUILD_INPUTS = {"pkg-config", "wrapGAppsHook4"} - -# Direct dependency -> nixpkgs attributes its build or link step requires. -# Attributes that nixpkgs propagates through another entry are left out: pango -# propagates cairo, glib and harfbuzz, and gtk4 propagates its own stack. -SYSTEM_LIBRARIES: dict[str, tuple[str, ...]] = { - "anyhow": (), - "cairo-rs": ("cairo",), - "flate2": (), - "getrandom": (), - "glib": (), - "gtk4": ("gtk4",), - "gtk4-layer-shell": ("gtk4-layer-shell",), - # Behind the opt-in `input-monitor` feature (system-wide input capture for - # the input HUD), so these are mapped but not required by the default - # package. Adding `input-monitor` to the default features would make the - # attributes below mandatory in package.nix and flake.nix. - "input": ("libinput",), - "ksni": (), - "libc": (), - "log": (), - "pango": ("pango",), - "pangocairo": ("pango", "cairo"), - "png": (), - "schemars": (), - "serde": (), - "serde_ignored": (), - "serde_json": (), - "smithay-client-toolkit": ("libxkbcommon",), - # Pure Rust: temporary files for the OCR engine handoff. - "tempfile": (), - "tokio": (), - "toml": (), - "toml_edit": (), - "udev": ("udev",), - "unicode-segmentation": (), - "wayland-client": ("wayland",), - "wayland-protocols": (), - "wayland-protocols-wlr": (), - "xkbcommon": ("libxkbcommon",), - "zbus": (), - "zune-jpeg": (), -} - - -class RecipeError(RuntimeError): - """The recipe or flake could not be inspected.""" - - -def read_text(path: Path) -> str: - return (REPO_ROOT / path).read_text(encoding="utf-8") - - -def load_metadata() -> tuple[dict, dict, dict[str, str]]: - """Load Cargo's parsed manifest and Linux default-feature dependency graph.""" - try: - result = subprocess.run( - [ - "cargo", - "metadata", - "--locked", - "--format-version", - "1", - "--filter-platform", - LINUX_TARGET, - ], - cwd=REPO_ROOT, - check=False, - capture_output=True, - text=True, - ) - except OSError as error: - raise RecipeError(f"could not run cargo metadata: {error}") from error - - if result.returncode != 0: - detail = result.stderr.strip() or f"exit status {result.returncode}" - raise RecipeError(f"cargo metadata failed: {detail}") - - try: - metadata = json.loads(result.stdout) - except json.JSONDecodeError as error: - raise RecipeError(f"could not parse cargo metadata: {error}") from error - - manifest_path = str((REPO_ROOT / CARGO_TOML).resolve()) - manifest = next( - ( - package - for package in metadata.get("packages", []) - if package.get("manifest_path") == manifest_path - ), - None, - ) - if manifest is None: - raise RecipeError(f"cargo metadata did not contain {CARGO_TOML}") - - resolve = metadata.get("resolve") or {} - node = next( - (entry for entry in resolve.get("nodes", []) if entry.get("id") == manifest.get("id")), - None, - ) - if node is None: - raise RecipeError(f"cargo metadata did not resolve {CARGO_TOML}") - - package_names = { - package["id"]: package["name"] - for package in metadata.get("packages", []) - if "id" in package and "name" in package - } - return manifest, node, package_names - - -def direct_normal_dependencies(manifest: dict) -> set[str]: - """All declared direct normal dependencies, including target-specific ones.""" - return { - dependency["name"] - for dependency in manifest.get("dependencies", []) - if dependency.get("kind") is None and "name" in dependency - } - - -def crates_enabled_by_default(node: dict, package_names: dict[str, str]) -> set[str]: - """Direct normal dependencies resolved for a default-feature Linux build.""" - crates: set[str] = set() - for dependency in node.get("deps", []): - if not any(kind.get("kind") is None for kind in dependency.get("dep_kinds", [])): - continue - package_id = dependency.get("pkg") - if package_id not in package_names: - raise RecipeError(f"cargo metadata did not describe dependency {package_id!r}") - crates.add(package_names[package_id]) - return crates - - -def nix_list_entries(text: str, attribute: str, *, start: int = 0) -> list[str]: - """Return the identifiers inside the first `attribute = [ ... ];` after start.""" - # The lookbehind keeps `buildInputs` from matching inside `nativeBuildInputs` - # should a future edit change its capitalisation. - pattern = re.compile( - rf"(? tuple[set[str], set[str], set[str]]: - text = read_text(RECIPE) - native_build_inputs = set(nix_list_entries(text, "nativeBuildInputs")) - build_inputs = set(nix_list_entries(text, "buildInputs")) - - arguments_match = re.match(r"\s*\{(.*?)\}\s*:", text, re.DOTALL) - if arguments_match is None: - raise RecipeError(f"{RECIPE}: could not read the function argument set") - arguments = set(re.findall(r"[A-Za-z_][A-Za-z0-9_'-]*", arguments_match.group(1))) - - if not native_build_inputs: - raise RecipeError(f"{RECIPE}: no nativeBuildInputs list found") - if not build_inputs: - raise RecipeError(f"{RECIPE}: no buildInputs list found") - return native_build_inputs, build_inputs, arguments - - -def flake_inputs() -> tuple[set[str], set[str]]: - text = read_text(FLAKE) - marker = text.find(FLAKE_PACKAGE_MARKER) - if marker == -1: - raise RecipeError( - f"{FLAKE}: could not find `{FLAKE_PACKAGE_MARKER}`; " - "update FLAKE_PACKAGE_MARKER after restructuring the flake" - ) - - native_build_inputs = set(nix_list_entries(text, "nativeBuildInputs", start=marker)) - build_inputs = set(nix_list_entries(text, "buildInputs", start=marker)) - if not native_build_inputs: - raise RecipeError( - f"{FLAKE}: no nativeBuildInputs list found for the wayscriber package" - ) - if not build_inputs: - raise RecipeError(f"{FLAKE}: no buildInputs list found for the wayscriber package") - return native_build_inputs, build_inputs - - -def main() -> int: - try: - manifest, node, package_names = load_metadata() - crates = crates_enabled_by_default(node, package_names) - recipe_native, declared_recipe, recipe_arguments = recipe_inputs() - flake_native, declared_flake = flake_inputs() - except (OSError, RecipeError) as error: - print(f"nixpkgs recipe check failed: {error}", file=sys.stderr) - return 1 - - errors: list[str] = [] - - unmapped = sorted(direct_normal_dependencies(manifest) - set(SYSTEM_LIBRARIES)) - for crate in unmapped: - errors.append( - f"dependency `{crate}` has no entry in SYSTEM_LIBRARIES; add the nixpkgs " - "attributes it needs (or an empty tuple for a pure Rust crate)" - ) - - required: dict[str, set[str]] = {} - for crate in sorted(crates): - for attribute in SYSTEM_LIBRARIES.get(crate, ()): - required.setdefault(attribute, set()).add(crate) - - for attribute in sorted(NATIVE_BUILD_INPUTS): - if attribute not in recipe_native: - errors.append(f"{RECIPE}: nativeBuildInputs is missing `{attribute}`") - if attribute not in recipe_arguments: - errors.append(f"{RECIPE}: function arguments are missing `{attribute}`") - if attribute not in flake_native: - errors.append(f"{FLAKE}: nativeBuildInputs is missing `{attribute}`") - - for attribute, sources in sorted(required.items()): - reason = ", ".join(sorted(sources)) - if attribute not in declared_recipe: - errors.append(f"{RECIPE}: buildInputs is missing `{attribute}` (required by {reason})") - if attribute not in recipe_arguments: - errors.append(f"{RECIPE}: function arguments are missing `{attribute}`") - if attribute not in declared_flake: - errors.append(f"{FLAKE}: buildInputs is missing `{attribute}` (required by {reason})") - - if errors: - print("nixpkgs recipe check failed:", file=sys.stderr) - for error in errors: - print(f"- {error}", file=sys.stderr) - print( - "\nThe nixpkgs-update bot only rewrites version and hashes. A missing build " - "input here means the next automated bump fails to build in nixpkgs; see " - "packaging/nixpkgs/README.md.", - file=sys.stderr, - ) - return 1 - - print( - f"nixpkgs recipe OK: {len(crates)} default-feature dependencies require " - f"{len(required)} system package(s) ({', '.join(sorted(required))}), " - f"plus {len(NATIVE_BUILD_INPUTS)} native input(s), " - "all declared in packaging/nixpkgs/package.nix and flake.nix." - ) - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/tools/check-process-sites.py b/tools/check-process-sites.py deleted file mode 100755 index 793b77e32..000000000 --- a/tools/check-process-sites.py +++ /dev/null @@ -1,115 +0,0 @@ -#!/usr/bin/env python3 -"""Fail when a process-creation site is not in the reviewed ownership map.""" - -from __future__ import annotations - -import re -import sys -from pathlib import Path - - -ROOT = Path(__file__).resolve().parent.parent -BROKER_ROOT = Path("src/process_broker") -BROKER_BOOTSTRAP = BROKER_ROOT / "bootstrap.rs" -DIRECT_PRODUCTION_ALLOWLIST = { - Path("configurator/src/app/session_catalog.rs"), # separate configurator process - Path("configurator/src/app/daemon_setup/command.rs"), - Path("configurator/src/app/daemon_setup/service.rs"), -} - -PROCESS_PATTERNS = ( - re.compile(r"\b(?:std::process::)?Command::new\b"), - re.compile(r"\bstd::process::Child\b"), - re.compile(r"\blibc::(?:fork|vfork|posix_spawn|posix_spawnp|pthread_atfork)\b"), - re.compile(r"\blibc::SYS_(?:clone|clone3|fork|vfork)\b"), - re.compile(r"\b(?:sh|bash|zsh)\s+-c\b"), -) - - -def is_test_source(path: Path) -> bool: - parts = path.parts - return path.parts[0] == "tests" or "tests" in parts or path.name == "tests.rs" - - -def rust_sources() -> list[Path]: - roots = (ROOT / "src", ROOT / "configurator" / "src", ROOT / "tests") - return sorted(path for root in roots for path in root.rglob("*.rs")) - - -def audit_sites() -> list[str]: - failures: list[str] = [] - for absolute in rust_sources(): - relative = absolute.relative_to(ROOT) - allowed = ( - relative.parts[:2] == BROKER_ROOT.parts - or relative in DIRECT_PRODUCTION_ALLOWLIST - or is_test_source(relative) - ) - for line_number, line in enumerate(absolute.read_text().splitlines(), 1): - code = line.split("//", 1)[0] - if any(pattern.search(code) for pattern in PROCESS_PATTERNS) and not allowed: - failures.append(f"{relative}:{line_number}: unclassified process site: {line.strip()}") - return failures - - -def audit_child_stub() -> list[str]: - source = (ROOT / BROKER_BOOTSTRAP).read_text() - start_marker = " if pid == 0 {" - end_marker = " drop(child_socket);" - if start_marker not in source or end_marker not in source: - return [f"{BROKER_BOOTSTRAP}: raw-clone child-stub markers changed"] - stub = source.split(start_marker, 1)[1].split(end_marker, 1)[0] - failures: list[str] = [] - banned = ( - "format!(", - "log::", - "panic!(", - ".unwrap(", - ".expect(", - "drop(", - "Command::", - "CString::", - "Vec::", - "String::", - "Box::", - ) - for token in banned: - if token in stub: - failures.append(f"{BROKER_BOOTSTRAP}: child stub reaches banned token {token!r}") - libc_calls = set(re.findall(r"libc::([A-Za-z0-9_]+)\s*\(", stub)) - unexpected_calls = libc_calls - {"syscall", "_exit"} - if unexpected_calls: - failures.append( - f"{BROKER_BOOTSTRAP}: child stub reaches unapproved libc calls: " - + ", ".join(sorted(unexpected_calls)) - ) - syscall_names = set(re.findall(r"libc::SYS_([A-Za-z0-9_]+)", stub)) - unexpected_syscalls = syscall_names - { - "fcntl", - "dup3", - "setpgid", - "exit_group", - "close_range", - "execve", - } - if unexpected_syscalls: - failures.append( - f"{BROKER_BOOTSTRAP}: child stub reaches unapproved syscalls: " - + ", ".join(sorted(unexpected_syscalls)) - ) - return failures - - -def main() -> int: - failures = audit_sites() + audit_child_stub() - if failures: - print("process-site audit failed:", file=sys.stderr) - for failure in failures: - print(f" {failure}", file=sys.stderr) - return 1 - print("process-site audit passed") - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/tools/check-rust-source-coverage.py b/tools/check-rust-source-coverage.py deleted file mode 100755 index 4a241817a..000000000 --- a/tools/check-rust-source-coverage.py +++ /dev/null @@ -1,227 +0,0 @@ -#!/usr/bin/env python3 -"""Fail when a repository Rust source is absent from the supported Cargo matrix.""" - -from __future__ import annotations - -import json -import shlex -import subprocess -import sys -from pathlib import Path -from typing import Any - - -REPO_ROOT = Path(__file__).resolve().parent.parent -CARGO_CONFIGURATIONS = ( - ("all features", "--all-features"), - ("no default features", "--no-default-features"), -) -EXPLICIT_ENTRY_POINTS = {Path("build.rs")} - - -class CoverageError(RuntimeError): - """A source-coverage prerequisite or Cargo check failed.""" - - -def run_text(command: list[str]) -> str: - try: - result = subprocess.run( - command, - cwd=REPO_ROOT, - check=False, - capture_output=True, - text=True, - ) - except OSError as error: - raise CoverageError(f"failed to run {command[0]}: {error}") from error - - if result.returncode != 0: - detail = result.stderr.strip() or result.stdout.strip() or "no diagnostic output" - raise CoverageError(f"{' '.join(command)} failed:\n{detail}") - return result.stdout - - -def workspace_package_ids() -> set[str]: - raw = run_text(["cargo", "metadata", "--locked", "--no-deps", "--format-version", "1"]) - try: - metadata = json.loads(raw) - package_ids = {package["id"] for package in metadata["packages"]} - except (KeyError, TypeError, json.JSONDecodeError) as error: - raise CoverageError(f"could not parse cargo metadata: {error}") from error - - if not package_ids: - raise CoverageError("cargo metadata returned no workspace packages") - return package_ids - - -def dep_info_for_artifact(filename: Path) -> Path | None: - if filename.suffix: - stem = filename.stem - else: - stem = filename.name - - candidates = [filename.with_name(f"{stem}.d")] - if stem.startswith("lib"): - candidates.append(filename.with_name(f"{stem[3:]}.d")) - - return next((candidate for candidate in candidates if candidate.is_file()), None) - - -def cargo_dep_info(package_ids: set[str], configuration_flag: str) -> set[Path]: - command = [ - "cargo", - "check", - "--workspace", - "--locked", - "--all-targets", - configuration_flag, - "--message-format=json-render-diagnostics", - ] - try: - process = subprocess.Popen( - command, - cwd=REPO_ROOT, - stdout=subprocess.PIPE, - text=True, - ) - except OSError as error: - raise CoverageError(f"failed to run cargo: {error}") from error - - if process.stdout is None: - process.kill() - raise CoverageError("cargo check did not provide a JSON output stream") - - dep_info_paths: set[Path] = set() - diagnostics: list[str] = [] - parse_errors: list[str] = [] - - for line_number, line in enumerate(process.stdout, start=1): - try: - message: dict[str, Any] = json.loads(line) - except json.JSONDecodeError as error: - parse_errors.append(f"line {line_number}: {error}") - continue - - if message.get("reason") == "compiler-message": - rendered = message.get("message", {}).get("rendered") - if rendered: - diagnostics.append(rendered.rstrip()) - continue - - if message.get("reason") != "compiler-artifact": - continue - if message.get("package_id") not in package_ids: - continue - if "custom-build" in message.get("target", {}).get("kind", []): - continue - - artifact_dep_info = { - dep_info - for raw_filename in message.get("filenames", []) - if (dep_info := dep_info_for_artifact(Path(raw_filename))) is not None - } - if not artifact_dep_info: - target_name = message.get("target", {}).get("name", "unknown target") - process.kill() - process.wait() - raise CoverageError(f"cargo artifact for {target_name} has no adjacent dep-info file") - dep_info_paths.update(artifact_dep_info) - - return_code = process.wait() - if return_code != 0: - detail = "\n".join(diagnostics) or f"cargo exited with status {return_code}" - raise CoverageError(f"{' '.join(command)} failed:\n{detail}") - if parse_errors: - raise CoverageError("invalid cargo JSON output:\n" + "\n".join(parse_errors)) - if not dep_info_paths: - raise CoverageError("cargo check returned no workspace dep-info files") - return dep_info_paths - - -def sources_from_dep_info(path: Path) -> set[Path]: - try: - text = path.read_text(encoding="utf-8") - except OSError as error: - raise CoverageError(f"could not read dep-info {path}: {error}") from error - - first_rule = text.replace("\\\n", " ").split("\n\n", maxsplit=1)[0] - if ":" not in first_rule: - raise CoverageError(f"dep-info has no dependency rule: {path}") - dependencies = first_rule.split(":", maxsplit=1)[1] - try: - tokens = shlex.split(dependencies) - except ValueError as error: - raise CoverageError(f"could not parse dep-info {path}: {error}") from error - - sources: set[Path] = set() - for token in tokens: - if not token.endswith(".rs"): - continue - candidate = Path(token) - if not candidate.is_absolute(): - candidate = REPO_ROOT / candidate - try: - relative = candidate.resolve().relative_to(REPO_ROOT) - except ValueError: - continue - if candidate.is_file(): - sources.add(relative) - return sources - - -def repository_rust_sources() -> set[Path]: - raw = run_text( - [ - "git", - "ls-files", - "-co", - "--exclude-standard", - "--", - "*.rs", - ] - ) - return { - path - for line in raw.splitlines() - if line and (path := Path(line)) and (REPO_ROOT / path).is_file() - } - - -def main() -> int: - try: - package_ids = workspace_package_ids() - - dep_info_paths: set[Path] = set() - for label, flag in CARGO_CONFIGURATIONS: - print(f"Checking Rust source coverage ({label})...", file=sys.stderr) - dep_info_paths.update(cargo_dep_info(package_ids, flag)) - - covered = set(EXPLICIT_ENTRY_POINTS) - for dep_info_path in dep_info_paths: - covered.update(sources_from_dep_info(dep_info_path)) - repository_sources = repository_rust_sources() - uncovered = sorted(repository_sources - covered) - except CoverageError as error: - print(f"Rust source coverage check failed: {error}", file=sys.stderr) - return 2 - - if uncovered: - print( - f"Rust source coverage check failed: {len(uncovered)} source file(s) are not compiled " - "by the supported Cargo matrix:", - file=sys.stderr, - ) - for path in uncovered: - print(f"- {path.as_posix()}", file=sys.stderr) - return 1 - - print( - f"Rust source coverage OK: {len(repository_sources)} source files covered across " - f"{len(CARGO_CONFIGURATIONS)} Cargo configurations " - f"({len(dep_info_paths)} current dep-info files)." - ) - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/tools/check-shared-dependencies.py b/tools/check-shared-dependencies.py deleted file mode 100755 index f7852f63c..000000000 --- a/tools/check-shared-dependencies.py +++ /dev/null @@ -1,108 +0,0 @@ -#!/usr/bin/env python3 -"""Guard explicit upward Rust paths in shared layers. - -This partial source guard understands rooted/parent-relative paths and grouped -use trees, and ignores comments and literals. It does not resolve aliases, -macro expansion, reexports, or Rust's complete module/dependency graph. -""" -from pathlib import Path -import re -import sys - -NON_CODE = re.compile( - r'r(?P#{0,16})".*?"(?P=hashes)|"(?:\\.|[^"\\])*"|' - r"'(?:\\.|[^'\\\n])'|//[^\n]*|/\*", re.S -) -TOKENS = re.compile(r'r#[A-Za-z_][A-Za-z_0-9]*|[A-Za-z_][A-Za-z_0-9]*|::|[{},;*]') -BOUNDARIES = [ - ("src/domain", {"config", "input", "draw", "backend", "ui", "session"}), - ("src/config/validate", {"input", "backend"}), -] - - -def strip_non_code(source): - pieces = [] - position = 0 - while match := NON_CODE.search(source, position): - pieces.append(source[position:match.start()]) - position = match.end() - if match.group() == "/*": - depth = 1 - while depth and (marker := re.search(r'/\*|\*/', source[position:])): - depth += 1 if marker.group() == "/*" else -1 - position += marker.end() - if depth: - position = len(source) - pieces.append(" ") - return ''.join(pieces) + source[position:] - - -def module_path(relative_path): - parts = list(Path(relative_path).with_suffix('').parts[1:]) - if parts[-1] == "mod": - parts.pop() - return parts - - -def has_upward_path(source, relative_path, forbidden): - tokens = [token.removeprefix("r#") for token in TOKENS.findall(strip_non_code(source))] - module = module_path(relative_path) - - def tree(index, prefix): - path = list(prefix) - while index < len(tokens): - token = tokens[index] - if token in {",", ";", "}", "as"}: - break - if token == "{": - index += 1 - while index < len(tokens) and tokens[index] != "}": - rejected, index = tree(index, path) - if rejected: - return True, index - if index < len(tokens) and tokens[index] == "as": - index += 2 - if index < len(tokens) and tokens[index] == ",": - index += 1 - elif index < len(tokens) and tokens[index] != "}": - break - return False, index + 1 - if token == "crate": - path = [] - elif token == "super": - path = path[:-1] - elif token not in {"self", "::", "*"}: - path.append(token) - if path and path[0] in forbidden: - return True, index - index += 1 - if index >= len(tokens) or tokens[index] != "::": - break - index += 1 - return False, index - - return any( - tree(index, module)[0] - for index, token in enumerate(tokens[:-1]) - if token in {"crate", "super", "self"} and tokens[index + 1] == "::" - ) - - -def check(root): - errors = [] - for directory, forbidden in BOUNDARIES: - for path in (root / directory).rglob("*.rs"): - relative = path.relative_to(root).as_posix() - if relative == "src/domain/tests.rs": - continue # Public-path compatibility assertions only. - if has_upward_path(path.read_text(), relative, forbidden): - errors.append(f"{relative}: upward dependency in shared layer") - return errors - - -if __name__ == "__main__": - errors = check(Path(__file__).resolve().parent.parent) - if errors: - print("\n".join(errors), file=sys.stderr) - sys.exit(1) - print("Shared domain and configuration-validation dependency paths passed.") diff --git a/tools/check-version-consistency.sh b/tools/check-version-consistency.sh deleted file mode 100755 index 52e771250..000000000 --- a/tools/check-version-consistency.sh +++ /dev/null @@ -1,395 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -usage() { - cat <<'EOF' -Usage: tools/check-version-consistency.sh [--release-version X.Y.Z[.N]] - -Checks that release/version metadata agrees across: - * Cargo.toml - * configurator/Cargo.toml - * Cargo.lock - * packaging/PKGBUILD - * packaging/.SRCINFO - * flake.nix - * global.json - -Also keeps the configurator's supported libadwaita floor at 1.4 across its -Cargo feature, package metadata, release assertions, and AUR generation. -Raise that floor only as a coordinated release-platform change. - -Also rejects hardcoded release tags in README.md install examples, which go -stale on the next release. Use a placeholder such as RELEASE_TAG, or link to -the latest release, instead. - -Repo packaging metadata is a release template and must use sha256sums=('SKIP'). -Release/AUR automation writes the real source archive checksum after the tag -exists. - -When --release-version is provided, it must either match Cargo.toml exactly -or be a packaging hotfix version of the Cargo version (for example, Cargo -0.9.19 with release 0.9.19.1). Hotfix releases require packaging/PKGBUILD -and packaging/.SRCINFO to use the hotfix version. -EOF -} - -release_version="" -while [[ $# -gt 0 ]]; do - case "$1" in - --release-version) - if [[ $# -lt 2 ]]; then - echo "error: --release-version requires a value" >&2 - usage - exit 1 - fi - release_version="$2" - shift 2 - ;; - -h|--help) - usage - exit 0 - ;; - *) - echo "Unknown arg: $1" >&2 - usage - exit 1 - ;; - esac -done - -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)" - -PYTHON="" -for candidate in python3 python; do - if command -v "$candidate" >/dev/null 2>&1 \ - && "$candidate" -c 'import sys; raise SystemExit(0 if sys.version_info[0] == 3 else 1)' >/dev/null 2>&1; then - PYTHON="$candidate" - break - fi -done - -if [[ -z "$PYTHON" ]]; then - echo "error: python3 or python pointing to Python 3 is required" >&2 - exit 1 -fi - -"$PYTHON" - "$REPO_ROOT" "$release_version" <<'PY' -import json -import pathlib -import re -import sys - -try: - import tomllib -except ImportError: - tomllib = None - -root = pathlib.Path(sys.argv[1]) -release_version = sys.argv[2] or None -errors = [] - -version_re = re.compile(r"^\d+\.\d+\.\d+(?:\.\d+)?$") -checksum_re = re.compile(r"^[0-9a-fA-F]{64}$") -supported_libadwaita_floor = "1.4" -repository_tool_sdk = "11.0.100-rc.1.26425.128" -# A floor raise must add a reviewed runner contract instead of inheriting the -# previous Ubuntu base image by accident. -release_runner_libadwaita_floors = { - "ubuntu-24.04": "1.4", -} - -def read_text(path): - return (root / path).read_text(encoding="utf-8") - -def global_json_sdk(path): - try: - document = json.loads(read_text(path)) - sdk = document["sdk"] - version = sdk["version"] - except (OSError, json.JSONDecodeError, KeyError, TypeError) as error: - errors.append(f"{path} SDK metadata is invalid: {error}") - return None - - if not isinstance(version, str) or not version.strip(): - errors.append(f"{path} SDK metadata is invalid: sdk.version must be a non-empty string") - return None - if sdk.get("rollForward") != "disable": - errors.append(f"{path} SDK rollForward must be disable") - if sdk.get("allowPrerelease") is not True: - errors.append(f"{path} SDK allowPrerelease must be true") - return version.strip() - -def cargo_version(path): - text = read_text(path) - if tomllib is not None: - return tomllib.loads(text)["package"]["version"] - - package_match = re.search(r"(?ms)^\[package\]\s*(.*?)(?:^\[|\Z)", text) - if not package_match: - errors.append(f"{path}: missing [package] section") - return "" - version_match = re.search(r'(?m)^version\s*=\s*"([^"]+)"', package_match.group(1)) - if not version_match: - errors.append(f"{path}: missing package version") - return "" - return version_match.group(1) - -def cargo_libadwaita_features(path): - text = read_text(path) - if tomllib is not None: - dependency = tomllib.loads(text).get("dependencies", {}).get("libadwaita") - if not isinstance(dependency, dict): - errors.append(f"{path}: missing structured libadwaita dependency") - return [] - features = dependency.get("features", []) - if not isinstance(features, list) or not all( - isinstance(feature, str) for feature in features - ): - errors.append(f"{path}: libadwaita features must be a string list") - return [] - return features - - dependency_match = re.search(r"(?m)^libadwaita\s*=\s*\{([^\n]+)\}$", text) - if not dependency_match: - errors.append(f"{path}: missing structured libadwaita dependency") - return [] - features_match = re.search(r"features\s*=\s*\[([^\]]*)\]", dependency_match.group(1)) - if not features_match: - return [] - return re.findall(r'"([^"]+)"', features_match.group(1)) - -def single_metadata_floor(label, path, pattern): - matches = re.findall(pattern, read_text(path), re.MULTILINE) - if len(matches) != 1: - errors.append(f"{label}: expected one libadwaita floor, found {len(matches)}") - return None - return matches[0] - -def workflow_job_runner(path, job_name): - text = read_text(path) - job_match = re.search( - rf"(?m)^ {re.escape(job_name)}:[ \t]*$", - text, - ) - if not job_match: - errors.append(f"{path}: missing {job_name} job") - return None - - remaining = text[job_match.end():] - next_job_match = re.search(r"(?m)^ [A-Za-z0-9_-]+:[ \t]*$", remaining) - job_text = remaining[:next_job_match.start()] if next_job_match else remaining - runners = re.findall(r"(?m)^ runs-on:[ \t]*([^#\n]+?)[ \t]*$", job_text) - if len(runners) != 1: - errors.append( - f"{path} {job_name} job: expected one literal runs-on value, " - f"found {len(runners)}" - ) - return None - return runners[0] - -def lock_package_version(path, package): - pattern = re.compile( - r'\[\[package\]\]\s+name\s*=\s*"' + re.escape(package) + r'"\s+version\s*=\s*"([^"]+)"', - re.MULTILINE, - ) - match = pattern.search(read_text(path)) - return match.group(1) if match else None - -def pkgbuild_version(path): - match = re.search(r"^pkgver=(.+)$", read_text(path), re.MULTILINE) - return match.group(1).strip() if match else None - -def srcinfo_version(path): - match = re.search(r"^\s*pkgver = (.+)$", read_text(path), re.MULTILINE) - return match.group(1).strip() if match else None - -def pkgbuild_sha256sums(path): - match = re.search(r"(?ms)^sha256sums=\((.*?)\)", read_text(path)) - if not match: - return [] - - values = [] - for value_match in re.finditer(r"'([^']*)'|\"([^\"]*)\"|(\S+)", match.group(1)): - value = next(group for group in value_match.groups() if group is not None) - values.append(value.strip()) - return values - -def srcinfo_sha256sums(path): - return [ - match.group(1).strip() - for match in re.finditer(r"^\s*sha256sums = (.+)$", read_text(path), re.MULTILINE) - ] - -def is_hotfix_of(version, base): - return re.fullmatch(re.escape(base) + r"\.\d+", version) is not None - -def require_equal(label, actual, expected): - if actual != expected: - errors.append(f"{label}: expected {expected}, got {actual or 'missing'}") - -def require_template_sha256sums(label, values): - if values == ["SKIP"]: - return - - if not values: - errors.append(f"{label}: expected SKIP template checksum, got missing") - return - - actual = ", ".join(values) - if any(checksum_re.fullmatch(value) for value in values): - errors.append( - f"{label}: expected SKIP template checksum, got fixed SHA {actual}; " - "release/AUR automation writes the real checksum after the tag exists" - ) - else: - errors.append(f"{label}: expected SKIP template checksum, got {actual}") - -root_version = cargo_version("Cargo.toml") -config_version = cargo_version("configurator/Cargo.toml") -require_equal("configurator/Cargo.toml", config_version, root_version) -require_equal("global.json SDK", global_json_sdk("global.json"), repository_tool_sdk) - -expected_libadwaita_feature = "v" + supported_libadwaita_floor.replace(".", "_") -libadwaita_features = cargo_libadwaita_features("configurator/Cargo.toml") -if libadwaita_features != [expected_libadwaita_feature]: - errors.append( - "configurator/Cargo.toml libadwaita features: " - f"expected [{expected_libadwaita_feature!r}], got {libadwaita_features!r}" - ) - -libadwaita_floors = { - "configurator deb libadwaita floor": single_metadata_floor( - "configurator deb libadwaita floor", - "packaging/package.configurator.yaml", - r"^\s*-\s*libadwaita-1-0 \(>= ([0-9]+\.[0-9]+)\)\s*$", - ), - "configurator rpm libadwaita floor": single_metadata_floor( - "configurator rpm libadwaita floor", - "packaging/package.configurator.yaml", - r"^\s*-\s*libadwaita >= ([0-9]+\.[0-9]+)\s*$", - ), - "packaging/PKGBUILD libadwaita floor": single_metadata_floor( - "packaging/PKGBUILD libadwaita floor", - "packaging/PKGBUILD", - r"^\s*'libadwaita>=([0-9]+\.[0-9]+)'\s*$", - ), - "packaging/.SRCINFO libadwaita floor": single_metadata_floor( - "packaging/.SRCINFO libadwaita floor", - "packaging/.SRCINFO", - r"^\s*depends = libadwaita>=([0-9]+\.[0-9]+)\s*$", - ), - "AUR updater generated libadwaita floor": single_metadata_floor( - "AUR updater generated libadwaita floor", - "tools/update-aur-from-manifest.sh", - r"^\s*ensure_runtime_dependency 'libadwaita>=([0-9]+\.[0-9]+)' gcc-libs\s*$", - ), - "AUR updater PKGBUILD validation floor": single_metadata_floor( - "AUR updater PKGBUILD validation floor", - "tools/update-aur-from-manifest.sh", - r'''^\s*&& grep -Eq "[^"\n]*libadwaita>=([0-9]+\.[0-9]+)[^"\n]*" PKGBUILD''', - ), - "AUR updater .SRCINFO validation floor": single_metadata_floor( - "AUR updater .SRCINFO validation floor", - "tools/update-aur-from-manifest.sh", - r"^\s*&& grep -Fxq .*depends = libadwaita>=([0-9]+\.[0-9]+).*\.SRCINFO", - ), -} -for label, floor in libadwaita_floors.items(): - if floor is not None: - require_equal(label, floor, supported_libadwaita_floor) - -release_package_runner = workflow_job_runner( - ".github/workflows/build-packages.yml", - "package", -) -if release_package_runner is not None: - release_runner_floor = release_runner_libadwaita_floors.get(release_package_runner) - if release_runner_floor is None: - errors.append( - "release package runner libadwaita floor: " - f"no reviewed contract for {release_package_runner}" - ) - else: - require_equal( - f"release package runner {release_package_runner} libadwaita floor", - supported_libadwaita_floor, - release_runner_floor, - ) - -require_equal( - "Cargo.lock wayscriber", - lock_package_version("Cargo.lock", "wayscriber"), - root_version, -) -require_equal( - "Cargo.lock wayscriber-configurator", - lock_package_version("Cargo.lock", "wayscriber-configurator"), - root_version, -) - -if not version_re.fullmatch(root_version): - errors.append(f"Cargo.toml version has unsupported format: {root_version}") - -packaging_version = pkgbuild_version("packaging/PKGBUILD") -srcinfo_pkgver = srcinfo_version("packaging/.SRCINFO") - -if release_version: - if not version_re.fullmatch(release_version): - errors.append(f"release version has unsupported format: {release_version}") - elif release_version != root_version and not is_hotfix_of(release_version, root_version): - errors.append( - f"release version {release_version} must equal Cargo version {root_version} " - f"or be a hotfix of it, such as {root_version}.1" - ) - expected_packaging_version = release_version -else: - expected_packaging_version = root_version - if packaging_version and is_hotfix_of(packaging_version, root_version): - expected_packaging_version = packaging_version - -require_equal("packaging/PKGBUILD pkgver", packaging_version, expected_packaging_version) -require_equal("packaging/.SRCINFO pkgver", srcinfo_pkgver, expected_packaging_version) -require_template_sha256sums( - "packaging/PKGBUILD sha256sums", pkgbuild_sha256sums("packaging/PKGBUILD") -) -require_template_sha256sums( - "packaging/.SRCINFO sha256sums", srcinfo_sha256sums("packaging/.SRCINFO") -) - -flake_text = read_text("flake.nix") -if "builtins.fromTOML (builtins.readFile ./Cargo.toml)" not in flake_text: - errors.append("flake.nix package version should be derived from Cargo.toml") -if not all( - token in flake_text - for token in ("package.rust-version", "rustToolchain.version", "versionAtLeast") -): - errors.append( - "flake.nix should compare the selected rustc against Cargo.toml rust-version" - ) - -# Install examples that pin a concrete tag are stale one release later. -readme_pin_patterns = ( - (r"wayscriber\?ref=v?\d+\.\d+\.\d+(?:\.\d+)?", "pinned flake ref"), - (r"/releases/(?:tag|download)/v?\d+\.\d+\.\d+(?:\.\d+)?", "pinned release URL"), -) -readme_text = read_text("README.md") -for pattern, label in readme_pin_patterns: - for match in sorted(set(re.findall(pattern, readme_text))): - errors.append( - f"README.md: {label} '{match}' goes stale on the next release; " - "use a RELEASE_TAG placeholder or link to /releases/latest" - ) - -if errors: - print("Version consistency check failed:", file=sys.stderr) - for error in errors: - print(f"- {error}", file=sys.stderr) - sys.exit(1) - -print( - f"Version consistency OK: Cargo={root_version}, " - f"packaging={expected_packaging_version}, checksum=SKIP, " - f"libadwaita={supported_libadwaita_floor}" -) -PY diff --git a/tools/code-health-report.sh b/tools/code-health-report.sh deleted file mode 100755 index 994ada6f7..000000000 --- a/tools/code-health-report.sh +++ /dev/null @@ -1,449 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd) -REPO_ROOT=$(cd -- "$SCRIPT_DIR/.." && pwd) - -if command -v python3 >/dev/null 2>&1; then - PYTHON=python3 -elif command -v python >/dev/null 2>&1; then - PYTHON=python -else - echo "report=wayscriber-code-health" - echo "status=error" - echo "error=python_not_found" - exit 0 -fi - -set +e -"$PYTHON" - "$REPO_ROOT" <<'PY' -from __future__ import annotations - -import re -import subprocess -import sys -from pathlib import Path - -repo_root = Path(sys.argv[1]).resolve() - - -def run_git_ls_files() -> tuple[list[Path], str | None, str | None]: - try: - result = subprocess.run( - ["git", "-C", str(repo_root), "ls-files", "-co", "--exclude-standard", "--", "*.rs"], - check=False, - capture_output=True, - text=True, - ) - except OSError as exc: - return [], f"git_unavailable\t{exc}", None - stderr = result.stderr.strip() - if result.returncode != 0: - return [], f"git_ls_files_failed\t{stderr}", None - warning = f"git_ls_files_stderr\t{stderr}" if stderr else None - files: list[Path] = [] - seen: set[str] = set() - for line in result.stdout.splitlines(): - if not line: - continue - relative_path = Path(line) - if not (repo_root / relative_path).is_file(): - continue - path_key = relative_path.as_posix() - if path_key in seen: - continue - seen.add(path_key) - files.append(relative_path) - return files, None, warning - - -def read_text(relative_path: Path) -> str: - return (repo_root / relative_path).read_text(encoding="utf-8", errors="replace") - - -def physical_line_count(text: str) -> int: - if not text: - return 0 - return text.count("\n") + (0 if text.endswith("\n") else 1) - - -def is_test_path(relative_path: Path) -> bool: - parts = relative_path.as_posix().split("/") - name = relative_path.name - return ( - "tests" in parts - or name in {"tests.rs", "test_helpers.rs", "test_support.rs"} - or name.startswith("test_") - or name.endswith("_tests.rs") - ) - - -def scrub_rust_code(text: str) -> str: - output: list[str] = [] - index = 0 - length = len(text) - state = "code" - block_depth = 0 - raw_hashes = "" - - def blank(char: str) -> str: - return "\n" if char == "\n" else " " - - while index < length: - char = text[index] - next_char = text[index + 1] if index + 1 < length else "" - - if state == "code": - if char == "/" and next_char == "/": - output.extend(" ") - index += 2 - state = "line_comment" - continue - if char == "/" and next_char == "*": - output.extend(" ") - index += 2 - state = "block_comment" - block_depth = 1 - continue - if char == '"': - output.append(" ") - index += 1 - state = "string" - continue - if char == "r" or (char == "b" and next_char == "r"): - raw_start = index + (2 if char == "b" else 1) - hash_end = raw_start - while hash_end < length and text[hash_end] == "#": - hash_end += 1 - if hash_end < length and text[hash_end] == '"': - output.extend(" " * (hash_end - index + 1)) - index = hash_end + 1 - raw_hashes = text[raw_start:hash_end] - state = "raw_string" - continue - output.append(char) - index += 1 - continue - - if state == "line_comment": - output.append(blank(char)) - index += 1 - if char == "\n": - state = "code" - continue - - if state == "block_comment": - if char == "/" and next_char == "*": - output.extend(" ") - index += 2 - block_depth += 1 - continue - if char == "*" and next_char == "/": - output.extend(" ") - index += 2 - block_depth -= 1 - if block_depth == 0: - state = "code" - continue - output.append(blank(char)) - index += 1 - continue - - if state == "string": - if char == "\\" and next_char: - output.append(blank(char)) - output.append(blank(next_char)) - index += 2 - continue - output.append(blank(char)) - index += 1 - if char == '"': - state = "code" - continue - - if state == "raw_string": - output.append(blank(char)) - index += 1 - if char == '"' and text.startswith(raw_hashes, index): - output.extend(" " * len(raw_hashes)) - index += len(raw_hashes) - state = "code" - continue - - return "".join(output) - - -cfg_attr_start_pattern = re.compile(r"#\s*\[\s*cfg\s*\(", re.MULTILINE) - - -def preserve_newlines_as_spaces(text: str) -> str: - return "".join("\n" if char == "\n" else " " for char in text) - - -def find_attribute_end(code: str, start: int) -> int: - index = start - bracket_depth = 0 - while index < len(code): - char = code[index] - if char == "[": - bracket_depth += 1 - elif char == "]": - bracket_depth -= 1 - if bracket_depth == 0: - return index + 1 - index += 1 - return start - - -def skip_attributes_and_whitespace(code: str, start: int) -> int: - index = start - while index < len(code): - while index < len(code) and code[index].isspace(): - index += 1 - if code.startswith("#[", index): - next_index = find_attribute_end(code, index + 1) - if next_index <= index: - return index - index = next_index - continue - return index - return index - - -def is_cfg_test_only_attribute(attribute: str) -> bool: - compact = re.sub(r"\s+", "", attribute) - if compact == "#[cfg(test)]": - return True - if "not(test)" in compact: - return False - return compact.startswith("#[cfg(all(") and re.search(r"\btest\b", attribute) is not None - - -def find_item_end(code: str, start: int) -> int: - paren_depth = 0 - bracket_depth = 0 - index = start - while index < len(code): - char = code[index] - if char == "(": - paren_depth += 1 - elif char == ")" and paren_depth > 0: - paren_depth -= 1 - elif char == "[": - bracket_depth += 1 - elif char == "]" and bracket_depth > 0: - bracket_depth -= 1 - elif char == ";" and paren_depth == 0 and bracket_depth == 0: - return index + 1 - elif char == "{" and paren_depth == 0 and bracket_depth == 0: - body_end = find_matching_brace(code, index) - return len(code) if body_end is None else body_end + 1 - index += 1 - return len(code) - - -def strip_cfg_test_code(code: str) -> str: - chars = list(code) - for match in cfg_attr_start_pattern.finditer(code): - attribute_end = find_attribute_end(code, match.start() + 1) - attribute = code[match.start() : attribute_end] - if not is_cfg_test_only_attribute(attribute): - continue - item_start = skip_attributes_and_whitespace(code, attribute_end) - item_end = find_item_end(code, item_start) - replacement = preserve_newlines_as_spaces(code[match.start() : item_end]) - chars[match.start() : item_end] = replacement - return "".join(chars) - - -fn_name_pattern = re.compile( - r"\bfn\s+([A-Za-z_][A-Za-z0-9_]*)\s*(?:<[^>{;]*>)?\s*\(", - re.MULTILINE, -) - - -def line_number_at(text: str, index: int) -> int: - return text.count("\n", 0, index) + 1 - - -def find_function_body(code: str, start: int) -> int | None: - paren_depth = 1 - bracket_depth = 0 - index = start - while index < len(code): - char = code[index] - if char == "(": - paren_depth += 1 - elif char == ")" and paren_depth > 0: - paren_depth -= 1 - elif char == "[": - bracket_depth += 1 - elif char == "]" and bracket_depth > 0: - bracket_depth -= 1 - elif char == "{" and paren_depth == 0 and bracket_depth == 0: - return index - elif char == ";" and paren_depth == 0 and bracket_depth == 0: - return None - index += 1 - return None - - -def find_matching_brace(code: str, body_start: int) -> int | None: - depth = 0 - for index in range(body_start, len(code)): - char = code[index] - if char == "{": - depth += 1 - elif char == "}": - depth -= 1 - if depth == 0: - return index - return None - - -def long_functions(relative_path: Path, code: str) -> list[tuple[int, int, str, str]]: - findings: list[tuple[int, int, str, str]] = [] - for match in fn_name_pattern.finditer(code): - body_start = find_function_body(code, match.end()) - if body_start is None: - continue - body_end = find_matching_brace(code, body_start) - if body_end is None: - continue - start_line = line_number_at(code, match.start()) - end_line = line_number_at(code, body_end) - lines = end_line - start_line + 1 - if lines > 120: - findings.append((lines, start_line, relative_path.as_posix(), match.group(1))) - return findings - - -prod_patterns = { - "unwrap": re.compile(r"\.\s*unwrap\s*\("), - "expect": re.compile(r"\.\s*expect\s*\("), - "panic": re.compile(r"\bpanic\s*!"), - "unsafe": re.compile(r"\bunsafe\b"), -} -allow_dead_code_pattern = re.compile(r"#\s*\[\s*allow\s*\([^)]*\bdead_code\b[^)]*\)\s*\]") -allow_unused_imports_pattern = re.compile( - r"#\s*\[\s*allow\s*\([^)]*\bunused_imports\b[^)]*\)\s*\]" -) -direct_fs_write_pattern = re.compile(r"(? 500), reverse=True) -long_function_findings.sort(reverse=True) -direct_fs_write_files.sort() - -report_errors: list[str] = [] -report_warnings: list[str] = [] -if discovery_error: - report_errors.append("discovery") -if discovery_warning: - report_warnings.append("discovery") -if read_errors: - report_errors.append("read") - -print("report=wayscriber-code-health") -if report_errors: - print("status=error") -elif report_warnings: - print("status=warning") -else: - print("status=ok") -if report_errors: - print(f"errors={','.join(report_errors)}") -if report_warnings: - print(f"warnings={','.join(report_warnings)}") -if discovery_error: - error_name, _, error_detail = discovery_error.partition("\t") - print(f"error={error_name}") - if error_detail: - print(f"error_detail={error_detail}") -if discovery_warning: - warning_name, _, warning_detail = discovery_warning.partition("\t") - print(f"warning={warning_name}") - if warning_detail: - print(f"warning_detail={warning_detail}") -print(f"repo_root={repo_root}") -print(f"rust_files={len(rust_files)}") -print(f"rust_physical_lines={total_lines}") -print(f"files_over_500={len(files_over_500)}") -print(f"functions_over_120={len(long_function_findings)}") -print(f"production_unwrap={production_counts['unwrap']}") -print(f"production_expect={production_counts['expect']}") -print(f"production_panic={production_counts['panic']}") -print(f"production_unsafe={production_counts['unsafe']}") -print(f"allow_dead_code={allow_dead_code}") -print(f"allow_unused_imports={allow_unused_imports}") -print(f"direct_fs_write_files={len(direct_fs_write_files)}") -print(f"read_errors={len(read_errors)}") - - -def print_section(name: str, rows: list[str]) -> None: - print() - print(f"{name}:") - if not rows: - print(" none") - return - for row in rows: - print(f" {row}") - - -print_section( - "files_over_500", - [f"{lines}\t{path}" for lines, path in files_over_500], -) -print_section( - "functions_over_120", - [f"{lines}\t{path}:{line}\t{name}" for lines, line, path, name in long_function_findings], -) -print_section("direct_fs_write_files", direct_fs_write_files) -print_section("read_errors", read_errors) -PY -status=$? -set -e - -if [ "$status" -ne 0 ]; then - echo "report=wayscriber-code-health" - echo "status=error" - echo "error=python_report_failed" - echo "python_exit=$status" -fi - -exit 0 diff --git a/tools/create-release-tag.sh b/tools/create-release-tag.sh deleted file mode 100755 index 9fab1956b..000000000 --- a/tools/create-release-tag.sh +++ /dev/null @@ -1,59 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)" - -usage() { - cat <<'EOF' -Usage: tools/create-release-tag.sh - -Creates an annotated git tag "v". - -Requirements: -- Clean working tree (no staged/unstaged/untracked changes) -- Tag must not already exist - -Examples: - tools/create-release-tag.sh 0.9.2 - tools/create-release-tag.sh 0.9.9.1 -EOF -} - -require_bin() { - if ! command -v "$1" >/dev/null 2>&1; then - echo "error: $1 is required" >&2 - exit 1 - fi -} - -if [[ "${1-}" == "-h" || "${1-}" == "--help" || $# -ne 1 ]]; then - usage - exit 0 -fi - -require_bin git - -version="$1" -tag="v${version}" - -if ! [[ "$version" =~ ^[0-9]+\.[0-9]+\.[0-9]+(\.[0-9]+)?$ ]]; then - echo "error: invalid version format: $version (expected MAJOR.MINOR.PATCH[.HOTFIX])" >&2 - exit 1 -fi - -cd "$REPO_ROOT" -bash tools/check-version-consistency.sh --release-version "$version" - -if [[ -n "$(git status --porcelain --untracked-files=all)" ]]; then - echo "error: working tree is not clean; commit/stash changes before tagging" >&2 - exit 1 -fi - -if git rev-parse -q --verify "refs/tags/${tag}" >/dev/null; then - echo "error: tag ${tag} already exists" >&2 - exit 1 -fi - -git tag -a "${tag}" -m "Release ${tag}" -echo "Created tag ${tag}" diff --git a/tools/csharp-tests/CoreTests.cs b/tools/csharp-tests/CoreTests.cs index 82a835a5f..efe854e63 100644 --- a/tools/csharp-tests/CoreTests.cs +++ b/tools/csharp-tests/CoreTests.cs @@ -5,6 +5,9 @@ namespace Wayscriber.Tools.Tests; public sealed class CoreTests { + // The exit code git uses for a fatal error, such as running outside a repository. + private const int GitFatalExitCode = 128; + [Theory] [InlineData( "1.2.3", "1.2.3", false )] [InlineData( "1.2.3.4", "1.2.3", true )] @@ -16,9 +19,33 @@ public void ReleaseVersionParses( string value, string cargo, bool hotfix ) Assert.Equal( value, version.ToString( ) ); } + [Theory] + [InlineData( "1.2" )] + [InlineData( "01.2.3" )] + [InlineData( "1.2.3\n" )] + [InlineData( "1.2.3.04" )] + [InlineData( "99999999999.0.0" )] + public void ReleaseVersionRejectsMalformedInput( string value ) => + Assert.Throws( ( ) => ReleaseVersion.Parse( value ) ); + + [Theory] + [InlineData( "wayscriber-v1.2.3-linux-x86_64", true )] + [InlineData( "wayscriber-v1.2.3.1-linux-x86_64", true )] + [InlineData( "wayscriber-v01.2.3-linux-x86_64", false )] + [InlineData( "wayscriber-v1.2-linux-x86_64", false )] + [InlineData( "wayscriber-v-linux-x86_64", false )] + [InlineData( "wayscriber-1.2.3-linux-x86_64", false )] + [InlineData( "wayscriber-v1.2.3-linux-aarch64", false )] + public void ReleaseArchiveRootsNameAReleaseVersion( string root, bool accepted ) => + Assert.Equal( accepted, PackagingCommands.IsReleaseArchiveRoot( root ) ); + [Fact] - public void ReleaseVersionRejectsMalformedInput( ) => - Assert.Throws( ( ) => ReleaseVersion.Parse( "1.2" ) ); + public void NextPatchReportsAnExhaustedPatchNumber( ) + { + var error = Assert.Throws( ( ) => ReleaseVersion.Parse( $"1.2.{int.MaxValue}" ).NextPatch( ) ); + + Assert.Equal( ExitCodes.InvalidArguments, error.ExitCode ); + } [Fact] public void ProcessArgumentsAreQuotedForDiagnostics( ) @@ -31,7 +58,7 @@ public void ProcessArgumentsAreQuotedForDiagnostics( ) public void RustMaskPreservesLineNumbersAndRemovesCommentsAndStrings( ) { const string source = "fn one() { /* { */ call(); }\n// call()\nlet text = \"call()\";\n"; - var masked = ConfigWriterAudit.StripRustCommentsAndStrings( source ); + var masked = RustSource.StripRustCommentsAndStrings( source ); Assert.Equal( source.Count( character => character == '\n' ), masked.Count( character => character == '\n' ) ); Assert.Single( System.Text.RegularExpressions.Regex.Matches( masked, @"\bcall\s*\(" ).Cast( ) ); } @@ -40,12 +67,51 @@ public void RustMaskPreservesLineNumbersAndRemovesCommentsAndStrings( ) public void CfgTestRemovalHandlesAllTestPredicate( ) { const string source = "live();\n#[cfg(all(test, feature = \"x\"))]\nfn test_only() { hidden(); }\nlive_again();"; - var production = ConfigWriterAudit.RemoveCfgTestBlocks( ConfigWriterAudit.StripRustCommentsAndStrings( source ) ); + var production = RustSource.RemoveCfgTestBlocks( RustSource.StripRustCommentsAndStrings( source ) ); Assert.Contains( "live();", production ); Assert.Contains( "live_again();", production ); Assert.DoesNotContain( "hidden", production ); } + // The report never fails: discovery and read problems show in its status lines. + [Fact] + public async Task CodeHealthReportsDiscoveryWarningsAndUnreadableFiles( ) + { + if ( !OperatingSystem.IsLinux( ) || Environment.IsPrivilegedProcess ) + { + return; + } + + using var directory = new TemporaryDirectory( "wayscriber-code-health-test" ); + Directory.CreateDirectory( Path.Combine( directory.Path, "src" ) ); + File.WriteAllText( Path.Combine( directory.Path, "src/lib.rs" ), "fn main() {}\n" ); + var locked = Path.Combine( directory.Path, "src/locked.rs" ); + File.WriteAllText( locked, "fn hidden() {}\n" ); + File.SetUnixFileMode( locked, UnixFileMode.None ); + var git = new GitListing( ExitCodes.Success, "src/lib.rs\nsrc/locked.rs\n", "warning: index is stale\n" ); + + var report = await RunCodeHealth( directory.Path, git ); + + Assert.Contains( "status=error\nerrors=read\nwarnings=discovery\n" + + "warning=git_ls_files_stderr\nwarning_detail=warning: index is stale\n", report, StringComparison.Ordinal ); + Assert.Contains( "\nrust_files=2\n", report, StringComparison.Ordinal ); + Assert.Contains( "\nread_errors=1\n", report, StringComparison.Ordinal ); + Assert.Contains( "\nread_errors:\n src/locked.rs\t", report, StringComparison.Ordinal ); + } + + [Fact] + public async Task CodeHealthReportsAFailedDiscovery( ) + { + using var directory = new TemporaryDirectory( "wayscriber-code-health-test" ); + var git = new GitListing( GitFatalExitCode, string.Empty, "fatal: not a git repository\n" ); + + var report = await RunCodeHealth( directory.Path, git ); + + Assert.StartsWith( "report=wayscriber-code-health\nstatus=error\nerrors=discovery\nerror=git_ls_files_failed\n" + + "error_detail=fatal: not a git repository\n", report, StringComparison.Ordinal ); + Assert.Contains( "\nrust_files=0\n", report, StringComparison.Ordinal ); + } + [Fact] public void InstallerManifestRejectsDuplicatePaths( ) { @@ -68,7 +134,7 @@ public void AtomicFileSetReplacesCompleteContent( ) [Fact] public void CurrentAssetManifestsProduceAllRecipeChannels( ) { - var root = FindRepository( ); + var root = TestRepository.Root; var recipe = AssetsCommand.CreateRecipe( root ); Assert.NotNull( recipe["source"] ); Assert.NotNull( recipe["bin"] ); @@ -78,12 +144,35 @@ public void CurrentAssetManifestsProduceAllRecipeChannels( ) [Fact] public void CurrentVersionMetadataIsConsistent( ) { - Assert.Empty( VersionCommands.Validate( FindRepository( ) ) ); + Assert.Empty( VersionCommands.Validate( TestRepository.Root ) ); } - private static string FindRepository( ) + private static async Task RunCodeHealth( string root, IProcessRunner git ) { - var directory = AppContext.GetData( "EntryPointFileDirectoryPath" ) as string ?? Environment.CurrentDirectory; - return Path.GetFullPath( Path.Combine( directory, ".." ) ); + using var output = new StringWriter( ); + var command = ReportCommands.Commands.Single( command => command.Name == CommandNames.CodeHealth ); + var context = new ToolContext( root, output, TextWriter.Null, git, CancellationToken.None ); + + Assert.Equal( ExitCodes.Success, await command.Handler( context, [] ) ); + + return output.ToString( ); + } + + // Answers the report's `git ls-files` with a fixed listing, rejecting an exit + // code the request does not allow as the real process runner does. + private sealed class GitListing( int exitCode, string output, string error ) : IProcessRunner + { + public Task RunAsync( ProcessRequest request, CancellationToken cancellationToken ) + { + Assert.Equal( Programs.Git, request.FileName ); + + var result = new ProcessResult( exitCode, output, error ); + if ( !result.IsSuccess && request.AllowedExitCodes?.Contains( exitCode ) != true ) + { + throw new ToolException( $"{request.FileName} exited with code {exitCode}.", exitCode ); + } + + return Task.FromResult( result ); + } } } diff --git a/tools/csharp-tests/GtkGateParityTests.cs b/tools/csharp-tests/GtkGateParityTests.cs index ed06e3d1a..07540fcac 100644 --- a/tools/csharp-tests/GtkGateParityTests.cs +++ b/tools/csharp-tests/GtkGateParityTests.cs @@ -60,7 +60,7 @@ public GtkGateFixture( string? omittedMarker, int childExitCode ) Directory.CreateDirectory( _bin ); var tools = Path.Combine( _directory.Path, "tools" ); Directory.CreateDirectory( tools ); - File.Copy( Path.Combine( FindRepository( ), "tools/test-gtk-widgets.sh" ), Path.Combine( tools, "test-gtk-widgets.sh" ) ); + File.Copy( Path.Combine( TestRepository.Root, "tools/test-gtk-widgets.sh" ), Path.Combine( tools, "test-gtk-widgets.sh" ) ); var socket = Path.Combine( _directory.Path, "socket" ); _socket.Bind( new UnixDomainSocketEndPoint( socket ) ); @@ -154,10 +154,4 @@ private void WriteExecutable( string name, string content ) File.SetUnixFileMode( path, UnixFileMode.UserRead | UnixFileMode.UserWrite | UnixFileMode.UserExecute ); } } - - private static string FindRepository( ) - { - var directory = AppContext.GetData( "EntryPointFileDirectoryPath" ) as string ?? Environment.CurrentDirectory; - return Path.GetFullPath( Path.Combine( directory, ".." ) ); - } } diff --git a/tools/csharp-tests/ReleaseParityRegressionTests.cs b/tools/csharp-tests/ReleaseParityRegressionTests.cs index fcbf1948f..a14b14cee 100644 --- a/tools/csharp-tests/ReleaseParityRegressionTests.cs +++ b/tools/csharp-tests/ReleaseParityRegressionTests.cs @@ -11,7 +11,7 @@ public sealed class ReleaseParityRegressionTests private const string SyntheticPassphrase = "synthetic-passphrase"; private const string RepositoryDeployKey = "repository-deploy-key"; private const string TestDotnetSdk = "11.0.100-rc.1.26425.128"; - private const string DotnetLogVariable = "WAYSCRIBER_DOTNET_LOG"; + private const string CommandLogVariable = "WAYSCRIBER_COMMAND_LOG"; [Fact] public async Task AurAskpassEnvironmentIsScopedToSshAdd( ) @@ -148,7 +148,7 @@ public async Task RepositoryDeployKeyAlwaysEndsWithOnePreservedNewline( string c [Fact] public void ManagedDesktopAssetBlocksReplaceAndRejectStaleEntries( ) { - var recipe = AssetsCommand.CreateRecipe( FindRepository( ) )["source"]!.AsObject( ); + var recipe = AssetsCommand.CreateRecipe( TestRepository.Root )["source"]!.AsObject( ); var marker = recipe["marker"]!.GetValue( ); var end = recipe["end_marker"]!.GetValue( ); var anchor = recipe["anchor"]!.GetValue( ); @@ -170,7 +170,7 @@ public void ManagedDesktopAssetBlocksReplaceAndRejectStaleEntries( ) [InlineData( PackageChannels.Configurator )] public void CompleteMarkerlessDesktopAssetsAreMigratedAndStaleEntriesAreRemoved( string channel ) { - var recipe = AssetsCommand.CreateRecipe( FindRepository( ) )[channel]!.AsObject( ); + var recipe = AssetsCommand.CreateRecipe( TestRepository.Root )[channel]!.AsObject( ); var anchor = recipe["anchor"]!.GetValue( ); var expectedLines = recipe["lines"]!.AsArray( ).Select( item => item!.GetValue( ) ).ToArray( ); var staleName = channel == PackageChannels.Configurator ? "retired-legacy-icon" : "retired-icon"; @@ -191,7 +191,7 @@ public void CompleteMarkerlessDesktopAssetsAreMigratedAndStaleEntriesAreRemoved( [InlineData( PackageChannels.Configurator )] public void MarkerlessPreviousManifestSubsetsAreMigrated( string channel ) { - var recipe = AssetsCommand.CreateRecipe( FindRepository( ) )[channel]!.AsObject( ); + var recipe = AssetsCommand.CreateRecipe( TestRepository.Root )[channel]!.AsObject( ); var anchor = recipe["anchor"]!.GetValue( ); var previousManifestLines = recipe["lines"]!.AsArray( ).Select( item => item!.GetValue( ) ).SkipLast( 1 ); var input = anchor + "\n\n" + string.Join( '\n', previousManifestLines ); @@ -207,50 +207,104 @@ public void InstallDirectoryNormalizationPreservesFilesystemRoot( ) Assert.Equal( Path.GetPathRoot( Environment.CurrentDirectory ), NativeDesktopCommands.NormalizeBinDirectory( "/" ) ); } + // The local gate builds the C# apps, then runs `ci lint-and-test`'s steps in the same order. [Fact] - public async Task StandaloneGateUsesSdkResolutionAndRunsEveryCsharpBuild( ) + public async Task LocalGateBuildsTheCsharpAppsThenRunsTheCiPlan( ) { if ( !OperatingSystem.IsLinux( ) ) { return; } - using var fixture = CreateStandaloneGateFixture( dotnetSdkAvailable: true ); - var dotnetLog = Path.Combine( fixture.Path, "dotnet.log" ); + using var fixture = CreateLocalGateFixture( dotnetSdkAvailable: true ); + var commandLog = Path.Combine( fixture.Path, "commands.log" ); - var result = await RunStandaloneGate( fixture.Path, dotnetLog, new HashSet { ExitCodes.Success } ); + var result = await RunLocalGate( fixture.Path, commandLog, new HashSet { ExitCodes.Success } ); Assert.Equal( ExitCodes.Success, result.ExitCode ); - var invocations = File.ReadAllText( dotnetLog ); - Assert.Contains( "build tools/wayscriber.cs --disable-build-servers --verbosity quiet", invocations, StringComparison.Ordinal ); - Assert.Contains( "build tools/install.cs --disable-build-servers --verbosity quiet", invocations, StringComparison.Ordinal ); - Assert.Contains( "build tools/wayscriber.tests.cs --disable-build-servers --verbosity quiet", invocations, StringComparison.Ordinal ); - Assert.Contains( "format style tools/wayscriber.cs --no-restore --verify-no-changes", invocations, StringComparison.Ordinal ); - Assert.Contains( "format whitespace tools/install.cs --no-restore --verify-no-changes", invocations, StringComparison.Ordinal ); - Assert.Contains( "format whitespace tools/wayscriber.tests.cs --no-restore --verify-no-changes", invocations, StringComparison.Ordinal ); - Assert.Contains( "run tools/wayscriber.tests.cs --no-build --verbosity quiet", invocations, StringComparison.Ordinal ); + var commands = File.ReadAllLines( commandLog ); + var builds = new[] { "tools/wayscriber.cs", "tools/install.cs", "tools/wayscriber.tests.cs" } + .Select( app => $"{Programs.Dotnet} build {app} --disable-build-servers --verbosity quiet" ); + // The gate runs a repository check through the built tool; C# runs it in process. + var plan = DevelopmentCommands.LintAndTestPlan.Select( step => step.Program is { } program + ? $"{program} {string.Join( ' ', step.Arguments )}" + : $"{Programs.Dotnet} run tools/wayscriber.cs --no-build -- {string.Join( ' ', step.Arguments )}" ); + Assert.Equal( [.. builds, .. plan], commands ); } [Fact] - public async Task StandaloneGateSkipsOnlyCsharpChecksWhenPinnedSdkIsUnavailable( ) + public async Task LocalGateRequiresEachIsolatedRenderTestToPassExactlyOnce( ) { if ( !OperatingSystem.IsLinux( ) ) { return; } - using var fixture = CreateStandaloneGateFixture( dotnetSdkAvailable: false ); - var dotnetLog = Path.Combine( fixture.Path, "dotnet.log" ); + using var fixture = CreateLocalGateFixture( dotnetSdkAvailable: true, cargoOutput: "test result: ok. 0 passed; 0 failed;" ); + var commandLog = Path.Combine( fixture.Path, "commands.log" ); - var result = await RunStandaloneGate( fixture.Path, dotnetLog, new HashSet { ExitCodes.Success } ); + var result = await RunLocalGate( fixture.Path, commandLog, new HashSet { ExitCodes.Failure } ); - Assert.Equal( ExitCodes.Success, result.ExitCode ); - Assert.Contains( "SDK selected by global.json is unavailable", result.StandardOutput, StringComparison.Ordinal ); - Assert.False( File.Exists( dotnetLog ) ); + Assert.Equal( ExitCodes.Failure, result.ExitCode ); + Assert.Contains( "Expected exactly one passing isolated render test", result.StandardError, StringComparison.Ordinal ); + } + + // Each kind of `ci lint-and-test` step runs what it names: repository checks in + // process, the others as dotnet or cargo with the step's arguments. + [Fact] + public async Task LintStepsRunTheProgramTheirKindNames( ) + { + var runner = new LintStepRunner( "test result: ok. 1 passed; 0 failed;\n" ); + using var output = new StringWriter( ); + var context = new ToolContext( TestRepository.Root, output, TextWriter.Null, runner, CancellationToken.None ); + + await DevelopmentCommands.RunLintStep( context, new( LintStepKind.RepositoryCheck, [CommandAreas.Version, CommandNames.Check] ) ); + await DevelopmentCommands.RunLintStep( context, new( LintStepKind.Dotnet, ["format", "style", "tools/wayscriber.cs"] ) ); + await DevelopmentCommands.RunLintStep( context, new( LintStepKind.Cargo, ["fmt", "--all"] ) ); + await DevelopmentCommands.RunLintStep( context, new( LintStepKind.IsolatedRenderTest, ["test", "--lib", "probe"], "probe" ) ); + + Assert.Equal( + [$"{Programs.Dotnet} format style tools/wayscriber.cs", $"{Programs.Cargo} fmt --all", $"{Programs.Cargo} test --lib probe"], + runner.Requests.Select( request => $"{request.FileName} {string.Join( ' ', request.Arguments )}" ) ); + Assert.Contains( "Running: wayscriber version check", output.ToString( ), StringComparison.Ordinal ); + Assert.Contains( "Version consistency OK:", output.ToString( ), StringComparison.Ordinal ); + } + + [Fact] + public async Task AnIsolatedRenderTestMustReportExactlyOnePassingTest( ) + { + var runner = new LintStepRunner( "test result: ok. 0 passed; 0 failed;\n" ); + var context = new ToolContext( TestRepository.Root, TextWriter.Null, TextWriter.Null, runner, CancellationToken.None ); + + var step = new LintStep( LintStepKind.IsolatedRenderTest, ["test", "--lib", "probe"], "probe (--all-features)" ); + + var error = await Assert.ThrowsAsync( ( ) => DevelopmentCommands.RunLintStep( context, step ) ); + + Assert.Contains( "Expected exactly one passing isolated render test: probe (--all-features)", error.Message, + StringComparison.Ordinal ); + } + + [Fact] + public async Task LocalGateFailsBeforeAnyCheckWhenPinnedSdkIsUnavailable( ) + { + if ( !OperatingSystem.IsLinux( ) ) + { + return; + } + + using var fixture = CreateLocalGateFixture( dotnetSdkAvailable: false ); + var commandLog = Path.Combine( fixture.Path, "commands.log" ); + + var result = await RunLocalGate( fixture.Path, commandLog, new HashSet { ExitCodes.Failure } ); + + Assert.Equal( ExitCodes.Failure, result.ExitCode ); + Assert.Contains( "the complete gate needs the .NET SDK selected by global.json", result.StandardError, StringComparison.Ordinal ); + Assert.DoesNotContain( "Running:", result.StandardOutput, StringComparison.Ordinal ); + Assert.False( File.Exists( commandLog ) ); } private static ToolContext CreateContext( IProcessRunner runner, Func? environment = null ) => - new( FindRepository( ), TextWriter.Null, TextWriter.Null, runner, CancellationToken.None, environment ); + new( TestRepository.Root, TextWriter.Null, TextWriter.Null, runner, CancellationToken.None, environment ); private static TemporaryDirectory CreateArchiveFixture( ) { @@ -300,34 +354,39 @@ private static async Task RunProcess( string directory, string fileName, IReadOn } [SupportedOSPlatform( "linux" )] - private static TemporaryDirectory CreateStandaloneGateFixture( bool dotnetSdkAvailable ) + private static TemporaryDirectory CreateLocalGateFixture( bool dotnetSdkAvailable, + string cargoOutput = "test result: ok. 1 passed; 0 failed;" ) { - var fixture = new TemporaryDirectory( "wayscriber-standalone-gate-test" ); + var fixture = new TemporaryDirectory( "wayscriber-local-gate-test" ); var tools = Path.Combine( fixture.Path, "tools" ); var fakeBin = Path.Combine( fixture.Path, "fake-bin" ); Directory.CreateDirectory( tools ); Directory.CreateDirectory( fakeBin ); - File.Copy( Path.Combine( FindRepository( ), "tools/lint-and-test.sh" ), Path.Combine( tools, "lint-and-test.sh" ) ); - File.Copy( Path.Combine( FindRepository( ), "global.json" ), Path.Combine( fixture.Path, "global.json" ) ); - File.CreateSymbolicLink( Path.Combine( fakeBin, "bash" ), "/usr/bin/true" ); - WriteExecutable( Path.Combine( fakeBin, "cargo" ), "#!/usr/bin/bash\nprintf 'test result: ok. 1 passed; 0 failed;\\n'\n" ); - WriteExecutable( Path.Combine( fakeBin, "dotnet" ), $$""" + File.Copy( Path.Combine( TestRepository.Root, "tools/lint-and-test.sh" ), Path.Combine( tools, "lint-and-test.sh" ) ); + File.Copy( Path.Combine( TestRepository.Root, "global.json" ), Path.Combine( fixture.Path, "global.json" ) ); + // Only `tools/lint-and-test.sh` is copied, so any other script the gate runs fails it. + WriteExecutable( Path.Combine( fakeBin, Programs.Cargo ), $$""" +#!/usr/bin/bash +{{LogInvocation( Programs.Cargo )}} +printf '%s\n' '{{cargoOutput}}' +""" ); + WriteExecutable( Path.Combine( fakeBin, Programs.Dotnet ), $$""" #!/usr/bin/bash if [[ "$1" == "--version" ]]; then {{(dotnetSdkAvailable ? $"printf '%s\\n' '{TestDotnetSdk}'" : "exit 1")}} else - printf '%s\n' "$*" >> "${WAYSCRIBER_DOTNET_LOG:?}" + {{LogInvocation( Programs.Dotnet )}} fi """ ); - foreach ( var check in new[] { "check-nixpkgs-recipe.py", "check-rust-source-coverage.py", "check-process-sites.py", - "check-config-writers.py", "check-shared-dependencies.py", "test-shared-dependencies.py" } ) - { - WriteExecutable( Path.Combine( tools, check ), "#!/usr/bin/true\n" ); - } return fixture; } + // Logs one line per invocation: the program, then each argument as `printf %q` quotes + // it, so an argument the gate splits or joins differently changes the line. + private static string LogInvocation( string program ) => + $"printf '%s%s\\n' '{program}' \"$(printf ' %q' \"$@\")\" >> \"${{{CommandLogVariable}:?}}\""; + [SupportedOSPlatform( "linux" )] private static void WriteExecutable( string path, string content ) { @@ -335,23 +394,30 @@ private static void WriteExecutable( string path, string content ) File.SetUnixFileMode( path, UnixFileMode.UserRead | UnixFileMode.UserWrite | UnixFileMode.UserExecute ); } - private static Task RunStandaloneGate( string repository, string dotnetLog, IReadOnlySet allowedExitCodes ) + private static Task RunLocalGate( string repository, string commandLog, IReadOnlySet allowedExitCodes ) { var environment = new Dictionary { [EnvironmentVariables.Path] = Path.Combine( repository, "fake-bin" ) + Path.PathSeparator + Environment.GetEnvironmentVariable( EnvironmentVariables.Path ), - [DotnetLogVariable] = dotnetLog, + [CommandLogVariable] = commandLog, }; var request = new ProcessRequest( "/usr/bin/bash", [Path.Combine( repository, "tools/lint-and-test.sh" )], repository, environment, CaptureOutput: true, Trace: false, AllowedExitCodes: allowedExitCodes ); return new ProcessRunner( TextWriter.Null, TextWriter.Null ).RunAsync( request, CancellationToken.None ); } - private static string FindRepository( ) + // Records each launch and answers it with a fixed standard output. + private sealed class LintStepRunner( string output ) : IProcessRunner { - var directory = AppContext.GetData( "EntryPointFileDirectoryPath" ) as string ?? Environment.CurrentDirectory; - return Path.GetFullPath( Path.Combine( directory, ".." ) ); + public List Requests { get; } = []; + + public Task RunAsync( ProcessRequest request, CancellationToken cancellationToken ) + { + Requests.Add( request ); + + return Task.FromResult( new ProcessResult( ExitCodes.Success, output, string.Empty ) ); + } } private sealed class AskpassAurRunner : IProcessRunner diff --git a/tools/csharp-tests/ReleaseRegressionTests.cs b/tools/csharp-tests/ReleaseRegressionTests.cs index 2ab0265dd..cea37c266 100644 --- a/tools/csharp-tests/ReleaseRegressionTests.cs +++ b/tools/csharp-tests/ReleaseRegressionTests.cs @@ -48,7 +48,7 @@ public void InstallerManifestRejectsUnsupportedLinesAmongValidEntries( ) [InlineData( true )] public void IncompleteDesktopAssetBlockIsNormalizedWithoutRemovingPackagePayload( bool omitLastAsset ) { - var recipe = AssetsCommand.CreateRecipe( FindRepository( ) )["configurator"]!.AsObject( ); + var recipe = AssetsCommand.CreateRecipe( TestRepository.Root )["configurator"]!.AsObject( ); var marker = recipe["marker"]!.GetValue( ); var endMarker = recipe["end_marker"]!.GetValue( ); var anchor = recipe["anchor"]!.GetValue( ); @@ -68,7 +68,7 @@ public void IncompleteDesktopAssetBlockIsNormalizedWithoutRemovingPackagePayload [Fact] public void IncompleteWayscriberAssetBlockPreservesTheConfiguratorBlock( ) { - var recipes = AssetsCommand.CreateRecipe( FindRepository( ) ); + var recipes = AssetsCommand.CreateRecipe( TestRepository.Root ); var sourceRecipe = recipes[PackageChannels.Source]!.AsObject( ); var configuratorRecipe = recipes[PackageChannels.Configurator]!.AsObject( ); var sourceAnchor = sourceRecipe["anchor"]!.GetValue( ); @@ -288,10 +288,9 @@ public void DesktopExecPathPreservesBothEscapeLayers( ) [Theory] [InlineData( "tools/test-package-repo-layout.sh" )] [InlineData( "tools/test-release-packaging.sh" )] - [InlineData( "tools/test-aur-desktop-assets.sh" )] public async Task StandaloneReleaseContractsPassInTheCanonicalTestApp( string relativePath ) { - var root = FindRepository( ); + var root = TestRepository.Root; var runner = new ProcessRunner( TextWriter.Null, TextWriter.Null ); var result = await runner.RunAsync( @@ -302,7 +301,7 @@ public async Task StandaloneReleaseContractsPassInTheCanonicalTestApp( string re } private static ToolContext CreateContext( IProcessRunner runner, Func? environment = null ) => - new( FindRepository( ), TextWriter.Null, TextWriter.Null, runner, CancellationToken.None, environment ); + new( TestRepository.Root, TextWriter.Null, TextWriter.Null, runner, CancellationToken.None, environment ); private static ToolCommand AurUpdateCommand( ) => ReleaseAurCommands.Commands.Single( item => item.Area == "aur" && item.Name == "update" ); @@ -378,12 +377,6 @@ cd wayscriber """ ); } - private static string FindRepository( ) - { - var directory = AppContext.GetData( "EntryPointFileDirectoryPath" ) as string ?? Environment.CurrentDirectory; - return Path.GetFullPath( Path.Combine( directory, ".." ) ); - } - private sealed class RejectingRunner : IProcessRunner { public List Requests { get; } = []; diff --git a/tools/csharp-tests/RepositoryContractTests.cs b/tools/csharp-tests/RepositoryContractTests.cs index d6395f950..4b5c8374a 100644 --- a/tools/csharp-tests/RepositoryContractTests.cs +++ b/tools/csharp-tests/RepositoryContractTests.cs @@ -6,39 +6,10 @@ namespace Wayscriber.Tools.Tests; public sealed class RepositoryContractTests { - [Fact] - public async Task SharedDependencyCommandsEnforceTheSharedSyntaxCorpus( ) - { - var root = FindRepository( ); - using var corpus = JsonDocument.Parse( File.ReadAllText( Path.Combine( root, "tools/shared-dependency-fixtures.json" ) ) ); - var command = ChecksCommand.Commands.Single( command => command.Name == CommandNames.SharedDependencies ); - foreach ( var fixture in corpus.RootElement.EnumerateArray( ) ) - { - using var directory = new TemporaryDirectory( "wayscriber-shared-dependency-test" ); - Directory.CreateDirectory( Path.Combine( directory.Path, "src/domain" ) ); - Directory.CreateDirectory( Path.Combine( directory.Path, "src/config/validate" ) ); - var sourcePath = Path.Combine( directory.Path, fixture.GetProperty( "path" ).GetString( )! ); - Directory.CreateDirectory( Path.GetDirectoryName( sourcePath )! ); - File.WriteAllText( sourcePath, fixture.GetProperty( "source" ).GetString( ) ); - var context = new ToolContext( directory.Path, TextWriter.Null, TextWriter.Null, - new ProcessRunner( TextWriter.Null, TextWriter.Null ), CancellationToken.None ); - - var error = await Record.ExceptionAsync( ( ) => command.Handler( context, [] ) ); - Assert.True( (error is ToolException) == fixture.GetProperty( "reject" ).GetBoolean( ), - $"C# fixture {fixture.GetProperty( "name" ).GetString( )}: {error}" ); - Assert.True( error is null or ToolException ); - } - - var request = new ProcessRequest( "python3", [Path.Combine( root, "tools/test-shared-dependencies.py" )], root, - CaptureOutput: true, Trace: false ); - var result = await new ProcessRunner( TextWriter.Null, TextWriter.Null ).RunAsync( request, CancellationToken.None ); - Assert.Equal( ExitCodes.Success, result.ExitCode ); - } - [Fact] public void StandaloneInstallersRemainAvailableWithoutDotnet( ) { - var root = FindRepository( ); + var root = TestRepository.Root; Assert.True( File.Exists( Path.Combine( root, "tools/install.sh" ) ) ); Assert.True( File.Exists( Path.Combine( root, "tools/install-configurator.sh" ) ) ); Assert.DoesNotContain( "dotnet", File.ReadAllText( Path.Combine( root, "tools/install.sh" ) ), StringComparison.OrdinalIgnoreCase ); @@ -48,7 +19,7 @@ public void StandaloneInstallersRemainAvailableWithoutDotnet( ) [Fact] public void StandaloneDevelopmentRunnerUsesTheBuiltBinary( ) { - var source = File.ReadAllText( Path.Combine( FindRepository( ), "tools/run.sh" ) ); + var source = File.ReadAllText( Path.Combine( TestRepository.Root, "tools/run.sh" ) ); Assert.Contains( "target/release/wayscriber", source, StringComparison.Ordinal ); Assert.Contains( "--daemon", source, StringComparison.Ordinal ); @@ -59,7 +30,7 @@ public void StandaloneDevelopmentRunnerUsesTheBuiltBinary( ) [Fact] public void CsharpInstallShortcutRoutesToTheSharedAppInstaller( ) { - var source = File.ReadAllText( Path.Combine( FindRepository( ), "tools/install.cs" ) ); + var source = File.ReadAllText( Path.Combine( TestRepository.Root, "tools/install.cs" ) ); Assert.StartsWith( "#!/usr/bin/env -S dotnet run --disable-build-servers --file\n", source, StringComparison.Ordinal ); Assert.Contains( "#:include csharp/includes.cs", source, StringComparison.Ordinal ); @@ -68,47 +39,40 @@ public void CsharpInstallShortcutRoutesToTheSharedAppInstaller( ) Assert.DoesNotContain( "install.sh", source, StringComparison.Ordinal ); } - [Fact] - public void StandaloneShellToolsDoNotRedirectToDotnet( ) - { - var root = Path.Combine( FindRepository( ), "tools" ); - foreach ( var path in Directory.EnumerateFiles( root, "*.sh", SearchOption.TopDirectoryOnly ) ) - { - var source = File.ReadAllText( path ); - Assert.DoesNotContain( "dotnet run tools/wayscriber.cs", source, StringComparison.OrdinalIgnoreCase ); - Assert.DoesNotMatch( @"(?m)^\s*(?:exec\s+)?dotnet\b", source ); - } - } - [Fact] public void WorkflowRunStepsUseTheCsharpEntryPoint( ) { foreach ( var path in new[] { ".github/workflows/ci.yml", ".github/workflows/build-packages.yml" } ) { - var text = File.ReadAllText( Path.Combine( FindRepository( ), path ) ); + var text = File.ReadAllText( Path.Combine( TestRepository.Root, path ) ); var commands = Regex.Matches( text, @"(?m)^\s+run:\s*(.+)$" ).Select( match => match.Groups[1].Value.Trim( ) ).ToArray( ); Assert.NotEmpty( commands ); Assert.All( commands, command => Assert.StartsWith( "dotnet ", command ) ); Assert.DoesNotContain( "bash", text, StringComparison.Ordinal ); - Assert.DoesNotContain( "python", text, StringComparison.Ordinal ); } } [Fact] public void CanonicalCsharpGateEnforcesCsharpFormatting( ) { - var source = File.ReadAllText( Path.Combine( FindRepository( ), "tools/csharp/Commands/DevelopmentCommands.cs" ) ); + var formatSteps = DevelopmentCommands.LintAndTestPlan + .Where( step => step.Kind == LintStepKind.Dotnet && step.Arguments[0] == "format" ) + .Select( step => string.Join( ' ', step.Arguments ) ) + .ToArray( ); - Assert.Contains( "CsharpFileApps", source, StringComparison.Ordinal ); - Assert.Contains( "CsharpFormatModes", source, StringComparison.Ordinal ); - Assert.Contains( "RunCsharpFormattingChecks( context )", source, StringComparison.Ordinal ); - Assert.Contains( "CommandLineOptions.VerifyNoChanges", source, StringComparison.Ordinal ); + foreach ( var app in new[] { "tools/wayscriber.cs", "tools/install.cs", "tools/wayscriber.tests.cs" } ) + { + foreach ( var mode in new[] { "style", "whitespace" } ) + { + Assert.Contains( $"format {mode} {app} --no-restore --verify-no-changes", formatSteps ); + } + } } [Fact] public void CancellationDiagnosticDoesNotReuseTheCanceledToken( ) { - var source = File.ReadAllText( Path.Combine( FindRepository( ), "tools/csharp/Application/ToolApplication.cs" ) ); + var source = File.ReadAllText( Path.Combine( TestRepository.Root, "tools/csharp/Application/ToolApplication.cs" ) ); Assert.Contains( "WriteLineAsync( ToolMessages.Canceled );", source, StringComparison.Ordinal ); Assert.DoesNotContain( "WriteLineAsync( ToolMessages.Canceled, cancellation.Token )", source, StringComparison.Ordinal ); @@ -117,7 +81,7 @@ public void CancellationDiagnosticDoesNotReuseTheCanceledToken( ) [Fact] public void ToolModulesAreExplicitlyIncluded( ) { - var root = FindRepository( ); + var root = TestRepository.Root; var csharpRoot = Path.Combine( root, "tools/csharp" ); var actual = Directory.EnumerateFiles( csharpRoot, "*.cs", SearchOption.AllDirectories ) .Where( path => Path.GetFileName( path ) != "includes.cs" ) @@ -131,7 +95,7 @@ public void ToolModulesAreExplicitlyIncluded( ) [Fact] public void ToolSdkIsPinnedInGlobalJson( ) { - var root = FindRepository( ); + var root = TestRepository.Root; using var document = JsonDocument.Parse( File.ReadAllText( Path.Combine( root, "global.json" ) ) ); var sdk = document.RootElement.GetProperty( "sdk" ); Assert.Equal( VersionCommands.ToolSdkVersion, sdk.GetProperty( "version" ).GetString( ) ); @@ -140,20 +104,53 @@ public void ToolSdkIsPinnedInGlobalJson( ) } [Fact] - public void CsharpCommandsNeverLaunchAShellOrPythonInterpreter( ) + public async Task LegacyToolsCheckPassesOnTheRepository( ) + { + using var output = new StringWriter( ); + + Assert.Equal( ExitCodes.Success, await RunLegacyTools( TestRepository.Root, output ) ); + Assert.Contains( "C# does not launch a shell", output.ToString( ), StringComparison.Ordinal ); + } + + [Theory] + [InlineData( "csharp/Commands/Probe.cs", "await context.Run( \"bash\", [\"-c\", \"true\"] );", + "C# automation launches a shell: tools/csharp/Commands/Probe.cs" )] + [InlineData( "csharp/Infrastructure/Shells.cs", "public const string Bash = \"/usr/bin/bash\";", + "C# automation launches a shell: tools/csharp/Infrastructure/Shells.cs" )] + [InlineData( "extra.sh", "#!/usr/bin/env bash\n", "Shell tool inventory is incomplete:\n- extra.sh" )] + [InlineData( "build.sh", "#!/usr/bin/env bash\nexec dotnet run tools/wayscriber.cs -- dev build\n", + "Shell tool redirects to .NET: tools/build.sh" )] + public async Task LegacyToolsCheckRejectsShellLaunchesAndUnlistedOrRedirectingScripts( string tool, string source, string expected ) { - var root = Path.Combine( FindRepository( ), "tools/csharp" ); - foreach ( var path in Directory.EnumerateFiles( root, "*.cs", SearchOption.AllDirectories ) ) + using var fixture = new TemporaryDirectory( "wayscriber-legacy-tools-test" ); + var tools = Path.Combine( fixture.Path, "tools" ); + Directory.CreateDirectory( Path.Combine( tools, "csharp" ) ); + foreach ( var script in Directory.EnumerateFiles( Path.Combine( TestRepository.Root, "tools" ), "*.sh" ) ) { - var source = File.ReadAllText( path ); - Assert.DoesNotMatch( "(?:context\\.Run|ProcessRequest)\\(\\s*\"(?:(?:ba|z)?sh|python(?:3(?:\\.\\d+)?)?)\"", source ); + File.Copy( script, Path.Combine( tools, Path.GetFileName( script ) ) ); } + var path = Path.Combine( tools, tool ); + Directory.CreateDirectory( Path.GetDirectoryName( path )! ); + File.WriteAllText( path, source ); + + var error = await Assert.ThrowsAsync( ( ) => RunLegacyTools( fixture.Path, TextWriter.Null ) ); + + Assert.Contains( expected, error.Message, StringComparison.Ordinal ); + } + + private static Task RunLegacyTools( string root, TextWriter output ) + { + var command = ChecksCommand.Commands.Single( command => command.Name == CommandNames.LegacyTools ); + var context = new ToolContext( root, output, TextWriter.Null, new ProcessRunner( TextWriter.Null, TextWriter.Null ), + CancellationToken.None ); + + return command.Handler( context, [] ); } [Fact] public void CsharpProcessAndEnvironmentContractsUseNamedConstants( ) { - var root = Path.Combine( FindRepository( ), "tools/csharp" ); + var root = Path.Combine( TestRepository.Root, "tools/csharp" ); foreach ( var path in Directory.EnumerateFiles( root, "*.cs", SearchOption.AllDirectories ) .Where( path => Path.GetFileName( path ) != "ToolConstants.cs" ) ) { @@ -170,7 +167,8 @@ public async Task ProcessRunnerPreservesArgumentBoundaries( ) var output = new StringWriter( ); var error = new StringWriter( ); var result = await new ProcessRunner( output, error ).RunAsync( - new ProcessRequest( "/usr/bin/printf", ["%s", "two words;$(ignored)"], FindRepository( ), CaptureOutput: true ), CancellationToken.None ); + new ProcessRequest( "/usr/bin/printf", ["%s", "two words;$(ignored)"], TestRepository.Root, CaptureOutput: true ), + CancellationToken.None ); Assert.Equal( "two words;$(ignored)", result.StandardOutput ); } @@ -178,7 +176,9 @@ public async Task ProcessRunnerPreservesArgumentBoundaries( ) public async Task ProcessRunnerAllowsDeclaredNonzeroExit( ) { var runner = new ProcessRunner( TextWriter.Null, TextWriter.Null ); - var result = await runner.RunAsync( new ProcessRequest( "/usr/bin/false", [], FindRepository( ), AllowedExitCodes: new HashSet { ExitCodes.Failure } ), CancellationToken.None ); + var request = new ProcessRequest( "/usr/bin/false", [], TestRepository.Root, + AllowedExitCodes: new HashSet { ExitCodes.Failure } ); + var result = await runner.RunAsync( request, CancellationToken.None ); Assert.Equal( ExitCodes.Failure, result.ExitCode ); } @@ -187,7 +187,8 @@ public async Task ConcurrentProcessRunsDoNotShareALock( ) { var runner = new ProcessRunner( TextWriter.Null, TextWriter.Null ); var runs = Enumerable.Range( 0, 12 ).Select( value => runner.RunAsync( - new ProcessRequest( "/usr/bin/printf", ["%s", value.ToString( )], FindRepository( ), CaptureOutput: true ), CancellationToken.None ) ); + new ProcessRequest( "/usr/bin/printf", ["%s", value.ToString( )], TestRepository.Root, CaptureOutput: true ), + CancellationToken.None ) ); var results = await Task.WhenAll( runs ); Assert.Equal( Enumerable.Range( 0, 12 ).Select( value => value.ToString( ) ).Order( ), results.Select( result => result.StandardOutput ).Order( ) ); } @@ -195,18 +196,54 @@ public async Task ConcurrentProcessRunsDoNotShareALock( ) [Fact] public void AssetRecipesAreDeterministicUnderConcurrency( ) { - var values = Enumerable.Range( 0, 16 ).AsParallel( ).Select( _ => AssetsCommand.CreateRecipe( FindRepository( ) ).ToJsonString( ) ).ToArray( ); + var values = Enumerable.Range( 0, 16 ).AsParallel( ) + .Select( _ => AssetsCommand.CreateRecipe( TestRepository.Root ).ToJsonString( ) ) + .ToArray( ); Assert.Single( values.Distinct( StringComparer.Ordinal ) ); } - [Fact] - public async Task StandaloneAndCsharpAssetRecipesHaveExactParity( ) + [Theory] + [InlineData( "0644" )] + [InlineData( "420" )] + [InlineData( "0o644" )] + public void EquivalentDesktopAssetModesProduceTheSameRecipe( string mode ) { - var root = FindRepository( ); - var runner = new ProcessRunner( TextWriter.Null, TextWriter.Null ); - var result = await runner.RunAsync( new ProcessRequest( "/usr/bin/bash", [Path.Combine( root, "tools/aur-desktop-assets.sh" ), root], - root, CaptureOutput: true, Trace: false ), CancellationToken.None ); - Assert.Equal( AssetsCommand.CreateRecipe( root ).ToJsonString( ), result.StandardOutput.Trim( ) ); + using var reference = CreateAssetModeFixture( "0644" ); + using var fixture = CreateAssetModeFixture( mode ); + + Assert.Equal( AssetsCommand.CreateRecipe( reference.Path ).ToJsonString( ), + AssetsCommand.CreateRecipe( fixture.Path ).ToJsonString( ) ); + } + + [Theory] + [InlineData( "nonsense" )] + [InlineData( "0oBAD" )] + [InlineData( "999999999999999999999999" )] + [InlineData( "\"0644\"" )] + [InlineData( "!!str 0644" )] + [InlineData( "644" )] + public void DesktopAssetModesOtherThanOctal644AreRejected( string mode ) + { + using var fixture = CreateAssetModeFixture( mode ); + + var error = Assert.Throws( ( ) => AssetsCommand.CreateRecipe( fixture.Path ) ); + + Assert.Contains( "package.wayscriber.yaml: desktop asset /usr/share/applications/wayscriber.desktop must have mode 0644", + error.Message, StringComparison.Ordinal ); + } + + [Theory] + [InlineData( "contents:\n - src: packaging/wayscriber.desktop\n dst: /usr/share/applications/wayscriber.desktop\n", + "package.wayscriber.yaml: desktop asset /usr/share/applications/wayscriber.desktop has no file_info mapping" )] + [InlineData( "{}\n", "package.wayscriber.yaml: expected one package contents sequence" )] + public void MalformedDesktopAssetManifestsAreRejected( string manifest, string expected ) + { + using var fixture = CreateAssetModeFixture( "0644" ); + File.WriteAllText( Path.Combine( fixture.Path, "packaging/package.wayscriber.yaml" ), manifest ); + + var error = Assert.Throws( ( ) => AssetsCommand.CreateRecipe( fixture.Path ) ); + + Assert.Contains( expected, error.Message, StringComparison.Ordinal ); } [Fact] @@ -251,7 +288,7 @@ public async Task AurSourceUpdateDoesNotNeedMakepkgAndIsIdempotent( ) var manifest = Path.Combine( fixture.Path, "manifest.json" ); File.WriteAllText( manifest, """{"version":"9.9.9","artifacts":[]}""" ); var runner = new RecordingAurRunner( ); - var context = new ToolContext( FindRepository( ), TextWriter.Null, TextWriter.Null, runner, CancellationToken.None ); + var context = new ToolContext( TestRepository.Root, TextWriter.Null, TextWriter.Null, runner, CancellationToken.None ); var command = ReleaseAurCommands.Commands.Single( item => item.Area == "aur" && item.Name == "update" ); var arguments = new[] { "--manifest", manifest, "--source-dir", source, "--bin-dir", Path.Combine( fixture.Path, "missing" ), "--no-configurator", "--source-sha256", new string( 'a', HashingConstants.Sha256HexLength ) }; @@ -288,7 +325,7 @@ public async Task RepositoryBuildReplacesCompleteOutputFromAStagingDirectory( ) File.WriteAllText( Path.Combine( artifacts, name ), name ); } var runner = new RepositoryLayoutRunner( ); - var context = new ToolContext( FindRepository( ), TextWriter.Null, TextWriter.Null, runner, CancellationToken.None ); + var context = new ToolContext( TestRepository.Root, TextWriter.Null, TextWriter.Null, runner, CancellationToken.None ); var command = PackagingCommands.Commands.Single( item => item.Area == "package" && item.Name == "build-repositories" ); Assert.Equal( ExitCodes.Success, await command.Handler( context, ["--artifact-root", artifacts, "--output-root", output] ) ); @@ -323,17 +360,17 @@ public void InstallerManifestRejectsUnsafeEntries( string entry ) [Fact] public void VersionCheckAcceptsPackagingHotfixOfCurrentCargoVersion( ) { - var current = ReleaseVersion.Parse( VersionCommands.ReadCargoVersion( Path.Combine( FindRepository( ), "Cargo.toml" ) ) ); - var packageVersion = File.ReadAllText( Path.Combine( FindRepository( ), "packaging/PKGBUILD" ) ); + var current = ReleaseVersion.Parse( VersionCommands.ReadCargoVersion( Path.Combine( TestRepository.Root, "Cargo.toml" ) ) ); + var packageVersion = File.ReadAllText( Path.Combine( TestRepository.Root, "packaging/PKGBUILD" ) ); var currentPackage = Regex.Match( packageVersion, @"(?m)^pkgver=(.+)$" ).Groups[1].Value; var release = currentPackage.StartsWith( current.CargoVersion + ".", StringComparison.Ordinal ) ? currentPackage : current.CargoVersion; - Assert.Empty( VersionCommands.Validate( FindRepository( ), release ) ); + Assert.Empty( VersionCommands.Validate( TestRepository.Root, release ) ); } [Fact] public void VersionCheckRejectsUnrelatedReleaseVersion( ) { - var errors = VersionCommands.Validate( FindRepository( ), "99.98.97" ); + var errors = VersionCommands.Validate( TestRepository.Root, "99.98.97" ); Assert.Contains( errors, error => error.Contains( "must equal Cargo version", StringComparison.Ordinal ) ); } @@ -364,15 +401,43 @@ private static TemporaryDirectory AssetFixture( ) Directory.CreateDirectory( Path.Combine( fixture.Path, "packaging" ) ); foreach ( var name in new[] { "package.wayscriber.yaml", "package.configurator.yaml" } ) { - File.Copy( Path.Combine( FindRepository( ), "packaging", name ), Path.Combine( fixture.Path, "packaging", name ) ); + File.Copy( Path.Combine( TestRepository.Root, "packaging", name ), Path.Combine( fixture.Path, "packaging", name ) ); } return fixture; } - private static string FindRepository( ) + /// The repository's packaging, with the launcher's mode written as given. + private static TemporaryDirectory CreateAssetModeFixture( string mode ) + { + var fixture = new TemporaryDirectory( "wayscriber-asset-mode-test" ); + CopyDirectory( Path.Combine( TestRepository.Root, "packaging" ), Path.Combine( fixture.Path, "packaging" ) ); + File.WriteAllText( Path.Combine( fixture.Path, "packaging/package.wayscriber.yaml" ), $""" +contents: + - src: packaging/wayscriber.desktop + dst: /usr/share/applications/wayscriber.desktop + file_info: + mode: {mode} + - src: packaging/icons/wayscriber.svg + dst: /usr/share/icons/hicolor/scalable/apps/wayscriber.svg + file_info: + mode: 0644 + +""" ); + + return fixture; + } + + private static void CopyDirectory( string source, string destination ) { - var directory = AppContext.GetData( "EntryPointFileDirectoryPath" ) as string ?? Environment.CurrentDirectory; - return Path.GetFullPath( Path.Combine( directory, ".." ) ); + Directory.CreateDirectory( destination ); + foreach ( var file in Directory.EnumerateFiles( source ) ) + { + File.Copy( file, Path.Combine( destination, Path.GetFileName( file ) ) ); + } + foreach ( var directory in Directory.EnumerateDirectories( source ) ) + { + CopyDirectory( directory, Path.Combine( destination, Path.GetFileName( directory ) ) ); + } } private sealed class RecordingAurRunner : IProcessRunner diff --git a/tools/csharp-tests/SignedRepositoryIntegrationTests.cs b/tools/csharp-tests/SignedRepositoryIntegrationTests.cs index 1459eb15e..33bee4a53 100644 --- a/tools/csharp-tests/SignedRepositoryIntegrationTests.cs +++ b/tools/csharp-tests/SignedRepositoryIntegrationTests.cs @@ -32,7 +32,7 @@ public async Task RepositoryBuildProducesAVerifiableSignedRpm( ) [EnvironmentVariables.GpgPassphrase] = KeyPassphrase, [EnvironmentVariables.SignRpms] = "1", }; - var context = new ToolContext( FindRepository( ), TextWriter.Null, TextWriter.Null, + var context = new ToolContext( TestRepository.Root, TextWriter.Null, TextWriter.Null, new ProcessRunner( TextWriter.Null, TextWriter.Null ), CancellationToken.None, name => environment.GetValueOrDefault( name ) ); var command = PackagingCommands.Commands.Single( item => item.Area == "package" && item.Name == "build-repositories" ); @@ -121,10 +121,4 @@ private static Task Run( string workingDirectory, string fileName Trace: false, AllowedExitCodes: allowedExitCodes ); return runner.RunAsync( request, CancellationToken.None ); } - - private static string FindRepository( ) - { - var directory = AppContext.GetData( "EntryPointFileDirectoryPath" ) as string ?? Environment.CurrentDirectory; - return Path.GetFullPath( Path.Combine( directory, ".." ) ); - } } diff --git a/tools/csharp-tests/TestRepository.cs b/tools/csharp-tests/TestRepository.cs new file mode 100644 index 000000000..2b926d9be --- /dev/null +++ b/tools/csharp-tests/TestRepository.cs @@ -0,0 +1,14 @@ +namespace Wayscriber.Tools.Tests; + +// The checkout under test: the parent of the directory that holds the test entry point. +internal static class TestRepository +{ + public static string Root + { + get + { + var directory = AppContext.GetData( "EntryPointFileDirectoryPath" ) as string ?? Environment.CurrentDirectory; + return Path.GetFullPath( Path.Combine( directory, ".." ) ); + } + } +} diff --git a/tools/csharp-tests/VersionConsistencyTests.cs b/tools/csharp-tests/VersionConsistencyTests.cs new file mode 100644 index 000000000..0880b617e --- /dev/null +++ b/tools/csharp-tests/VersionConsistencyTests.cs @@ -0,0 +1,253 @@ +using Xunit; + +namespace Wayscriber.Tools.Tests; + +// Each case edits one independently editable metadata surface of a copied repository, +// so a partial floor bump, stale recipe, or pinned install example cannot pass `version check`. +public sealed class VersionConsistencyTests +{ + private const string CargoPlaceholder = "{cargo}"; + private const string FixedChecksum = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"; + private const string ShellAurUpdater = "tools/update-aur-from-manifest.sh"; + private const string ReleaseWorkflow = ".github/workflows/build-packages.yml"; + private const string PackageRunner = + " # This runner defines the oldest supported release ABI (glibc 2.39).\n runs-on: ubuntu-24.04"; + private const string StaleReadme = "goes stale on the next release; use a RELEASE_TAG placeholder or link to /releases/latest"; + + [Fact] + public async Task CopiedRepositoryMetadataPasses( ) + { + using var fixture = new VersionMetadataFixture( ); + + var output = await fixture.Check( ); + + var cargo = fixture.CargoVersion; + Assert.Equal( $"Version consistency OK: Cargo={cargo}, packaging={cargo}, checksum=SKIP, libadwaita=1.4\n", output ); + } + + [Theory] + [InlineData( """{"sdk":{"version":"11.0.100-rc.1.26425.128","rollForward":"disable","allowPrerelease":true}}""" )] + [InlineData( """{"sdk":{"version":" 11.0.100-rc.1.26425.128 ","rollForward":"disable","allowPrerelease":true}}""" )] + public async Task CompactGlobalJsonWithTheReviewedSdkPasses( string globalJson ) + { + using var fixture = new VersionMetadataFixture( ); + fixture.Write( "global.json", globalJson ); + + Assert.StartsWith( "Version consistency OK:", await fixture.Check( ), StringComparison.Ordinal ); + } + + [Theory] + [InlineData( """{"sdk":{}}""", "global.json SDK metadata is invalid" )] + [InlineData( "not json", "global.json SDK metadata is invalid" )] + [InlineData( """{"sdk":{"version":" ","rollForward":"disable","allowPrerelease":true}}""", + "global.json SDK metadata is invalid: sdk.version must be a non-empty string" )] + [InlineData( """{"sdk":{"version":11,"rollForward":"disable","allowPrerelease":true}}""", + "global.json SDK metadata is invalid: sdk.version must be a non-empty string" )] + [InlineData( """{"sdk":{"version":"11.0.100-rc.1.26425.127","rollForward":"disable","allowPrerelease":true}}""", + "global.json SDK: expected 11.0.100-rc.1.26425.128, got 11.0.100-rc.1.26425.127" )] + [InlineData( """{"sdk":{"version":"11.0.100-rc.1.26425.128","rollForward":"latestPatch","allowPrerelease":false}}""", + "global.json SDK rollForward must be disable" )] + [InlineData( """{"sdk":{"version":"11.0.100-rc.1.26425.128","rollForward":"latestPatch","allowPrerelease":false}}""", + "global.json SDK allowPrerelease must be true" )] + [InlineData( """{"sdk":{"version":"11.0.100-rc.1.26425.128","rollForward":"disable","allowPrerelease":"true"}}""", + "global.json SDK allowPrerelease must be true" )] + public async Task GlobalJsonMustPinTheReviewedSdk( string globalJson, string expected ) + { + using var fixture = new VersionMetadataFixture( ); + fixture.Write( "global.json", globalJson ); + + Assert.Contains( expected, await fixture.CheckExpectingFailure( ), StringComparison.Ordinal ); + } + + [Theory] + // One supported libadwaita floor governs compilation and every package channel. + [InlineData( "configurator/Cargo.toml", """features = ["v1_4"]""", """features = ["v1_5"]""", + "configurator/Cargo.toml libadwaita features: expected ['v1_4'], got ['v1_5']" )] + [InlineData( "configurator/Cargo.toml", """features = ["v1_4"]""", """features = ["v1_4", 4]""", + "configurator/Cargo.toml: libadwaita features must be a string list" )] + [InlineData( "configurator/Cargo.toml", """libadwaita = { version = "0.9", features = ["v1_4"] }""", "libadwaita = \"0.9\"", + "configurator/Cargo.toml: missing structured libadwaita dependency" )] + [InlineData( "configurator/Cargo.toml", """libadwaita = { version = "0.9", features = ["v1_4"] }""", + """libadwaita = { version = "0.9" }""", "configurator/Cargo.toml libadwaita features: expected ['v1_4'], got []" )] + [InlineData( "packaging/package.configurator.yaml", "libadwaita-1-0 (>= 1.4)", "libadwaita-1-0 (>= 1.5)", + "configurator deb libadwaita floor: expected 1.4, got 1.5" )] + [InlineData( "packaging/package.configurator.yaml", "libadwaita >= 1.4", "libadwaita >= 1.5", + "configurator rpm libadwaita floor: expected 1.4, got 1.5" )] + [InlineData( "packaging/PKGBUILD", "'libadwaita>=1.4'", "'libadwaita>=1.5'", + "packaging/PKGBUILD libadwaita floor: expected 1.4, got 1.5" )] + [InlineData( "packaging/.SRCINFO", "depends = libadwaita>=1.4", "depends = libadwaita>=1.5", + "packaging/.SRCINFO libadwaita floor: expected 1.4, got 1.5" )] + // The standalone AUR updater generates and validates recipes with its own literal floor. + [InlineData( ShellAurUpdater, "ensure_runtime_dependency 'libadwaita>=1.4' gcc-libs", + "ensure_runtime_dependency 'libadwaita>=1.5' gcc-libs", "AUR updater generated libadwaita floor: expected 1.4, got 1.5" )] + [InlineData( ShellAurUpdater, """'libadwaita>=1.4'[[:space:]]*$" PKGBUILD""", """'libadwaita>=1.5'[[:space:]]*$" PKGBUILD""", + "AUR updater PKGBUILD validation floor: expected 1.4, got 1.5" )] + [InlineData( ShellAurUpdater, "depends = libadwaita>=1.4' .SRCINFO", "depends = libadwaita>=1.5' .SRCINFO", + "AUR updater .SRCINFO validation floor: expected 1.4, got 1.5" )] + [InlineData( ShellAurUpdater, " ensure_runtime_dependency 'libadwaita>=1.4' gcc-libs\n", + " ensure_runtime_dependency 'libadwaita>=1.4' gcc-libs\n ensure_runtime_dependency 'libadwaita>=1.4' gcc-libs\n", + "AUR updater generated libadwaita floor: expected one libadwaita floor, found 2" )] + // The package job's runner defines the binary ABI and needs a reviewed floor contract. + [InlineData( ReleaseWorkflow, PackageRunner, " runs-on: ubuntu-26.04", + "release package runner libadwaita floor: no reviewed contract for ubuntu-26.04" )] + [InlineData( ReleaseWorkflow, PackageRunner, PackageRunner + "\n runs-on: ubuntu-24.04", + ".github/workflows/build-packages.yml package job: expected one literal runs-on value, found 2" )] + [InlineData( ReleaseWorkflow, "jobs:\n package:\n", "jobs:\n packages:\n", + ".github/workflows/build-packages.yml: missing package job" )] + // Workspace versions, lockfile, and the packaging template follow Cargo.toml. + [InlineData( "Cargo.toml", "version = \"{cargo}\"\n", "version = \"{cargo}-rc.1\"\n", + "Cargo.toml version has unsupported format: {cargo}-rc.1" )] + [InlineData( "configurator/Cargo.toml", "version = \"{cargo}\"\n", "version = \"0.0.1\"\n", + "configurator/Cargo.toml: expected {cargo}, got 0.0.1" )] + [InlineData( "Cargo.lock", "name = \"wayscriber\"\nversion = \"{cargo}\"", "name = \"wayscriber\"\nversion = \"0.0.1\"", + "Cargo.lock wayscriber: expected {cargo}, got 0.0.1" )] + [InlineData( "packaging/PKGBUILD", "pkgver={cargo}\n", "pkgver=0.0.1\n", "packaging/PKGBUILD pkgver: expected {cargo}, got 0.0.1" )] + [InlineData( "packaging/.SRCINFO", "\tpkgver = {cargo}\n", "", "packaging/.SRCINFO pkgver: expected {cargo}, got missing" )] + // Repo packaging metadata is a template; automation writes the real checksum after tagging. + [InlineData( "packaging/PKGBUILD", "sha256sums=('SKIP')", "sha256sums=('" + FixedChecksum + "')", + "packaging/PKGBUILD sha256sums: expected SKIP template checksum, got fixed SHA " + FixedChecksum + + "; release/AUR automation writes the real checksum after the tag exists" )] + [InlineData( "packaging/PKGBUILD", "sha256sums=('SKIP')", "sha256sums=('SKIP!')", + "packaging/PKGBUILD sha256sums: expected SKIP template checksum, got SKIP!" )] + [InlineData( "packaging/PKGBUILD", "sha256sums=('SKIP')", "sha256sums=('SKIP' 'SKIP')", + "packaging/PKGBUILD sha256sums: expected SKIP template checksum, got SKIP, SKIP" )] + [InlineData( "packaging/PKGBUILD", "sha256sums=('SKIP')", "", + "packaging/PKGBUILD sha256sums: expected SKIP template checksum, got missing" )] + [InlineData( "packaging/.SRCINFO", "sha256sums = SKIP", "sha256sums = " + FixedChecksum, + "packaging/.SRCINFO sha256sums: expected SKIP template checksum, got fixed SHA " + FixedChecksum )] + [InlineData( "packaging/.SRCINFO", "sha256sums = SKIP", "sha256sums = 'SKIP'", + "packaging/.SRCINFO sha256sums: expected SKIP template checksum, got 'SKIP'" )] + [InlineData( "packaging/.SRCINFO", "\tsha256sums = SKIP\n", "", + "packaging/.SRCINFO sha256sums: expected SKIP template checksum, got missing" )] + // The flake derives its version and toolchain floor from Cargo.toml. + [InlineData( "flake.nix", "builtins.readFile ./Cargo.toml", "builtins.readFile ./other.toml", + "flake.nix package version should be derived from Cargo.toml" )] + [InlineData( "flake.nix", "versionAtLeast", "versionOlder", + "flake.nix should compare the selected rustc against Cargo.toml rust-version" )] + // Install examples that pin a concrete tag are stale one release later. + [InlineData( "README.md", "wayscriber?ref=RELEASE_TAG", "wayscriber?ref=v{cargo}", + "README.md: pinned flake ref 'wayscriber?ref=v{cargo}' " + StaleReadme )] + [InlineData( "README.md", "Open the [latest release](https://github.com/devmobasa/wayscriber/releases/latest)", + "Open the [release](https://github.com/devmobasa/wayscriber/releases/tag/v{cargo})", + "README.md: pinned release URL '/releases/tag/v{cargo}' " + StaleReadme )] + [InlineData( "README.md", "wget -O wayscriber-amd64.deb https://github.com/devmobasa/wayscriber/releases/latest/download/", + "wget -O wayscriber-amd64.deb https://github.com/devmobasa/wayscriber/releases/download/{cargo}/", + "README.md: pinned release URL '/releases/download/{cargo}' " + StaleReadme )] + public async Task MetadataDriftIsRejected( string relativePath, string current, string replacement, string expected ) + { + using var fixture = new VersionMetadataFixture( ); + string WithCargo( string value ) => value.Replace( CargoPlaceholder, fixture.CargoVersion, StringComparison.Ordinal ); + fixture.Replace( relativePath, WithCargo( current ), WithCargo( replacement ) ); + + var error = await fixture.CheckExpectingFailure( ); + + Assert.Contains( WithCargo( expected ), error, StringComparison.Ordinal ); + } + + [Fact] + public async Task TableFormLibadwaitaDependencyIsReadStructurally( ) + { + using var fixture = new VersionMetadataFixture( ); + fixture.Replace( "configurator/Cargo.toml", """libadwaita = { version = "0.9", features = ["v1_4"] }""", string.Empty ); + var manifest = fixture.Read( "configurator/Cargo.toml" ); + var tableForm = "\n[dependencies.libadwaita]\nversion = \"0.9\"\nfeatures = [\n \"v1_4\", # floor\n]\n"; + fixture.Write( "configurator/Cargo.toml", manifest + tableForm ); + + Assert.StartsWith( "Version consistency OK:", await fixture.Check( ), StringComparison.Ordinal ); + + fixture.Replace( "configurator/Cargo.toml", "\"v1_4\", # floor", "\"v1_5\"" ); + + Assert.Contains( "configurator/Cargo.toml libadwaita features: expected ['v1_4'], got ['v1_5']", + await fixture.CheckExpectingFailure( ), StringComparison.Ordinal ); + } + + [Fact] + public async Task LibadwaitaOutsideRuntimeDependenciesIsRejected( ) + { + using var fixture = new VersionMetadataFixture( ); + const string dependency = """libadwaita = { version = "0.9", features = ["v1_4"] }"""; + fixture.Replace( "configurator/Cargo.toml", dependency + "\n", string.Empty ); + fixture.Replace( "configurator/Cargo.toml", "[dev-dependencies]\n", "[dev-dependencies]\n" + dependency + "\n" ); + + var error = await fixture.CheckExpectingFailure( ); + + Assert.Contains( "configurator/Cargo.toml: missing structured libadwaita dependency", error, StringComparison.Ordinal ); + } + + [Fact] + public async Task MissingGlobalJsonIsInvalidSdkMetadata( ) + { + using var fixture = new VersionMetadataFixture( ); + File.Delete( fixture.PathFor( "global.json" ) ); + + var error = await fixture.CheckExpectingFailure( ); + + Assert.Contains( "global.json SDK metadata is invalid", error, StringComparison.Ordinal ); + Assert.Contains( "global.json SDK: expected 11.0.100-rc.1.26425.128, got missing", error, StringComparison.Ordinal ); + } + + [Theory] + [InlineData( "1.2", "release version has unsupported format: 1.2" )] + [InlineData( "v{cargo}", "release version has unsupported format: v{cargo}" )] + [InlineData( "{cargo}.1.2", "release version has unsupported format: {cargo}.1.2" )] + [InlineData( "0{cargo}", "release version has unsupported format: 0{cargo}" )] + [InlineData( "99.98.97", "release version 99.98.97 must equal Cargo version {cargo} or be a hotfix of it, such as {cargo}.1" )] + [InlineData( "{cargo}0", "release version {cargo}0 must equal Cargo version {cargo} or be a hotfix of it, such as {cargo}.1" )] + [InlineData( "{cargo}.1", "packaging/PKGBUILD pkgver: expected {cargo}.1, got {cargo}" )] + [InlineData( "{cargo}.1", "packaging/.SRCINFO pkgver: expected {cargo}.1, got {cargo}" )] + public async Task ReleaseVersionMustMatchCargoAndThePackagingRecipe( string release, string expected ) + { + using var fixture = new VersionMetadataFixture( ); + string WithCargo( string value ) => value.Replace( CargoPlaceholder, fixture.CargoVersion, StringComparison.Ordinal ); + + var error = await fixture.CheckExpectingFailure( "--release-version", WithCargo( release ) ); + + Assert.Contains( WithCargo( expected ), error, StringComparison.Ordinal ); + } + + [Fact] + public async Task PackagingHotfixIsAcceptedOnlyAsTheNamedRelease( ) + { + using var fixture = new VersionMetadataFixture( ); + var cargo = fixture.CargoVersion; + fixture.SetPackagingVersion( cargo + ".1" ); + + Assert.StartsWith( $"Version consistency OK: Cargo={cargo}, packaging={cargo}.1,", + await fixture.Check( ), StringComparison.Ordinal ); + Assert.StartsWith( $"Version consistency OK: Cargo={cargo}, packaging={cargo}.1,", + await fixture.Check( "--release-version", cargo + ".1" ), StringComparison.Ordinal ); + Assert.Contains( $"packaging/PKGBUILD pkgver: expected {cargo}, got {cargo}.1", + await fixture.CheckExpectingFailure( "--release-version", cargo ), StringComparison.Ordinal ); + Assert.Contains( $"packaging/.SRCINFO pkgver: expected {cargo}.2, got {cargo}.1", + await fixture.CheckExpectingFailure( "--release-version", cargo + ".2" ), StringComparison.Ordinal ); + } + + [Fact] + public async Task HotfixOfAnotherCargoVersionIsRejected( ) + { + using var fixture = new VersionMetadataFixture( ); + fixture.SetPackagingVersion( "0.0.1.1" ); + + var error = await fixture.CheckExpectingFailure( ); + + Assert.Contains( $"packaging/PKGBUILD pkgver: expected {fixture.CargoVersion}, got 0.0.1.1", error, StringComparison.Ordinal ); + Assert.Contains( $"packaging/.SRCINFO pkgver: expected {fixture.CargoVersion}, got 0.0.1.1", error, StringComparison.Ordinal ); + } + + // Updating every metadata floor is still incomplete until the package runner + // moves to a reviewed release-platform contract that supports the new ABI. + [Fact] + public void RaisingEveryFloorStillRequiresAReviewedRunnerContract( ) + { + using var fixture = new VersionMetadataFixture( ); + fixture.Replace( "configurator/Cargo.toml", """features = ["v1_4"]""", """features = ["v1_7"]""" ); + foreach ( var path in new[] { "packaging/PKGBUILD", "packaging/.SRCINFO", "packaging/package.configurator.yaml", ShellAurUpdater } ) + { + fixture.Replace( path, "1.4", "1.7" ); + } + + var errors = VersionCommands.Validate( fixture.Root, supportedLibadwaitaFloor: "1.7" ); + + Assert.Equal( "release package runner ubuntu-24.04 libadwaita floor: expected 1.4, got 1.7", Assert.Single( errors ) ); + } +} diff --git a/tools/csharp-tests/VersionMetadataFixture.cs b/tools/csharp-tests/VersionMetadataFixture.cs new file mode 100644 index 000000000..83ab0a435 --- /dev/null +++ b/tools/csharp-tests/VersionMetadataFixture.cs @@ -0,0 +1,100 @@ +using System.Text.RegularExpressions; +using Xunit; + +namespace Wayscriber.Tools.Tests; + +// A temporary repository holding a copy of every file the version checker reads. +// The packaging recipes start at the Cargo version so mutations have a fixed baseline. +internal sealed class VersionMetadataFixture : IDisposable +{ + private static readonly string[] MetadataFiles = + [ + "Cargo.toml", "Cargo.lock", "README.md", "flake.nix", "global.json", "configurator/Cargo.toml", + "packaging/PKGBUILD", "packaging/.SRCINFO", "packaging/package.configurator.yaml", + ".github/workflows/build-packages.yml", "tools/update-aur-from-manifest.sh", + ]; + + private readonly TemporaryDirectory _directory = new( "wayscriber-version-metadata-test" ); + + public VersionMetadataFixture( ) + { + var repository = TestRepository.Root; + foreach ( var relativePath in MetadataFiles ) + { + var destination = PathFor( relativePath ); + Directory.CreateDirectory( Path.GetDirectoryName( destination )! ); + File.Copy( Path.Combine( [repository, .. relativePath.Split( '/' )] ), destination ); + } + + CargoVersion = VersionCommands.ReadCargoVersion( PathFor( "Cargo.toml" ) ); + SetPackagingVersion( CargoVersion ); + } + + public string Root => _directory.Path; + + public string CargoVersion + { + get; + } + + public string PathFor( string relativePath ) => Path.Combine( [Root, .. relativePath.Split( '/' )] ); + + public string Read( string relativePath ) => File.ReadAllText( PathFor( relativePath ) ); + + public void Write( string relativePath, string content ) + { + Directory.CreateDirectory( Path.GetDirectoryName( PathFor( relativePath ) )! ); + File.WriteAllText( PathFor( relativePath ), content ); + } + + // Replaces every occurrence; the current text must exist so a stale fixture cannot pass vacuously. + public void Replace( string relativePath, string current, string replacement ) + { + var text = Read( relativePath ); + Assert.Contains( current, text, StringComparison.Ordinal ); + Write( relativePath, text.Replace( current, replacement, StringComparison.Ordinal ) ); + } + + public void SetPackagingVersion( string version ) + { + Write( "packaging/PKGBUILD", Regex.Replace( Read( "packaging/PKGBUILD" ), "(?m)^pkgver=.*$", $"pkgver={version}" ) ); + Write( "packaging/.SRCINFO", Regex.Replace( Read( "packaging/.SRCINFO" ), "(?m)^\tpkgver = .*$", $"\tpkgver = {version}" ) ); + } + + public ToolContext Context( IProcessRunner runner, TextWriter? output = null ) => + new( Root, output ?? TextWriter.Null, TextWriter.Null, runner, CancellationToken.None ); + + public async Task Check( params string[] arguments ) + { + using var output = new StringWriter( ); + + var exitCode = await VersionCommand( CommandNames.Check ).Handler( Context( new NoProcessRunner( ), output ), arguments ); + + Assert.Equal( ExitCodes.Success, exitCode ); + + return output.ToString( ); + } + + public async Task CheckExpectingFailure( params string[] arguments ) + { + var error = await Assert.ThrowsAsync( ( ) => + VersionCommand( CommandNames.Check ).Handler( Context( new NoProcessRunner( ) ), arguments ) ); + + Assert.StartsWith( "Version consistency check failed:\n", error.Message, StringComparison.Ordinal ); + return error.Message; + } + + public static ToolCommand VersionCommand( string name ) => + VersionCommands.Commands.Single( command => command.Area == CommandAreas.Version && command.Name == name ); + + public void Dispose( ) => _directory.Dispose( ); + + // Metadata checks read files only; any process launch is a regression. + private sealed class NoProcessRunner : IProcessRunner + { + public Task RunAsync( ProcessRequest request, CancellationToken cancellationToken ) + { + throw new Xunit.Sdk.XunitException( $"Unexpected process: {request.FileName} {string.Join( ' ', request.Arguments )}" ); + } + } +} diff --git a/tools/csharp-tests/VersionReleaseCommandTests.cs b/tools/csharp-tests/VersionReleaseCommandTests.cs new file mode 100644 index 000000000..12c1bc706 --- /dev/null +++ b/tools/csharp-tests/VersionReleaseCommandTests.cs @@ -0,0 +1,290 @@ +using System.Text.RegularExpressions; +using Xunit; + +namespace Wayscriber.Tools.Tests; + +public sealed class VersionReleaseCommandTests +{ + private const string FixtureChecksum = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + private const string OfflineFailure = "cannot resolve locked dependencies offline"; + private static readonly string[] VersionFiles = + ["Cargo.toml", "configurator/Cargo.toml", "Cargo.lock", "packaging/PKGBUILD", "packaging/.SRCINFO"]; + + // A version bump must retain a locked registry dependency even when a newer + // compatible release exists. A local directory source makes this fully offline. + [Fact] + public async Task BumpRetainsLockedDependenciesAndKeepsHotfixesPackagingOnly( ) + { + using var fixture = await CreateBumpFixture( ); + var lockBefore = fixture.Read( "Cargo.lock" ); + var runner = new CargoWithPackagingRunner( ); + + Assert.Equal( ExitCodes.Success, await Bump( fixture, runner, "--dry-run", "1.0.1" ) ); + Assert.Equal( lockBefore, fixture.Read( "Cargo.lock" ) ); + Assert.Equal( "1.0.0", VersionCommands.ReadCargoVersion( fixture.PathFor( "Cargo.toml" ) ) ); + + foreach ( var (release, cargo) in new[] { ("1.0.1", "1.0.1"), ("1.0.1.1", "1.0.1") } ) + { + Assert.Equal( ExitCodes.Success, await Bump( fixture, runner, release ) ); + + Assert.Equal( cargo, VersionCommands.ReadCargoVersion( fixture.PathFor( "Cargo.toml" ) ) ); + Assert.Equal( cargo, VersionCommands.ReadCargoVersion( fixture.PathFor( "configurator/Cargo.toml" ) ) ); + Assert.Contains( $"\npkgver={release}\n", fixture.Read( "packaging/PKGBUILD" ), StringComparison.Ordinal ); + Assert.Contains( $"\n\tpkgver = {release}\n", fixture.Read( "packaging/.SRCINFO" ), StringComparison.Ordinal ); + // Both workspace packages change, but all remaining bytes must stay put. + var lockAfter = fixture.Read( "Cargo.lock" ).Replace( "version = \"1.0.1\"", "version = \"1.0.0\"", StringComparison.Ordinal ); + Assert.Equal( lockBefore, lockAfter ); + } + + // Prove the fixture exposes the original bug: resolving afresh selects 1.1.0. + await RunCargo( fixture, "generate-lockfile", "--offline" ); + Assert.Contains( "name = \"release-fixture\"\nversion = \"1.1.0\"", fixture.Read( "Cargo.lock" ), StringComparison.Ordinal ); + } + + [Fact] + public async Task BumpRollsBackEveryVersionFileWhenTheResultFailsTheVersionCheck( ) + { + using var fixture = await CreateBumpFixture( ); + fixture.Replace( "README.md", "wayscriber?ref=RELEASE_TAG", "wayscriber?ref=v1.0.0" ); + var before = VersionFiles.Select( fixture.Read ).ToArray( ); + var runner = new CargoWithPackagingRunner( ); + + var error = await Assert.ThrowsAsync( ( ) => Bump( fixture, runner, "1.0.1" ) ); + + Assert.StartsWith( "Version consistency check failed:", error.Message, StringComparison.Ordinal ); + Assert.Contains( "README.md: pinned flake ref 'wayscriber?ref=v1.0.0'", error.Message, StringComparison.Ordinal ); + Assert.Equal( before, VersionFiles.Select( fixture.Read ).ToArray( ) ); + } + + [Theory] + [InlineData( true )] + [InlineData( false )] + public async Task BumpStopsBeforeEditingWhenLockedDependenciesCannotResolveOffline( bool dryRun ) + { + using var fixture = await CreateBumpFixture( ); + Directory.Delete( fixture.PathFor( "vendor" ), recursive: true ); + Directory.CreateDirectory( fixture.PathFor( "vendor" ) ); + var before = VersionFiles.Select( fixture.Read ).ToArray( ); + var runner = new CargoWithPackagingRunner( ); + string[] arguments = dryRun ? ["--dry-run", "1.0.1"] : ["1.0.1"]; + + var error = await Assert.ThrowsAsync( ( ) => Bump( fixture, runner, arguments ) ); + + Assert.Contains( OfflineFailure, error.Message, StringComparison.Ordinal ); + Assert.Contains( "No version files changed.", error.Message, StringComparison.Ordinal ); + Assert.Equal( before, VersionFiles.Select( fixture.Read ).ToArray( ) ); + Assert.DoesNotContain( runner.Requests, request => request.FileName == Programs.Makepkg ); + } + + [Theory] + [InlineData( CommandNames.CreateTag )] + [InlineData( CommandNames.PublishTag )] + public async Task ReleaseTagsValidateTheReleaseVersionBeforeTouchingGit( string command ) + { + using var fixture = new VersionMetadataFixture( ); + var git = new GitRunner( ); + + var error = await Assert.ThrowsAsync( ( ) => + ReleaseCommand( command ).Handler( fixture.Context( git ), TagArguments( command, "99.98.97" ) ) ); + + Assert.Contains( $"release version 99.98.97 must equal Cargo version {fixture.CargoVersion}", error.Message, + StringComparison.Ordinal ); + Assert.Empty( git.Requests ); + } + + [Theory] + [InlineData( CommandNames.CreateTag, " M Cargo.toml\n", false, "", "Working tree is not clean" )] + [InlineData( CommandNames.PublishTag, "?? notes.txt\n", false, "", "Working tree is not clean" )] + [InlineData( CommandNames.CreateTag, "", true, "", "already exists" )] + [InlineData( CommandNames.PublishTag, "", true, "", "already exists locally" )] + [InlineData( CommandNames.PublishTag, "", false, "0000000000000000000000000000000000000000\trefs/tags/{tag}\n", + "already exists on origin" )] + public async Task ReleaseTagsRefuseADirtyTreeOrAnExistingTag( string command, string status, bool localTag, string remoteTags, + string expected ) + { + using var fixture = new VersionMetadataFixture( ); + var tag = "v" + fixture.CargoVersion; + var git = new GitRunner( status, localTag, remoteTags.Replace( "{tag}", tag, StringComparison.Ordinal ) ); + + var error = await Assert.ThrowsAsync( ( ) => + ReleaseCommand( command ).Handler( fixture.Context( git ), TagArguments( command, fixture.CargoVersion ) ) ); + + Assert.Contains( expected, error.Message, StringComparison.Ordinal ); + Assert.DoesNotContain( git.Requests, request => request.Arguments[0] is "tag" or "push" ); + } + + [Theory] + [InlineData( CommandNames.CreateTag, false, true, false )] + [InlineData( CommandNames.PublishTag, false, true, true )] + [InlineData( CommandNames.PublishTag, true, false, false )] + public async Task ReleaseTagsAcceptAPackagingHotfixRelease( string command, bool dryRun, bool tags, bool pushes ) + { + using var fixture = new VersionMetadataFixture( ); + var release = fixture.CargoVersion + ".1"; + var tag = "v" + release; + fixture.SetPackagingVersion( release ); + var git = new GitRunner( ); + using var output = new StringWriter( ); + string[] arguments = dryRun ? [.. TagArguments( command, release ), "--dry-run"] : TagArguments( command, release ); + + Assert.Equal( ExitCodes.Success, await ReleaseCommand( command ).Handler( fixture.Context( git, output ), arguments ) ); + + Assert.Equal( tags, git.Requests.Any( request => request.Arguments.SequenceEqual( ["tag", "-a", tag, "-m", $"Release {tag}"] ) ) ); + Assert.Equal( pushes, git.Requests.Any( request => request.Arguments.SequenceEqual( ["push", "origin", tag] ) ) ); + Assert.Equal( pushes, output.ToString( ).Contains( $"Pushing {tag} to origin", StringComparison.Ordinal ) ); + var announcesTag = command == CommandNames.PublishTag; + Assert.Equal( announcesTag, output.ToString( ).Contains( $"Creating annotated tag {tag}", StringComparison.Ordinal ) ); + } + + // A branch named like the tag would make the pushed tag ambiguous, so publishing + // looks the bare name up, which finds any local ref; creating a tag checks tags only. + [Theory] + [InlineData( CommandNames.PublishTag, "{tag}" )] + [InlineData( CommandNames.CreateTag, "refs/tags/{tag}" )] + public async Task ReleaseTagsLookUpTheExistingNameTheyWouldCollideWith( string command, string lookup ) + { + using var fixture = new VersionMetadataFixture( ); + var tag = "v" + fixture.CargoVersion; + var git = new GitRunner( localTag: true ); + + await Assert.ThrowsAsync( ( ) => + ReleaseCommand( command ).Handler( fixture.Context( git ), TagArguments( command, fixture.CargoVersion ) ) ); + + string[] expected = ["rev-parse", "-q", "--verify", lookup.Replace( "{tag}", tag, StringComparison.Ordinal )]; + Assert.Contains( git.Requests, request => request.Arguments.SequenceEqual( expected ) ); + } + + private static async Task CreateBumpFixture( ) + { + var fixture = new VersionMetadataFixture( ); + try + { + await WriteCargoWorkspace( fixture ); + return fixture; + } + catch + { + fixture.Dispose( ); + throw; + } + } + + // The copied release metadata stays real; only the Cargo workspace is replaced by + // a two-member workspace whose registry is a vendored directory source. + private static async Task WriteCargoWorkspace( VersionMetadataFixture fixture ) + { + fixture.Write( "Cargo.toml", """ +[package] +name = "wayscriber" +version = "1.0.0" +edition = "2024" +[workspace] +members = ["configurator"] +[dependencies] +release-fixture = "1" + +""" ); + fixture.Write( "configurator/Cargo.toml", """ +[package] +name = "wayscriber-configurator" +version = "1.0.0" +edition = "2024" +[dependencies] +wayscriber = { path = ".." } +libadwaita = { version = "0.9", features = ["v1_4"] } + +""" ); + fixture.Write( "src/lib.rs", string.Empty ); + fixture.Write( "configurator/src/lib.rs", string.Empty ); + fixture.Write( ".cargo/config.toml", $""" +[source.crates-io] +replace-with = "fixture" +[source.fixture] +directory = "{fixture.PathFor( "vendor" )}" + +""" ); + WriteVendoredCrate( fixture, "release-fixture", "1.0.0", string.Empty ); + WriteVendoredCrate( fixture, "libadwaita", "0.9.0", "[features]\nv1_4 = []\n" ); + fixture.SetPackagingVersion( "1.0.0" ); + await RunCargo( fixture, "generate-lockfile", "--offline" ); + + // Vendored only after locking, so a fresh resolution would now select it. + WriteVendoredCrate( fixture, "release-fixture", "1.1.0", string.Empty ); + } + + private static void WriteVendoredCrate( VersionMetadataFixture fixture, string name, string version, string features ) + { + var directory = $"vendor/{name}-{version}"; + var manifest = $"[package]\nname = \"{name}\"\nversion = \"{version}\"\nedition = \"2024\"\n{features}"; + fixture.Write( $"{directory}/Cargo.toml", manifest ); + fixture.Write( $"{directory}/src/lib.rs", string.Empty ); + fixture.Write( $"{directory}/.cargo-checksum.json", $$"""{"files":{},"package":"{{FixtureChecksum}}"}""" ); + } + + private static Task RunCargo( VersionMetadataFixture fixture, params string[] arguments ) => + new ProcessRunner( TextWriter.Null, TextWriter.Null ).RunAsync( + new ProcessRequest( Programs.Cargo, arguments, fixture.Root, CaptureOutput: true, Trace: false ), CancellationToken.None ); + + private static Task Bump( VersionMetadataFixture fixture, IProcessRunner runner, params string[] arguments ) => + VersionMetadataFixture.VersionCommand( CommandNames.Bump ).Handler( fixture.Context( runner ), arguments ); + + private static ToolCommand ReleaseCommand( string name ) => + ReleaseAurCommands.Commands.Single( command => command.Area == CommandAreas.Release && command.Name == name ); + + private static string[] TagArguments( string command, string version ) => + command == CommandNames.PublishTag ? [CommandLineOptions.Version, version] : [version]; + + // Runs the real Cargo against the fixture and stands in for makepkg by + // regenerating the copied .SRCINFO for the PKGBUILD's new pkgver. + private sealed class CargoWithPackagingRunner : IProcessRunner + { + private readonly ProcessRunner _processes = new( TextWriter.Null, TextWriter.Null ); + + public List Requests { get; } = []; + + public Task RunAsync( ProcessRequest request, CancellationToken cancellationToken ) + { + Requests.Add( request ); + if ( request.FileName == Programs.Cargo ) + { + return _processes.RunAsync( request with { CaptureOutput = true, Trace = false }, cancellationToken ); + } + if ( request.FileName == Programs.Makepkg && request.Arguments is ["--printsrcinfo"] ) + { + var packageBuild = File.ReadAllText( Path.Combine( request.WorkingDirectory, RepositoryNames.PackageBuildFile ) ); + var sourceInfo = File.ReadAllText( Path.Combine( request.WorkingDirectory, RepositoryNames.SourceInfoFile ) ); + var previous = Regex.Match( sourceInfo, "(?m)^\tpkgver = (.+)$" ).Groups[1].Value; + var next = Regex.Match( packageBuild, "(?m)^pkgver=(.+)$" ).Groups[1].Value; + var regenerated = sourceInfo.Replace( previous, next, StringComparison.Ordinal ); + return Task.FromResult( new ProcessResult( ExitCodes.Success, regenerated, string.Empty ) ); + } + + throw new Xunit.Sdk.XunitException( $"Unexpected process: {request.FileName} {string.Join( ' ', request.Arguments )}" ); + } + } + + private sealed class GitRunner( string status = "", bool localTag = false, string remoteTags = "" ) : IProcessRunner + { + private static readonly string[] KnownCommands = ["status", "rev-parse", "ls-remote", "tag", "push"]; + + public List Requests { get; } = []; + + public Task RunAsync( ProcessRequest request, CancellationToken cancellationToken ) + { + Requests.Add( request ); + if ( request.FileName != Programs.Git || !KnownCommands.Contains( request.Arguments[0] ) ) + { + throw new Xunit.Sdk.XunitException( $"Unexpected process: {request.FileName} {string.Join( ' ', request.Arguments )}" ); + } + + var exitCode = request.Arguments[0] == "rev-parse" && !localTag ? ExitCodes.Failure : ExitCodes.Success; + var output = request.Arguments[0] switch + { + "status" => status, + "ls-remote" => remoteTags, + _ => string.Empty, + }; + return Task.FromResult( new ProcessResult( exitCode, output, string.Empty ) ); + } + } +} diff --git a/tools/csharp-tests/includes.cs b/tools/csharp-tests/includes.cs index 360acde99..2a4f36053 100644 --- a/tools/csharp-tests/includes.cs +++ b/tools/csharp-tests/includes.cs @@ -4,3 +4,7 @@ #:include ReleaseParityRegressionTests.cs #:include ReleaseRegressionTests.cs #:include SignedRepositoryIntegrationTests.cs +#:include TestRepository.cs +#:include VersionMetadataFixture.cs +#:include VersionConsistencyTests.cs +#:include VersionReleaseCommandTests.cs diff --git a/tools/csharp/Commands/AssetsCommand.cs b/tools/csharp/Commands/AssetsCommand.cs index 0f06e0a7c..912500899 100644 --- a/tools/csharp/Commands/AssetsCommand.cs +++ b/tools/csharp/Commands/AssetsCommand.cs @@ -40,8 +40,8 @@ internal static JsonObject CreateRecipe( string root ) { try { - var main = ReadAssets( root, RepositoryNames.MainPackage, "package.wayscriber.yaml" ); - var configurator = ReadAssets( root, RepositoryNames.ConfiguratorPackage, "package.configurator.yaml" ); + var main = ReadAssets( root, RepositoryNames.MainPackage, RepositoryNames.MainPackageConfigFile ); + var configurator = ReadAssets( root, RepositoryNames.ConfiguratorPackage, RepositoryNames.ConfiguratorPackageConfigFile ); return new JsonObject { ["desktop_path_pattern"] = string.Join( "|", DesktopPrefixes.Select( Regex.Escape ) ), diff --git a/tools/csharp/Commands/ChecksCommand.cs b/tools/csharp/Commands/ChecksCommand.cs index 68228beb9..2c09fd545 100644 --- a/tools/csharp/Commands/ChecksCommand.cs +++ b/tools/csharp/Commands/ChecksCommand.cs @@ -7,16 +7,19 @@ namespace Wayscriber.Tools; internal static class ChecksCommand { private const string LibraryFilePrefix = "lib"; - private static readonly string[] StandaloneToolFiles = + // The complete local gate: it runs the C# checks, so it is the one script that needs .NET. + private const string GateEntryPoint = "lint-and-test.sh"; + // C# that hands a command to bash, sh, or zsh, by name or path: passed straight to + // ProcessRequest or context.Run, or assigned to a constant or variable that a launch uses. + private const string ShellLaunchPattern = + "(?:(?:ProcessRequest|context\\.Run)\\s*\\(\\s*|=\\s*)\"(?:/usr)?(?:/bin/)?(?:ba|z)?sh\""; + private static readonly string[] ShellToolFiles = [ - "build-package-repos.sh", "build.sh", "bump-version.sh", "check-arch-installer-manifest.sh", - "check-config-writers.py", "check-nixpkgs-recipe.py", "check-process-sites.py", - "check-rust-source-coverage.py", "check-shared-dependencies.py", "test-shared-dependencies.py", "check-version-consistency.sh", - "code-health-report.sh", "create-release-tag.sh", "fetch-all-deps.sh", "install-configurator.sh", - "install-gtk4-layer-shell.sh", "install.sh", "lint-and-test.sh", "package.sh", - "publish-release-tag.sh", "reload-daemon.sh", "run.sh", "set-portal-shortcut.sh", "test-aur-desktop-assets.sh", "test-gtk-widgets.sh", + "build-package-repos.sh", "build.sh", "check-arch-installer-manifest.sh", "fetch-all-deps.sh", + "install-configurator.sh", "install-gtk4-layer-shell.sh", "install.sh", GateEntryPoint, "package.sh", + "reload-daemon.sh", "run.sh", "set-portal-shortcut.sh", "test-gtk-widgets.sh", "test-package-repo-layout.sh", "test-release-packaging.sh", "test.sh", "update-aur-from-manifest.sh", - "update-aur.sh", "verify-static-gtk4-layer-shell.sh", "aur-desktop-assets.sh", "aur-desktop-assets.py", + "update-aur.sh", "verify-static-gtk4-layer-shell.sh", ]; public static IReadOnlyList Commands @@ -24,112 +27,12 @@ public static IReadOnlyList Commands get; } = [ - new( CommandAreas.Check, CommandNames.SharedDependencies, "Guard shared-layer Rust dependencies.", SideEffect.ReadOnly, SharedDependencies ), - new( CommandAreas.Check, CommandNames.ProcessSites, "Audit Rust process-creation ownership.", SideEffect.ReadOnly, ProcessSites ), new( CommandAreas.Check, CommandNames.RustSourceCoverage, "Verify every Rust source is compiled.", SideEffect.ReadOnly, RustSourceCoverage ), new( CommandAreas.Check, CommandNames.NixpkgsRecipe, "Check Cargo native dependencies against Nix.", SideEffect.ReadOnly, NixpkgsRecipe ), - new( CommandAreas.Check, CommandNames.ConfigWriters, "Audit config write-capability ownership.", SideEffect.ReadOnly, ConfigWriters ), - new( CommandAreas.Check, CommandNames.LegacyTools, "Verify standalone scripts remain independent fallbacks.", SideEffect.ReadOnly, LegacyTools ), + new( CommandAreas.Check, CommandNames.LegacyTools, "Verify the shell tool inventory and that C# never launches a shell.", + SideEffect.ReadOnly, LegacyTools ), ]; - private static Task SharedDependencies( ToolContext context, string[] args ) - { - new Arguments( args ).RequireEmpty( "check shared-dependencies" ); - var errors = new List( ); - foreach ( var (directory, forbidden) in new[] - { - ("src/domain", new HashSet( ["config", "input", "draw", "backend", "ui", "session"] )), - ("src/config/validate", new HashSet( ["input", "backend"] )), - } ) - { - foreach ( var path in Directory.EnumerateFiles( context.Path( directory.Split( '/' ) ), "*.rs", SearchOption.AllDirectories ) ) - { - if ( Path.GetRelativePath( context.RepositoryRoot, path ) == "src/domain/tests.rs" ) - { - continue; - } - var relative = Path.GetRelativePath( context.RepositoryRoot, path ).Replace( Path.DirectorySeparatorChar, '/' ); - if ( SharedDependencyGuard.HasUpwardPath( Files.Read( path ), relative, forbidden ) ) - { - errors.Add( $"{Path.GetRelativePath( context.RepositoryRoot, path )}: upward dependency in shared layer" ); - } - } - } - Failures( context, errors, "Shared domain and configuration-validation dependency paths passed." ); - return Task.FromResult( ExitCodes.Success ); - } - - private static Task ProcessSites( ToolContext context, string[] args ) - { - new Arguments( args ).RequireEmpty( "check process-sites" ); - var errors = new List( ); - var allow = new HashSet( StringComparer.Ordinal ) - { - "configurator/src/app/session_catalog.rs", - "configurator/src/app/daemon_setup/command.rs", - "configurator/src/app/daemon_setup/service.rs", - }; - Regex[] patterns = - [ - new( @"\b(?:std::process::)?Command::new\b" ), - new( @"\bstd::process::Child\b" ), - new( @"\blibc::(?:fork|vfork|posix_spawn|posix_spawnp|pthread_atfork)\b" ), - new( @"\blibc::SYS_(?:clone|clone3|fork|vfork)\b" ), - new( @"\b(?:sh|bash|zsh)\s+-c\b" ), - ]; - foreach ( var root in new[] { "src", "configurator/src", "tests" } ) - { - foreach ( var path in Directory.EnumerateFiles( context.Path( root.Split( '/' ) ), "*.rs", SearchOption.AllDirectories ) ) - { - var relative = Path.GetRelativePath( context.RepositoryRoot, path ).Replace( Path.DirectorySeparatorChar, '/' ); - var parts = relative.Split( '/' ); - var allowed = relative.StartsWith( "src/process_broker/", StringComparison.Ordinal ) || allow.Contains( relative ) || - parts.Contains( "tests" ) || Path.GetFileName( relative ) == "tests.rs"; - var lines = File.ReadAllLines( path ); - for ( var index = 0; index < lines.Length; index++ ) - { - var code = lines[index].Split( "//", 2 )[0]; - if ( !allowed && patterns.Any( pattern => pattern.IsMatch( code ) ) ) - { - errors.Add( $"{relative}:{index + 1}: unclassified process site: {lines[index].Trim( )}" ); - } - } - } - } - - var bootstrap = context.Path( "src", "process_broker", "bootstrap.rs" ); - var source = Files.Read( bootstrap ); - const string start = " if pid == 0 {"; - const string end = " drop(child_socket);"; - if ( !source.Contains( start, StringComparison.Ordinal ) || !source.Contains( end, StringComparison.Ordinal ) ) - { - errors.Add( "src/process_broker/bootstrap.rs: raw-clone child-stub markers changed" ); - } - else - { - var stub = source.Split( start, 2, StringSplitOptions.None )[1].Split( end, 2, StringSplitOptions.None )[0]; - foreach ( var token in new[] { "format!(", "log::", "panic!(", ".unwrap(", ".expect(", "drop(", "Command::", "CString::", "Vec::", "String::", "Box::" } ) - { - if ( stub.Contains( token, StringComparison.Ordinal ) ) - { - errors.Add( $"src/process_broker/bootstrap.rs: child stub reaches banned token '{token}'" ); - } - } - var libcCalls = Regex.Matches( stub, @"libc::([A-Za-z0-9_]+)\s*\(" ).Select( match => match.Groups[1].Value ).ToHashSet( ); - foreach ( var unexpected in libcCalls.Except( ["syscall", "_exit"] ).Order( ) ) - { - errors.Add( $"src/process_broker/bootstrap.rs: child stub reaches unapproved libc call: {unexpected}" ); - } - var syscalls = Regex.Matches( stub, @"libc::SYS_([A-Za-z0-9_]+)" ).Select( match => match.Groups[1].Value ).ToHashSet( ); - foreach ( var unexpected in syscalls.Except( ["fcntl", "dup3", "setpgid", "exit_group", "close_range", "execve"] ).Order( ) ) - { - errors.Add( $"src/process_broker/bootstrap.rs: child stub reaches unapproved syscall: {unexpected}" ); - } - } - Failures( context, errors, "process-site audit passed", "process-site audit failed:" ); - return Task.FromResult( ExitCodes.Success ); - } - private static async Task RustSourceCoverage( ToolContext context, string[] args ) { new Arguments( args ).RequireEmpty( "check rust-source-coverage" ); @@ -308,7 +211,7 @@ private static async Task NixpkgsRecipe( ToolContext context, string[] args var required = enabled.SelectMany( crate => SystemLibraries.GetValueOrDefault( crate, [] ).Select( attribute => (crate, attribute) ) ) .GroupBy( pair => pair.attribute ).ToDictionary( group => group.Key, group => group.Select( pair => pair.crate ).ToArray( ) ); var recipe = Files.Read( context.Path( RepositoryPaths.PackagingDirectory, "nixpkgs", "package.nix" ) ); - var flake = Files.Read( context.Path( "flake.nix" ) ); + var flake = Files.Read( context.Path( RepositoryNames.FlakeFile ) ); var marker = flake.IndexOf( "wayscriber = rustPlatform.buildRustPackage", StringComparison.Ordinal ); if ( marker < 0 ) { @@ -367,50 +270,47 @@ private static void RequireNixInput( List errors, string attribute, Hash } } - private static Task ConfigWriters( ToolContext context, string[] args ) - { - new Arguments( args ).RequireEmpty( "check config-writers" ); - var errors = ConfigWriterAudit.Run( context.RepositoryRoot ); - Failures( context, errors, $"config-writer audit passed ({ConfigWriterAudit.LastScannedCount} sources)", "config-writer audit failed:" ); - return Task.FromResult( ExitCodes.Success ); - } - private static Task LegacyTools( ToolContext context, string[] args ) { new Arguments( args ).RequireEmpty( "check legacy-tools" ); - var expected = StandaloneToolFiles.ToHashSet( StringComparer.Ordinal ); - var actual = Directory.EnumerateFiles( context.Path( RepositoryPaths.ToolsDirectory ), "*", SearchOption.TopDirectoryOnly ) - .Where( path => Path.GetExtension( path ) is ".sh" or ".py" ) + var expected = ShellToolFiles.ToHashSet( StringComparer.Ordinal ); + var actual = Directory.EnumerateFiles( context.Path( RepositoryPaths.ToolsDirectory ), "*.sh", SearchOption.TopDirectoryOnly ) .Select( Path.GetFileName ).ToHashSet( StringComparer.Ordinal )!; + var missing = expected.Except( actual ).Order( ).ToArray( ); if ( missing.Length > 0 ) { - throw new ToolException( "Standalone fallback scripts are missing:\n" + string.Join( '\n', missing.Select( name => $"- {name}" ) ) ); + throw new ToolException( "Shell tools are missing:\n" + string.Join( '\n', missing.Select( name => $"- {name}" ) ) ); } var unlisted = actual.Except( expected ).Order( ).ToArray( ); if ( unlisted.Length > 0 ) { - throw new ToolException( "Standalone fallback inventory is incomplete:\n" + string.Join( '\n', unlisted.Select( name => $"- {name}" ) ) ); + var names = string.Join( '\n', unlisted.Select( name => $"- {name}" ) ); + throw new ToolException( "Shell tool inventory is incomplete:\n" + names ); } - foreach ( var name in StandaloneToolFiles.Where( name => name.EndsWith( ".sh", StringComparison.Ordinal ) ) ) + // The complete local gate runs the C# checks, which exist only here; every + // other shell tool stays usable without .NET. + foreach ( var name in ShellToolFiles.Where( name => name != GateEntryPoint ) ) { var source = Files.Read( context.Path( RepositoryPaths.ToolsDirectory, name ) ); var toolRedirect = $"dotnet run {RepositoryPaths.ToolsDirectory}/{RepositoryNames.ToolEntryFile}"; if ( source.Contains( toolRedirect, StringComparison.OrdinalIgnoreCase ) || Regex.IsMatch( source, @"(?m)^\s*(?:exec\s+)?dotnet\b" ) ) { - throw new ToolException( $"Standalone fallback redirects to .NET: tools/{name}" ); + throw new ToolException( $"Shell tool redirects to .NET: tools/{name}" ); } } + + // Any Python spelling, an interpreter launch included, fails tests/repository_guards/no_python.rs. foreach ( var path in Directory.EnumerateFiles( context.Path( RepositoryPaths.ToolsDirectory, "csharp" ), "*.cs", SearchOption.AllDirectories ) ) { - var source = Files.Read( path ); - if ( Regex.IsMatch( source, "(?:ProcessRequest|context\\.Run)\\s*\\(\\s*\\\"(?:(?:ba|z)?sh|python(?:3(?:\\.\\d+)?)?)\\\"" ) ) + if ( Regex.IsMatch( Files.Read( path ), ShellLaunchPattern ) ) { - throw new ToolException( $"C# automation invokes a shell or Python interpreter: {Path.GetRelativePath( context.RepositoryRoot, path )}" ); + throw new ToolException( $"C# automation launches a shell: {Path.GetRelativePath( context.RepositoryRoot, path )}" ); } } - context.Output.WriteLine( "Standalone fallback scripts are inventoried and C# does not invoke a shell or Python interpreter." ); + + context.Output.WriteLine( "Shell tools are inventoried and usable without .NET, and C# does not launch a shell." ); return Task.FromResult( ExitCodes.Success ); } @@ -458,255 +358,3 @@ private static void Failures( ToolContext context, IReadOnlyCollection e ["zune-jpeg"] = [], }; } - -internal static class ConfigWriterAudit -{ - internal static int LastScannedCount - { - get; private set; - } - - internal static List Run( string root ) - { - var errors = new List( ); - var sources = new[] { "src", "configurator/src" }.SelectMany( directory => - Directory.EnumerateFiles( Path.Combine( root, directory ), "*.rs", SearchOption.AllDirectories ) ).Order( ).ToArray( ); - LastScannedCount = sources.Length; - var owners = new HashSet( StringComparer.Ordinal ) - { - "src/config/document.rs", "src/config/io.rs", "configurator/src/app/io.rs", - }; - string[] primitives = ["save_with_backup", "write_config_text_atomic", "create_config_backup", "prepare_config_parent"]; - string[] writers = ["persist_keybinding_edit", "persist_preset_slot", "persist_quick_color"]; - const string expectedCaller = "src/backend/wayland/config_edits.rs"; - var callers = writers.ToDictionary( name => name, _ => new HashSet( StringComparer.Ordinal ) ); - - ScanSources( root, sources, owners, primitives, writers, callers, errors ); - ValidateWriterCallers( writers, expectedCaller, callers, errors ); - ValidateWriterDefinitions( root, primitives, writers, errors ); - return errors; - } - - private static void ScanSources( string root, IEnumerable sources, HashSet owners, - IEnumerable primitives, IEnumerable writers, Dictionary> callers, List errors ) - { - foreach ( var path in sources ) - { - var relative = Path.GetRelativePath( root, path ).Replace( Path.DirectorySeparatorChar, '/' ); - if ( IsTestSource( relative ) ) - { - continue; - } - var masked = RemoveCfgTestBlocks( StripRustCommentsAndStrings( Files.Read( path ) ) ); - foreach ( var primitive in primitives ) - { - if ( !owners.Contains( relative ) && Regex.IsMatch( masked, $@"\b{primitive}\b" ) ) - { - errors.Add( $"{relative}: config write capability `{primitive}` outside the reviewed writers" ); - } - } - foreach ( var writer in writers ) - { - if ( relative is not "src/config/io.rs" and not "src/config/mod.rs" && Regex.IsMatch( masked, $@"\b{writer}\b" ) ) - { - callers[writer].Add( relative ); - } - if ( Regex.IsMatch( masked, $@"\b{writer}\s+as\s+\w+" ) ) - { - errors.Add( $"{relative}: renames config writer `{writer}`" ); - } - if ( relative is not "src/config/io.rs" and not "src/config/mod.rs" && Regex.IsMatch( masked, $@"\b{writer}_at\b" ) ) - { - errors.Add( $"{relative}: production code names `{writer}_at`" ); - } - } - } - } - - private static void ValidateWriterCallers( IEnumerable writers, string expectedCaller, - Dictionary> callers, List errors ) - { - foreach ( var writer in writers ) - { - foreach ( var unexpected in callers[writer].Where( path => path != expectedCaller ) ) - { - errors.Add( $"{unexpected}: unreviewed caller of `{writer}`" ); - } - if ( !callers[writer].Contains( expectedCaller ) ) - { - errors.Add( $"{expectedCaller}: expected to call `{writer}` but does not" ); - } - } - } - - private static void ValidateWriterDefinitions( string root, IEnumerable primitives, IEnumerable writers, List errors ) - { - var document = Files.Read( Path.Combine( root, "src/config/document.rs" ) ); - var io = Files.Read( Path.Combine( root, "src/config/io.rs" ) ); - if ( !document.Contains( "pub fn save_with_backup", StringComparison.Ordinal ) ) - { - errors.Add( "src/config/document.rs: save_with_backup is gone or renamed" ); - } - foreach ( var primitive in primitives.Where( primitive => primitive != "save_with_backup" ) ) - { - if ( !io.Contains( $"pub(super) fn {primitive}", StringComparison.Ordinal ) ) - { - errors.Add( $"src/config/io.rs: `{primitive}` is no longer pub(super)" ); - } - } - foreach ( var writer in writers ) - { - if ( !io.Contains( $"pub fn {writer}", StringComparison.Ordinal ) ) - { - errors.Add( $"src/config/io.rs: narrow config writer `{writer}` is gone" ); - } - if ( !Regex.IsMatch( io, $@"#\[cfg\(test\)\]\s*pub\(crate\) fn {writer}_at\b" ) ) - { - errors.Add( $"src/config/io.rs: `{writer}_at` is no longer a #[cfg(test)] pub(crate) fn" ); - } - } - if ( Regex.IsMatch( RemoveCfgTestBlocks( StripRustCommentsAndStrings( io ) ), @"\bauthored_config\b" ) ) - { - errors.Add( "src/config/io.rs: a narrow writer reads authored_config()" ); - } - } - - private static bool IsTestSource( string relative ) => - relative.StartsWith( "tests/", StringComparison.Ordinal ) || relative.Split( '/' ).Contains( "tests" ) || - Path.GetFileName( relative ) is "tests.rs" or "test_helpers.rs" or "test_support.rs" || - Path.GetFileName( relative ).StartsWith( "test_", StringComparison.Ordinal ) || - Path.GetFileName( relative ).EndsWith( "_tests.rs", StringComparison.Ordinal ); - - internal static string RemoveCfgTestBlocks( string text ) - { - var chars = text.ToCharArray( ); - foreach ( Match marker in Regex.Matches( text, @"#\[cfg\((?!\s*not\s*\(\s*test\s*\))(?=[^]]*\btest\b)[^]]*\)\]" ) ) - { - var opening = text.IndexOfAny( ['{', ';'], marker.Index + marker.Length ); - if ( opening < 0 ) - { - continue; - } - var end = opening + 1; - if ( text[opening] == '{' ) - { - var depth = 1; - while ( end < text.Length && depth > 0 ) - { - if ( text[end] == '{' ) - { - depth++; - } - else if ( text[end] == '}' ) - { - depth--; - } - - end++; - } - } - for ( var index = marker.Index; index < end; index++ ) - { - if ( chars[index] != '\n' ) - { - chars[index] = ' '; - } - } - } - return new string( chars ); - } - - internal static string StripRustCommentsAndStrings( string text ) - { - var output = text.ToCharArray( ); - var index = 0; - var blockDepth = 0; - while ( index < text.Length ) - { - if ( blockDepth == 0 && StartsWith( text, index, '/', '/' ) ) - { - MaskLineComment( text, output, ref index ); - } - else if ( StartsWith( text, index, '/', '*' ) ) - { - MaskPair( output, ref index ); - blockDepth++; - } - else if ( blockDepth > 0 ) - { - MaskBlockCommentCharacter( text, output, ref index, ref blockDepth ); - } - else if ( text[index] == '"' ) - { - MaskString( text, output, ref index ); - } - else - { - index++; - } - } - return new string( output ); - } - - private static bool StartsWith( string text, int index, char first, char second ) => - index + 1 < text.Length && text[index] == first && text[index + 1] == second; - - private static void MaskLineComment( string text, char[] output, ref int index ) - { - while ( index < text.Length && text[index] != '\n' ) - { - output[index++] = ' '; - } - } - - private static void MaskBlockCommentCharacter( string text, char[] output, ref int index, ref int blockDepth ) - { - if ( StartsWith( text, index, '*', '/' ) ) - { - MaskPair( output, ref index ); - blockDepth--; - return; - } - - if ( text[index] != '\n' ) - { - output[index] = ' '; - } - index++; - } - - private static void MaskPair( char[] output, ref int index ) - { - output[index++] = ' '; - output[index++] = ' '; - } - - private static void MaskString( string text, char[] output, ref int index ) - { - output[index++] = ' '; - while ( index < text.Length ) - { - var character = text[index]; - if ( character != '\n' ) - { - output[index] = ' '; - } - index++; - - if ( character == '\\' && index < text.Length ) - { - if ( text[index] != '\n' ) - { - output[index] = ' '; - } - index++; - continue; - } - - if ( character == '"' ) - { - return; - } - } - } -} diff --git a/tools/csharp/Commands/DevelopmentCommands.cs b/tools/csharp/Commands/DevelopmentCommands.cs index 5fc724c12..aabd07c86 100644 --- a/tools/csharp/Commands/DevelopmentCommands.cs +++ b/tools/csharp/Commands/DevelopmentCommands.cs @@ -4,8 +4,14 @@ internal static class DevelopmentCommands { private const int WestonStartupAttempts = 100; private const int WestonPollDelayMilliseconds = 100; - private static readonly string[] CsharpFileApps = ["tools/wayscriber.cs", "tools/install.cs", "tools/wayscriber.tests.cs"]; + private const string CsharpTestApp = "tools/wayscriber.tests.cs"; + private static readonly string[] CsharpFileApps = ["tools/wayscriber.cs", "tools/install.cs", CsharpTestApp]; private static readonly string[] CsharpFormatModes = ["style", "whitespace"]; + private static readonly string[] IsolatedRenderTests = + [ + "ui::context_menu::engine_tests::retained_context_menu_owner_preserves_layout_pixels_and_row_hits", + "ui::board_picker::tests::retained_board_text_owner_matches_fresh_during_unicode_rename_and_small_layouts", + ]; public static IReadOnlyList Commands { @@ -89,38 +95,86 @@ private static Task FetchManifest( ToolContext context, string ma private static async Task LintAndTest( ToolContext context, string[] args ) { new Arguments( args ).RequireEmpty( "ci lint-and-test" ); - await RunTool( context, CommandAreas.Assets, CommandNames.Check ); - await RunTool( context, CommandAreas.Version, CommandNames.Check ); - await RunTool( context, CommandAreas.Check, CommandNames.NixpkgsRecipe ); - await RunTool( context, CommandAreas.Check, CommandNames.RustSourceCoverage ); - await RunTool( context, CommandAreas.Check, CommandNames.ProcessSites ); - await RunTool( context, CommandAreas.Check, CommandNames.ConfigWriters ); - await RunTool( context, CommandAreas.Check, CommandNames.SharedDependencies ); - await RunTool( context, CommandAreas.Check, CommandNames.LegacyTools ); - await RunCsharpFormattingChecks( context ); - await context.Run( Programs.Dotnet, ["run", "tools/wayscriber.tests.cs", "--no-build", CommandLineOptions.EndOfOptions] ); - await RunCargo( context, ["fmt", "--all", CommandLineOptions.EndOfOptions, "--check"] ); - await RunCargo( context, ["clippy", CommandLineOptions.Locked, CommandLineOptions.Workspace, CommandLineOptions.AllTargets, CommandLineOptions.AllFeatures, CommandLineOptions.EndOfOptions, "-D", "warnings"] ); - await RunCargo( context, ["build", CommandLineOptions.Locked, CommandLineOptions.Workspace, CommandLineOptions.AllFeatures, CommandLineOptions.Binaries] ); - await RunCargo( context, ["test", CommandLineOptions.Locked, CommandLineOptions.Workspace, CommandLineOptions.AllFeatures, CommandLineOptions.EndOfOptions, CommandLineOptions.SingleTestThread] ); - await RunIsolatedRenderTests( context, CommandLineOptions.AllFeatures ); - await RunCargo( context, ["clippy", CommandLineOptions.Locked, CommandLineOptions.Workspace, CommandLineOptions.AllTargets, CommandLineOptions.NoDefaultFeatures, CommandLineOptions.EndOfOptions, "-D", "warnings"] ); - await RunCargo( context, ["build", CommandLineOptions.Locked, CommandLineOptions.Workspace, CommandLineOptions.NoDefaultFeatures, CommandLineOptions.Binaries] ); - await RunCargo( context, ["test", CommandLineOptions.Locked, CommandLineOptions.Workspace, CommandLineOptions.NoDefaultFeatures, CommandLineOptions.EndOfOptions, CommandLineOptions.SingleTestThread] ); - await RunIsolatedRenderTests( context, CommandLineOptions.NoDefaultFeatures ); + + foreach ( var step in LintAndTestPlan ) + { + await RunLintStep( context, step ); + } + return ExitCodes.Success; } - private static async Task RunCsharpFormattingChecks( ToolContext context ) + // `ci lint-and-test`, in order. After building the C# apps, `tools/lint-and-test.sh` + // runs the same steps, and the C# tests compare its commands with this list. + internal static IReadOnlyList LintAndTestPlan { + get; + } = BuildLintAndTestPlan( ); + + private static List BuildLintAndTestPlan( ) + { + List plan = + [ + new( LintStepKind.RepositoryCheck, [CommandAreas.Assets, CommandNames.Check] ), + new( LintStepKind.RepositoryCheck, [CommandAreas.Version, CommandNames.Check] ), + new( LintStepKind.RepositoryCheck, [CommandAreas.Check, CommandNames.NixpkgsRecipe] ), + new( LintStepKind.RepositoryCheck, [CommandAreas.Check, CommandNames.RustSourceCoverage] ), + new( LintStepKind.RepositoryCheck, [CommandAreas.Check, CommandNames.LegacyTools] ), + ]; + foreach ( var fileApp in CsharpFileApps ) { foreach ( var formatMode in CsharpFormatModes ) { - await context.Run( Programs.Dotnet, - ["format", formatMode, fileApp, CommandLineOptions.NoRestore, CommandLineOptions.VerifyNoChanges] ); + plan.Add( new( LintStepKind.Dotnet, + ["format", formatMode, fileApp, CommandLineOptions.NoRestore, CommandLineOptions.VerifyNoChanges] ) ); + } + } + + plan.Add( new( LintStepKind.Dotnet, ["run", CsharpTestApp, "--no-build", "--verbosity", "quiet"] ) ); + plan.Add( new( LintStepKind.Cargo, ["fmt", "--all", CommandLineOptions.EndOfOptions, "--check"] ) ); + + foreach ( var features in new[] { CommandLineOptions.AllFeatures, CommandLineOptions.NoDefaultFeatures } ) + { + string[] workspace = [CommandLineOptions.Locked, CommandLineOptions.Workspace]; + plan.Add( new( LintStepKind.Cargo, + ["clippy", .. workspace, CommandLineOptions.AllTargets, features, CommandLineOptions.EndOfOptions, "-D", "warnings"] ) ); + plan.Add( new( LintStepKind.Cargo, ["build", .. workspace, features, CommandLineOptions.Binaries] ) ); + plan.Add( new( LintStepKind.Cargo, + ["test", .. workspace, features, CommandLineOptions.EndOfOptions, CommandLineOptions.SingleTestThread] ) ); + + foreach ( var test in IsolatedRenderTests ) + { + plan.Add( new( LintStepKind.IsolatedRenderTest, + ["test", CommandLineOptions.Locked, "-p", RepositoryNames.MainPackage, features, "--lib", test, + CommandLineOptions.EndOfOptions, "--exact", "--ignored", CommandLineOptions.SingleTestThread], + $"{test} ({features})" ) ); } } + + return plan; + } + + internal static async Task RunLintStep( ToolContext context, LintStep step ) + { + switch ( step.Kind ) + { + case LintStepKind.RepositoryCheck: + await RunTool( context, step.Arguments[0], step.Arguments[1] ); + break; + case LintStepKind.Dotnet: + await context.Run( step.Program!, step.Arguments ); + break; + case LintStepKind.Cargo: + await context.Output.WriteLineAsync( $"\nRunning: {ProcessRunner.FormatCommand( step.Program!, step.Arguments )}" ); + await context.Run( step.Program!, step.Arguments, trace: false ); + break; + case LintStepKind.IsolatedRenderTest: + await RunIsolatedRenderTest( context, step ); + break; + default: + throw new ToolException( $"Unknown lint step: {step.Kind}" ); + } } private static async Task RunTool( ToolContext context, string area, string command ) @@ -133,28 +187,16 @@ private static async Task RunTool( ToolContext context, string area, string comm } } - private static async Task RunCargo( ToolContext context, IReadOnlyList arguments ) + // Parallel font tests can trigger an upstream Cairo/FreeType race, so these + // regressions run in their own processes and must each report one passing test. + private static async Task RunIsolatedRenderTest( ToolContext context, LintStep step ) { - await context.Output.WriteLineAsync( $"\nRunning: {ProcessRunner.FormatCommand( Programs.Cargo, arguments )}" ); - await context.Run( Programs.Cargo, arguments, trace: false ); - } + var result = await context.Run( step.Program!, step.Arguments, capture: true ); + await context.Output.WriteAsync( result.StandardOutput ); - private static async Task RunIsolatedRenderTests( ToolContext context, string feature ) - { - string[] tests = - [ - "ui::context_menu::engine_tests::retained_context_menu_owner_preserves_layout_pixels_and_row_hits", - "ui::board_picker::tests::retained_board_text_owner_matches_fresh_during_unicode_rename_and_small_layouts", - ]; - foreach ( var test in tests ) + if ( !result.StandardOutput.Contains( "test result: ok. 1 passed; 0 failed;", StringComparison.Ordinal ) ) { - var result = await context.Run( Programs.Cargo, ["test", CommandLineOptions.Locked, "-p", RepositoryNames.MainPackage, feature, "--lib", test, - CommandLineOptions.EndOfOptions, "--exact", "--ignored", CommandLineOptions.SingleTestThread], capture: true ); - await context.Output.WriteAsync( result.StandardOutput ); - if ( !result.StandardOutput.Contains( "test result: ok. 1 passed; 0 failed;", StringComparison.Ordinal ) ) - { - throw new ToolException( $"Expected exactly one passing isolated render test: {test} ({feature})" ); - } + throw new ToolException( $"Expected exactly one passing isolated render test: {step.Description}" ); } } @@ -235,7 +277,11 @@ private static async Task InstallDependencies( ToolContext context, string[ { "checks" => common.Concat( ["poppler-utils", "dbus-daemon", "clang", "cmake", "libxkbcommon-x11-dev", "libegl1-mesa-dev", "libgles2-mesa-dev", "libdbus-1-dev", "libinput-dev", "libudev-dev", "libpixman-1-dev", "libxcb-randr0-dev"] ) .Concat( repositoryPackages ), - "widgets" => common.Concat( ["weston", "dbus-x11", "python3", "libgl1-mesa-dri", "fonts-dejavu-core", "clang", "cmake", "libxkbcommon-x11-dev", "libegl1-mesa-dev", "libgles2-mesa-dev", "libdbus-1-dev", "libinput-dev", "libudev-dev", "libpixman-1-dev", "libxcb-randr0-dev"] ), + "widgets" => common.Concat( [ + "weston", "dbus-x11", "libgl1-mesa-dri", "fonts-dejavu-core", "clang", "cmake", "libxkbcommon-x11-dev", + "libegl1-mesa-dev", "libgles2-mesa-dev", "libdbus-1-dev", "libinput-dev", "libudev-dev", "libpixman-1-dev", + "libxcb-randr0-dev", + ] ), "package" => common.Concat( ["rpm"] ), "repositories" => repositoryPackages, _ => null, @@ -321,3 +367,24 @@ private static async Task NixInstantiation( ToolContext context, string[] a return ExitCodes.Success; } } + +internal enum LintStepKind +{ + // An in-process repository command: area, then command. + RepositoryCheck, + Dotnet, + Cargo, + // A Cargo test that must report exactly one passing test. + IsolatedRenderTest, +} + +internal sealed record LintStep( LintStepKind Kind, IReadOnlyList Arguments, string? Description = null ) +{ + // The program a process step runs; a repository check runs in process. + public string? Program => Kind switch + { + LintStepKind.Dotnet => Programs.Dotnet, + LintStepKind.Cargo or LintStepKind.IsolatedRenderTest => Programs.Cargo, + _ => null, + }; +} diff --git a/tools/csharp/Commands/PackagingCommands.cs b/tools/csharp/Commands/PackagingCommands.cs index ae6c74765..68b67d10e 100644 --- a/tools/csharp/Commands/PackagingCommands.cs +++ b/tools/csharp/Commands/PackagingCommands.cs @@ -23,7 +23,7 @@ internal static class PackagingCommands private const string InstallerManifestEndMarker = "# ARCH_INSTALL_MANIFEST_END"; private const string InstallerManifestFunction = "release_manifest() {"; private const string InstallerManifestPrint = "printf '%s\\n' \\"; - private const string ReleaseArchiveRootPattern = @"^wayscriber-v\d+\.\d+\.\d+(?:\.\d+)?-linux-x86_64$"; + private const string ReleaseArchiveRootPrefix = RepositoryNames.MainPackage + RepositoryNames.ReleaseArchiveVersionPrefix; private const string ReleaseArchivePathPattern = @"^[-A-Za-z0-9._/+]+$"; private const string ValidServiceCommandPattern = """^ExecStart=(?:")?/usr/bin/wayscriber(?:")? --daemon$"""; private const string SystemdExecDirectivePattern = @"^\s*Exec[A-Za-z]*="; @@ -184,7 +184,8 @@ await context.Error.WriteLineAsync( private static async Task BuildTar( ToolContext context, string output, string version, bool configurator ) { - var name = configurator ? $"wayscriber-configurator-v{version}-linux-x86_64" : $"wayscriber-v{version}-linux-x86_64"; + var package = configurator ? RepositoryNames.ConfiguratorPackage : RepositoryNames.MainPackage; + var name = RepositoryNames.ReleaseArchiveRoot( package, version ); var root = Path.Combine( output, name ); if ( Directory.Exists( root ) ) { @@ -216,7 +217,7 @@ void Copy( string source, string relative, UnixFileMode mode ) } Copy( context.Path( RepositoryPaths.PackagingDirectory, "icons", "wayscriber-configurator.svg" ), "usr/share/icons/hicolor/scalable/apps/wayscriber-configurator.svg", regular ); Copy( context.Path( RepositoryPaths.PackagingDirectory, "icons", "wayscriber-configurator-128.png" ), "usr/share/pixmaps/wayscriber-configurator.png", regular ); - Copy( context.Path( "README.md" ), "usr/share/doc/wayscriber-configurator/README.md", regular ); + Copy( context.Path( RepositoryNames.ReadmeFile ), "usr/share/doc/wayscriber-configurator/README.md", regular ); Copy( context.Path( "LICENSE" ), "usr/share/doc/wayscriber-configurator/LICENSE", regular ); } else @@ -233,7 +234,7 @@ void Copy( string source, string relative, UnixFileMode mode ) Copy( context.Path( RepositoryPaths.PackagingDirectory, "icons", "wayscriber.svg" ), "usr/share/icons/hicolor/scalable/apps/wayscriber.svg", regular ); Copy( context.Path( RepositoryPaths.PackagingDirectory, "icons", "wayscriber-symbolic.svg" ), "usr/share/icons/hicolor/symbolic/apps/wayscriber-symbolic.svg", regular ); Copy( context.Path( RepositoryPaths.PackagingDirectory, "icons", "wayscriber-128.png" ), "usr/share/pixmaps/wayscriber.png", regular ); - Copy( context.Path( "README.md" ), "usr/share/doc/wayscriber/README.md", regular ); + Copy( context.Path( RepositoryNames.ReadmeFile ), "usr/share/doc/wayscriber/README.md", regular ); Copy( context.Path( "config.example.toml" ), "usr/share/doc/wayscriber/config.example.toml", regular ); Copy( context.Path( "LICENSE" ), "usr/share/doc/wayscriber/LICENSE", regular ); Copy( context.Path( "LICENSE" ), "usr/share/licenses/wayscriber/LICENSE", regular ); @@ -264,8 +265,8 @@ private static async Task BuildNfpm( ToolContext context, string output, var architecture = format == PackageFormats.Debian ? "amd64" : "x86_64"; var target = Path.Combine( output, $"{prefix}-{architecture}.{format}" ); var configVariable = configurator ? EnvironmentVariables.NfpmConfiguratorConfig : EnvironmentVariables.NfpmMainConfig; - var config = context.Environment( configVariable ) ?? - context.Path( RepositoryPaths.PackagingDirectory, configurator ? "package.configurator.yaml" : "package.wayscriber.yaml" ); + var configFile = configurator ? RepositoryNames.ConfiguratorPackageConfigFile : RepositoryNames.MainPackageConfigFile; + var config = context.Environment( configVariable ) ?? context.Path( RepositoryPaths.PackagingDirectory, configFile ); File.Delete( target ); await context.Run( Programs.Nfpm, ["pkg", "--packager", format, "--config", config, "--target", target], environment: environment ); if ( !File.Exists( target ) ) @@ -385,11 +386,18 @@ private static string[] ParseSystemdExecDirectives( string service ) private static bool IsSystemdComment( string line ) => line.StartsWith( '#' ) || line.StartsWith( ';' ); + // `wayscriber-v-linux-x86_64`, the version read by the one release-version parser. + internal static bool IsReleaseArchiveRoot( string root ) => + root.StartsWith( ReleaseArchiveRootPrefix, StringComparison.Ordinal ) && + root.EndsWith( RepositoryNames.ReleaseArchiveSuffix, StringComparison.Ordinal ) && + root.Length > ReleaseArchiveRootPrefix.Length + RepositoryNames.ReleaseArchiveSuffix.Length && + ReleaseVersion.TryParse( root[ReleaseArchiveRootPrefix.Length..^RepositoryNames.ReleaseArchiveSuffix.Length], out _ ); + private static string ValidateArchiveListing( string listing ) { var paths = listing.Split( '\n', StringSplitOptions.RemoveEmptyEntries ); var root = paths.Select( path => path.TrimEnd( '/' ).Split( '/', 2 )[0] ).FirstOrDefault( ); - if ( root is null || !Regex.IsMatch( root, ReleaseArchiveRootPattern ) ) + if ( root is null || !IsReleaseArchiveRoot( root ) ) { throw new ToolException( $"Archive has an unexpected top-level directory: {root ?? ""}." ); } @@ -542,9 +550,11 @@ private static async Task VerifyArtifacts( ToolContext context, string[] ar [$"glibc >= {PackagingPlatform.MaximumGlibcVersion}", $"libadwaita >= {VersionCommands.SupportedLibadwaitaFloor}"], "configurator rpm dependencies" ); - var mainTar = await context.Run( Programs.Tar, ["-tzf", Path.Combine( root, $"wayscriber-v{version}-linux-x86_64.tar.gz" )], capture: true ); + var mainArchive = Path.Combine( root, RepositoryNames.ReleaseArchive( RepositoryNames.MainPackage, version ) ); + var mainTar = await context.Run( Programs.Tar, ["-tzf", mainArchive], capture: true ); RequireSuffixes( mainTar.StandardOutput, ["/usr/bin/wayscriber", "/usr/share/licenses/wayscriber/LICENSE.gtk4-layer-shell"], "wayscriber tar files" ); - var configTar = await context.Run( Programs.Tar, ["-tzf", Path.Combine( root, $"wayscriber-configurator-v{version}-linux-x86_64.tar.gz" )], capture: true ); + var configuratorArchive = Path.Combine( root, RepositoryNames.ReleaseArchive( RepositoryNames.ConfiguratorPackage, version ) ); + var configTar = await context.Run( Programs.Tar, ["-tzf", configuratorArchive], capture: true ); RequireSuffixes( configTar.StandardOutput, ["/usr/bin/wayscriber-configurator"], "configurator tar files" ); await context.Output.WriteLineAsync( $"Verified release artifacts for {version}." ); return ExitCodes.Success; diff --git a/tools/csharp/Commands/ReleaseAurCommands.cs b/tools/csharp/Commands/ReleaseAurCommands.cs index 5a3143c97..ece9095bf 100644 --- a/tools/csharp/Commands/ReleaseAurCommands.cs +++ b/tools/csharp/Commands/ReleaseAurCommands.cs @@ -72,7 +72,9 @@ private static async Task PublishTag( ToolContext context, string[] args ) _ = ReleaseVersion.Parse( version ); await EnsureVersionAndCleanTree( context, version ); var tag = "v" + version; - var local = await context.Run( Programs.Git, ["rev-parse", "-q", "--verify", $"refs/tags/{tag}"], capture: true, allowedExitCodes: new HashSet { ExitCodes.Success, ExitCodes.Failure } ); + // Any local ref with the tag's name, such as a branch, would make the pushed tag ambiguous. + var local = await context.Run( Programs.Git, ["rev-parse", "-q", "--verify", tag], capture: true, + allowedExitCodes: new HashSet { ExitCodes.Success, ExitCodes.Failure } ); if ( local.ExitCode == ExitCodes.Success ) { throw new ToolException( $"Tag {tag} already exists locally; aborting." ); @@ -82,12 +84,14 @@ private static async Task PublishTag( ToolContext context, string[] args ) { throw new ToolException( $"Tag {tag} already exists on origin; aborting." ); } + await context.Output.WriteLineAsync( $"Creating annotated tag {tag}" ); if ( dryRun ) { await context.Output.WriteLineAsync( $"[dry-run] git tag -a {tag} -m 'Release {tag}'\n[dry-run] git push origin {tag}" ); return ExitCodes.Success; } await context.Run( Programs.Git, ["tag", "-a", tag, "-m", $"Release {tag}"] ); + await context.Output.WriteLineAsync( $"Pushing {tag} to origin" ); try { await context.Run( Programs.Git, ["push", "origin", tag] ); @@ -96,13 +100,17 @@ private static async Task PublishTag( ToolContext context, string[] args ) return ExitCodes.Success; } + // `::`: the binary AUR package's source entry for a release. + private static string BinaryArchiveSource( string version ) + { + var archive = RepositoryNames.ReleaseArchive( RepositoryNames.MainPackage, version ); + return $"{archive}::https://github.com/devmobasa/wayscriber/releases/download/v{version}/{archive}"; + } + private static async Task EnsureVersionAndCleanTree( ToolContext context, string version ) { - var errors = VersionCommands.Validate( context.RepositoryRoot, version ); - if ( errors.Count > 0 ) - { - throw new ToolException( string.Join( '\n', errors ) ); - } + VersionCommands.EnsureConsistent( context.RepositoryRoot, version ); + var status = await context.Run( Programs.Git, ["status", "--porcelain", "--untracked-files=all"], capture: true ); if ( status.StandardOutput.Length > 0 ) { @@ -174,7 +182,10 @@ private static async Task UpdateAur( ToolContext context, string[] args ) var selections = SelectAurCheckouts( options.SourceDirectory, options.BinaryDirectory, options.ConfiguratorDirectory, options.NoConfigurator ); await ValidateAurCheckouts( context, selections ); - var binarySha = selections.Any( item => item.Channel == PackageChannels.Binary ) ? ArtifactSha( manifest.RootElement, $"wayscriber-v{version}-linux-x86_64.tar.gz" ) : string.Empty; + var binaryArchive = RepositoryNames.ReleaseArchive( RepositoryNames.MainPackage, version ); + var binarySha = selections.Any( item => item.Channel == PackageChannels.Binary ) + ? ArtifactSha( manifest.RootElement, binaryArchive ) + : string.Empty; var sourceSha = selections.Any( item => item.Channel is PackageChannels.Source or PackageChannels.Configurator ) ? await ResolveSourceChecksum( context, version, options.SourceChecksum ) : string.Empty; var recipe = AssetsCommand.CreateRecipe( context.RepositoryRoot ); await PreflightAurUpdates( context, selections, version, sourceSha, binarySha, recipe ); @@ -497,7 +508,7 @@ private static async Task TransformAur( ToolContext context, string channel, str { text = EnsureDependency( text, "gtk4", "wl-clipboard" ); text = RemoveDependency( text, RepositoryNames.Gtk4LayerShell ); - text = ReplaceArray( text, "source_x86_64", $"source_x86_64=(\"wayscriber-v{version}-linux-x86_64.tar.gz::https://github.com/devmobasa/wayscriber/releases/download/v{version}/wayscriber-v{version}-linux-x86_64.tar.gz\")" ); + text = ReplaceArray( text, "source_x86_64", $"source_x86_64=(\"{BinaryArchiveSource( version )}\")" ); text = ReplaceArray( text, "sha256sums_x86_64", $"sha256sums_x86_64=('{binarySha}')" ); if ( !text.Contains( "usr/share/licenses/wayscriber/LICENSE.gtk4-layer-shell", StringComparison.Ordinal ) ) { @@ -550,7 +561,7 @@ private static string TransformSrcInfo( string text, string channel, string vers { text = EnsureSrcInfoDependency( text, "gtk4", "wl-clipboard" ); text = RemoveSrcInfoValue( text, "depends", RepositoryNames.Gtk4LayerShell ); - text = SetSrcInfoField( text, "source_x86_64", $"wayscriber-v{version}-linux-x86_64.tar.gz::https://github.com/devmobasa/wayscriber/releases/download/v{version}/wayscriber-v{version}-linux-x86_64.tar.gz" ); + text = SetSrcInfoField( text, "source_x86_64", BinaryArchiveSource( version ) ); text = SetSrcInfoField( text, "sha256sums_x86_64", binarySha ); } else diff --git a/tools/csharp/Commands/ReportCommands.cs b/tools/csharp/Commands/ReportCommands.cs index 1e68382ae..2773282e5 100644 --- a/tools/csharp/Commands/ReportCommands.cs +++ b/tools/csharp/Commands/ReportCommands.cs @@ -8,6 +8,10 @@ internal static class ReportCommands private const int MaximumFunctionLines = 120; private const int LargeFileLineCount = 500; private const int OffsetNotFound = -1; + private const int ExitCodeLimit = 256; + + // `git ls-files` reports its own failures; the report records them instead of stopping. + private static readonly IReadOnlySet AnyExitCode = Enumerable.Range( 0, ExitCodeLimit ).ToHashSet( ); public static IReadOnlyList Commands { @@ -25,11 +29,10 @@ private static async Task CodeHealth( ToolContext context, string[] args ) parsed.RequireEmpty( "report code-health [--output FILE] [--github-summary]" ); - var git = await context.Run( Programs.Git, ["ls-files", "-co", "--exclude-standard", CommandLineOptions.EndOfOptions, "*.rs"], capture: true, trace: false ); - var paths = git.StandardOutput.Split( '\n', StringSplitOptions.RemoveEmptyEntries ).Distinct( ) - .Where( relative => File.Exists( context.Path( relative.Split( '/' ) ) ) ) - .ToArray( ); + var discovery = await DiscoverRustFiles( context ); + var paths = discovery.Paths; + var readErrors = new List( ); var files = new List<(int Lines, string Path)>( ); var functions = new List<(int Lines, string Path, int Line, string Name)>( ); var directWrites = new List( ); @@ -40,11 +43,21 @@ private static async Task CodeHealth( ToolContext context, string[] args ) foreach ( var relative in paths ) { - var text = Files.Read( context.Path( relative.Split( '/' ) ) ); + string text; + try + { + text = Files.Read( context.Path( relative.Split( '/' ) ) ); + } + catch ( Exception error ) when ( error is IOException or UnauthorizedAccessException ) + { + readErrors.Add( $"{relative}\t{error.Message}" ); + continue; + } + var lines = text.Length == 0 ? 0 : text.Count( character => character == '\n' ) + (text.EndsWith( '\n' ) ? 0 : 1); total += lines; files.Add( (lines, relative) ); - var code = ConfigWriterAudit.StripRustCommentsAndStrings( text ); + var code = RustSource.StripRustCommentsAndStrings( text ); foreach ( Match match in Regex.Matches( code, @"\bfn\s+([A-Za-z_][A-Za-z0-9_]*)\s*(?:<[^>{;]*>)?\s*\(" ) ) { var opening = FindBody( code, match.Index + match.Length ); @@ -73,7 +86,7 @@ private static async Task CodeHealth( ToolContext context, string[] args ) continue; } - var production = ConfigWriterAudit.RemoveCfgTestBlocks( code ); + var production = RustSource.RemoveCfgTestBlocks( code ); counts["unwrap"] += Regex.Matches( production, @"\.\s*unwrap\s*\(" ).Count; counts["expect"] += Regex.Matches( production, @"\.\s*expect\s*\(" ).Count; @@ -117,19 +130,22 @@ private static async Task CodeHealth( ToolContext context, string[] args ) directWrites.Sort( StringComparer.Ordinal ); var report = new StringBuilder( ); - report.AppendLine( "report=wayscriber-code-health" ).AppendLine( "status=ok" ).AppendLine( $"repo_root={context.RepositoryRoot}" ) + report.AppendLine( "report=wayscriber-code-health" ); + AppendStatus( report, discovery, readErrors.Count > 0 ); + report.AppendLine( $"repo_root={context.RepositoryRoot}" ) .AppendLine( $"rust_files={paths.Length}" ).AppendLine( $"rust_physical_lines={total}" ) .AppendLine( $"files_over_{LargeFileLineCount}={files.Count( item => item.Lines > LargeFileLineCount )}" ) .AppendLine( $"functions_over_120={functions.Count}" ).AppendLine( $"production_unwrap={counts["unwrap"]}" ).AppendLine( $"production_expect={counts["expect"]}" ) .AppendLine( $"production_panic={counts["panic"]}" ).AppendLine( $"production_unsafe={counts["unsafe"]}" ).AppendLine( $"allow_dead_code={allowDead}" ) - .AppendLine( $"allow_unused_imports={allowUnused}" ).AppendLine( $"direct_fs_write_files={directWrites.Count}" ).AppendLine( "read_errors=0" ); + .AppendLine( $"allow_unused_imports={allowUnused}" ).AppendLine( $"direct_fs_write_files={directWrites.Count}" ) + .AppendLine( $"read_errors={readErrors.Count}" ); var filesOverLimit = files.Where( item => item.Lines > LargeFileLineCount ).Select( item => $"{item.Lines}\t{item.Path}" ); Section( report, $"files_over_{LargeFileLineCount}", filesOverLimit ); Section( report, "functions_over_120", functions.Select( item => $"{item.Lines}\t{item.Path}:{item.Line}\t{item.Name}" ) ); Section( report, "direct_fs_write_files", directWrites ); - Section( report, "read_errors", [] ); + Section( report, "read_errors", readErrors ); if ( outputPath is not null ) { @@ -155,6 +171,83 @@ private static async Task CodeHealth( ToolContext context, string[] args ) return ExitCodes.Success; } + // The report is observational: a discovery or read problem is reported in its + // status lines, and the report still describes whatever it could read. + private static async Task DiscoverRustFiles( ToolContext context ) + { + ProcessResult git; + try + { + git = await context.Run( Programs.Git, ["ls-files", "-co", "--exclude-standard", CommandLineOptions.EndOfOptions, "*.rs"], + capture: true, trace: false, allowedExitCodes: AnyExitCode ); + } + catch ( ToolException error ) when ( error.ExitCode == ExitCodes.CommandNotFound ) + { + return new( [], new( "git_unavailable", error.Message ), null ); + } + + var stderr = git.StandardError.Trim( ); + if ( !git.IsSuccess ) + { + return new( [], new( "git_ls_files_failed", stderr ), null ); + } + + var paths = git.StandardOutput.Split( '\n', StringSplitOptions.RemoveEmptyEntries ).Distinct( ) + .Where( relative => File.Exists( context.Path( relative.Split( '/' ) ) ) ) + .ToArray( ); + return new( paths, null, stderr.Length > 0 ? new ReportProblem( "git_ls_files_stderr", stderr ) : null ); + } + + private static void AppendStatus( StringBuilder report, RustFileDiscovery discovery, bool readFailed ) + { + var errors = new List( ); + var warnings = new List( ); + if ( discovery.Error is not null ) + { + errors.Add( "discovery" ); + } + if ( readFailed ) + { + errors.Add( "read" ); + } + if ( discovery.Warning is not null ) + { + warnings.Add( "discovery" ); + } + + var status = errors.Count > 0 ? "error" : warnings.Count > 0 ? "warning" : "ok"; + report.AppendLine( $"status={status}" ); + if ( errors.Count > 0 ) + { + report.AppendLine( $"errors={string.Join( ',', errors )}" ); + } + if ( warnings.Count > 0 ) + { + report.AppendLine( $"warnings={string.Join( ',', warnings )}" ); + } + AppendProblem( report, "error", discovery.Error ); + AppendProblem( report, "warning", discovery.Warning ); + } + + private static void AppendProblem( StringBuilder report, string kind, ReportProblem? problem ) + { + if ( problem is not { } found ) + { + return; + } + + report.AppendLine( $"{kind}={found.Name}" ); + if ( found.Detail.Length > 0 ) + { + report.AppendLine( $"{kind}_detail={found.Detail}" ); + } + } + + private sealed record RustFileDiscovery( string[] Paths, ReportProblem? Error, ReportProblem? Warning ); + + // A named problem and its detail, printed as `error=`/`warning=` lines. + private sealed record ReportProblem( string Name, string Detail ); + private static void Section( StringBuilder report, string name, IEnumerable rows ) { report.AppendLine( ).AppendLine( name + ":" ); diff --git a/tools/csharp/Commands/VersionCommands.cs b/tools/csharp/Commands/VersionCommands.cs index 55ed745aa..a596b22cd 100644 --- a/tools/csharp/Commands/VersionCommands.cs +++ b/tools/csharp/Commands/VersionCommands.cs @@ -1,4 +1,3 @@ -using System.Text.Json; using System.Text.RegularExpressions; namespace Wayscriber.Tools; @@ -12,27 +11,67 @@ internal sealed record ReleaseVersion( int Major, int Minor, int Patch, int? Hot public string CargoVersion => $"{Major}.{Minor}.{Patch}"; public bool IsHotfix => Hotfix is not null; - public ReleaseVersion NextPatch( ) => new( Major, Minor, checked(Patch + 1), null ); + public ReleaseVersion NextPatch( ) + { + if ( Patch == int.MaxValue ) + { + throw new ToolException( $"cannot increment the patch version of {CargoVersion}; pass the next version explicitly", + ExitCodes.InvalidArguments ); + } + + return new( Major, Minor, Patch + 1, null ); + } public override string ToString( ) => Hotfix is null ? CargoVersion : $"{CargoVersion}.{Hotfix}"; + // ASCII digits without leading zeros, as SemVer numbers are, so a parsed + // version prints back as the text it came from. + private const string Number = "0|[1-9][0-9]*"; + public static ReleaseVersion Parse( string value ) { - var match = Regex.Match( value, - $@"^(?<{MajorGroup}>\d+)\.(?<{MinorGroup}>\d+)\.(?<{PatchGroup}>\d+)(?:\.(?<{HotfixGroup}>\d+))?$" ); - if ( !match.Success ) + if ( !TryParse( value, out var version ) ) { throw new ToolException( $"invalid version format: {value} (expected MAJOR.MINOR.PATCH[.HOTFIX])", ExitCodes.InvalidArguments ); } - return new( int.Parse( match.Groups[MajorGroup].Value ), int.Parse( match.Groups[MinorGroup].Value ), - int.Parse( match.Groups[PatchGroup].Value ), - match.Groups[HotfixGroup].Success ? int.Parse( match.Groups[HotfixGroup].Value ) : null ); + return version; + } + + public static bool TryParse( string value, [System.Diagnostics.CodeAnalysis.NotNullWhen( true )] out ReleaseVersion? version ) + { + version = null; + var match = Regex.Match( value, + $@"^(?<{MajorGroup}>{Number})\.(?<{MinorGroup}>{Number})\.(?<{PatchGroup}>{Number})(?:\.(?<{HotfixGroup}>{Number}))?\z" ); + if ( !match.Success || + !int.TryParse( match.Groups[MajorGroup].Value, out var major ) || + !int.TryParse( match.Groups[MinorGroup].Value, out var minor ) || + !int.TryParse( match.Groups[PatchGroup].Value, out var patch ) ) + { + return false; + } + + int? hotfix = null; + if ( match.Groups[HotfixGroup].Success ) + { + if ( !int.TryParse( match.Groups[HotfixGroup].Value, out var number ) ) + { + return false; + } + hotfix = number; + } + + version = new( major, minor, patch, hotfix ); + return true; } } -internal static class VersionCommands +internal static partial class VersionCommands { - private const string WorkflowRunnerGroup = "runner"; + private const string OfflineResolutionFailure = + "cannot resolve locked dependencies offline; run `dotnet run tools/wayscriber.cs --no-build -- dev fetch` " + + "before bumping the version. No version files changed."; + + // Raise this floor only as a coordinated release-platform change. internal const string SupportedLibadwaitaFloor = "1.4"; internal const string ToolSdkVersion = "11.0.100-rc.1.26425.128"; internal const string ToolSdkRollForward = "disable"; @@ -63,11 +102,7 @@ private static Task Check( ToolContext context, string[] args ) var parsed = new Arguments( args ); var releaseText = parsed.TakeOption( "--release-version" ); parsed.RequireEmpty( "version check [--release-version X.Y.Z[.N]]" ); - var errors = Validate( context.RepositoryRoot, releaseText ); - if ( errors.Count > 0 ) - { - throw new ToolException( "Version consistency check failed:\n" + string.Join( '\n', errors.Select( error => $"- {error}" ) ) ); - } + EnsureConsistent( context.RepositoryRoot, releaseText ); var cargo = ReadCargoVersion( context.Path( RepositoryPaths.CargoManifest ) ); var package = ReadAssignment( Files.Read( context.Path( RepositoryPaths.PackagingDirectory, RepositoryNames.PackageBuildFile ) ), @@ -77,146 +112,6 @@ private static Task Check( ToolContext context, string[] args ) return Task.FromResult( ExitCodes.Success ); } - internal static List Validate( string root, string? releaseText = null ) - { - var errors = new List( ); - string Read( params string[] parts ) => Files.Read( Path.Combine( [root, .. parts] ) ); - - ValidateToolSdk( Read, errors ); - var cargo = ReadCargoVersion( Path.Combine( root, RepositoryPaths.CargoManifest ) ); - ValidateConfiguratorVersion( root, Read, cargo, errors ); - ValidatePackageVersions( Read, cargo, releaseText, errors ); - ValidateFlakeAndReadme( Read, errors ); - return errors; - } - - private static void ValidateToolSdk( Func read, List errors ) - { - try - { - using var globalJson = JsonDocument.Parse( read( ["global.json"] ) ); - RequireEqual( errors, "global.json SDK", globalJson.RootElement.GetProperty( "sdk" ).GetProperty( "version" ).GetString( ), - ToolSdkVersion ); - var sdk = globalJson.RootElement.GetProperty( "sdk" ); - if ( !sdk.TryGetProperty( "rollForward", out var rollForward ) || rollForward.GetString( ) != ToolSdkRollForward ) - { - errors.Add( "global.json SDK rollForward must be disable" ); - } - if ( !sdk.TryGetProperty( "allowPrerelease", out var allowPrerelease ) || allowPrerelease.ValueKind != JsonValueKind.True ) - { - errors.Add( "global.json SDK allowPrerelease must be true" ); - } - } - catch ( Exception error ) when ( error is JsonException or KeyNotFoundException or InvalidOperationException ) - { - errors.Add( $"global.json SDK metadata is invalid: {error.Message}" ); - } - } - - private static void ValidateConfiguratorVersion( string root, Func read, string cargo, List errors ) - { - var configurator = ReadCargoVersion( Path.Combine( root, RepositoryPaths.ConfiguratorCargoManifest ) ); - RequireEqual( errors, RepositoryPaths.ConfiguratorCargoManifest, configurator, cargo ); - var configManifest = read( [RepositoryPaths.ConfiguratorCargoManifest] ); - var featureMatch = Regex.Match( configManifest, @"(?m)^libadwaita\s*=\s*\{[^\n]*features\s*=\s*\[([^]]*)\]" ); - var features = featureMatch.Success - ? Regex.Matches( featureMatch.Groups[1].Value, "\"([^\"]+)\"" ).Select( item => item.Groups[1].Value ).ToArray( ) - : []; - var expectedFeature = "v" + SupportedLibadwaitaFloor.Replace( '.', '_' ); - if ( !features.SequenceEqual( [expectedFeature] ) ) - { - errors.Add( - $"configurator/Cargo.toml libadwaita features: expected ['{expectedFeature}'], got [{string.Join( ", ", features )}]" ); - } - - var floors = new Dictionary - { - ["configurator deb libadwaita floor"] = - MatchOne( read( [RepositoryPaths.PackagingDirectory, "package.configurator.yaml"] ), - @"(?m)^\s*-\s*libadwaita-1-0 \(>= ([0-9]+\.[0-9]+)\)\s*$", errors, "configurator deb libadwaita floor" ), - ["configurator rpm libadwaita floor"] = - MatchOne( read( [RepositoryPaths.PackagingDirectory, "package.configurator.yaml"] ), - @"(?m)^\s*-\s*libadwaita >= ([0-9]+\.[0-9]+)\s*$", errors, "configurator rpm libadwaita floor" ), - ["packaging/PKGBUILD libadwaita floor"] = - MatchOne( read( [RepositoryPaths.PackagingDirectory, RepositoryNames.PackageBuildFile] ), - @"(?m)^\s*'libadwaita>=([0-9]+\.[0-9]+)'\s*$", errors, "packaging/PKGBUILD libadwaita floor" ), - ["packaging/.SRCINFO libadwaita floor"] = - MatchOne( read( [RepositoryPaths.PackagingDirectory, RepositoryNames.SourceInfoFile] ), - @"(?m)^\s*depends = libadwaita>=([0-9]+\.[0-9]+)\s*$", errors, "packaging/.SRCINFO libadwaita floor" ), - }; - foreach ( var pair in floors.Where( pair => pair.Value is not null ) ) - { - RequireEqual( errors, pair.Key, pair.Value, SupportedLibadwaitaFloor ); - } - - var workflow = read( [".github", "workflows", "build-packages.yml"] ); - var job = Regex.Match( workflow, @"(?ms)^ package:\s*$.*?(?=^ [A-Za-z0-9_-]+:\s*$|\z)" ); - var runner = job.Success ? Regex.Match( job.Value, $@"(?m)^ runs-on:\s*(?<{WorkflowRunnerGroup}>[^#\n]+?)\s*$" ) : Match.Empty; - if ( !runner.Success ) - { - errors.Add( ".github/workflows/build-packages.yml package job: expected one literal runs-on value" ); - } - else if ( runner.Groups[WorkflowRunnerGroup].Value != PackagingPlatform.UbuntuRunner ) - { - errors.Add( $"release package runner libadwaita floor: no reviewed contract for {runner.Groups[WorkflowRunnerGroup].Value}" ); - } - } - - private static void ValidatePackageVersions( Func read, string cargo, string? releaseText, List errors ) - { - RequireEqual( errors, "Cargo.lock wayscriber", LockVersion( read( [RepositoryPaths.CargoLock] ), RepositoryNames.MainPackage ), - cargo ); - RequireEqual( errors, "Cargo.lock wayscriber-configurator", - LockVersion( read( [RepositoryPaths.CargoLock] ), RepositoryNames.ConfiguratorPackage ), cargo ); - var baseVersion = ReleaseVersion.Parse( cargo ); - ReleaseVersion? release = releaseText is null ? null : ReleaseVersion.Parse( releaseText ); - if ( release is not null && release.CargoVersion != cargo ) - { - errors.Add( $"release version {release} must equal Cargo version {cargo} or be a hotfix of it, such as {cargo}.1" ); - } - - var pkgbuildVersion = ReadAssignment( read( [RepositoryPaths.PackagingDirectory, RepositoryNames.PackageBuildFile] ), "pkgver" ); - var srcinfoVersion = Regex - .Match( read( [RepositoryPaths.PackagingDirectory, RepositoryNames.SourceInfoFile] ), @"(?m)^\s*pkgver = (.+)$" ).Groups[1] - .Value.Trim( ); - var expectedPackage = release?.ToString( ) ?? - (pkgbuildVersion is not null && - Regex.IsMatch( pkgbuildVersion, $@"^{Regex.Escape( baseVersion.CargoVersion )}\.\d+$" ) - ? pkgbuildVersion - : cargo); - RequireEqual( errors, "packaging/PKGBUILD pkgver", pkgbuildVersion, expectedPackage ); - RequireEqual( errors, "packaging/.SRCINFO pkgver", srcinfoVersion, expectedPackage ); - CheckTemplateChecksum( errors, "packaging/PKGBUILD sha256sums", - Regex.Match( read( [RepositoryPaths.PackagingDirectory, RepositoryNames.PackageBuildFile] ), @"(?ms)^sha256sums=\((.*?)\)" ) - .Groups[1].Value ); - CheckTemplateChecksum( errors, "packaging/.SRCINFO sha256sums", - string.Join( ' ', - Regex.Matches( read( [RepositoryPaths.PackagingDirectory, RepositoryNames.SourceInfoFile] ), @"(?m)^\s*sha256sums = (.+)$" ) - .Select( item => item.Groups[1].Value ) ) ); - } - - private static void ValidateFlakeAndReadme( Func read, List errors ) - { - var flake = read( ["flake.nix"] ); - if ( !flake.Contains( "builtins.fromTOML (builtins.readFile ./Cargo.toml)", StringComparison.Ordinal ) ) - { - errors.Add( "flake.nix package version should be derived from Cargo.toml" ); - } - - if ( !(flake.Contains( "package.rust-version" ) && flake.Contains( "rustToolchain.version" ) && - flake.Contains( "versionAtLeast" )) ) - { - errors.Add( "flake.nix should compare selected rustc against Cargo.toml rust-version" ); - } - - var readme = read( ["README.md"] ); - foreach ( Match match in Regex.Matches( readme, - @"wayscriber\?ref=v?\d+\.\d+\.\d+(?:\.\d+)?|/releases/(?:tag|download)/v?\d+\.\d+\.\d+(?:\.\d+)?" ) ) - { - errors.Add( $"README.md: pinned release reference '{match.Value}' goes stale on the next release" ); - } - } - private static async Task Bump( ToolContext context, string[] args ) { var parsed = new Arguments( args ); @@ -235,9 +130,11 @@ private static async Task Bump( ToolContext context, string[] args ) await context.Output.WriteLineAsync( $"Cargo version: {next.CargoVersion} (release version has hotfix)" ); } + // Resolve against the existing manifests without writing the lockfile. An empty + // dependency cache must fail before either manifest or any package metadata changes. if ( File.Exists( context.Path( RepositoryPaths.CargoLock ) ) ) { - await context.Run( Programs.Cargo, ["update", CommandLineOptions.Workspace, "--offline", "--dry-run"], capture: true ); + await RequireOfflineResolution( context ); } var outputs = new AtomicFileSet( ); @@ -280,11 +177,7 @@ await context.Output.WriteLineAsync( var srcinfo = await context.Run( Programs.Makepkg, ["--printsrcinfo"], context.Path( RepositoryPaths.PackagingDirectory ), capture: true ); Files.WriteAtomic( context.Path( RepositoryPaths.PackagingDirectory, RepositoryNames.SourceInfoFile ), srcinfo.StandardOutput ); - var errors = Validate( context.RepositoryRoot, next.ToString( ) ); - if ( errors.Count > 0 ) - { - throw new ToolException( string.Join( '\n', errors ) ); - } + EnsureConsistent( context.RepositoryRoot, next.ToString( ) ); } catch { @@ -307,45 +200,15 @@ await context.Output.WriteLineAsync( return ExitCodes.Success; } - private static string? MatchOne( string text, string pattern, List errors, string label ) - { - var matches = Regex.Matches( text, pattern ); - if ( matches.Count != 1 ) - { - errors.Add( $"{label}: expected one libadwaita floor, found {matches.Count}" ); - return null; - } - - return matches[0].Groups[1].Value; - } - - private static string? LockVersion( string text, string package ) + private static async Task RequireOfflineResolution( ToolContext context ) { - var match = Regex.Match( text, - "(?ms)\\[\\[package\\]\\]\\s+name\\s*=\\s*\"" + Regex.Escape( package ) + "\"\\s+version\\s*=\\s*\"([^\"]+)\"" ); - return match.Success ? match.Groups[1].Value : null; - } - - private static string? ReadAssignment( string text, string name ) - { - var match = Regex.Match( text, $@"(?m)^{Regex.Escape( name )}=(.+)$" ); - return match.Success ? match.Groups[1].Value.Trim( ) : null; - } - - private static void RequireEqual( List errors, string label, string? actual, string expected ) - { - if ( actual != expected ) + try { - errors.Add( $"{label}: expected {expected}, got {actual ?? "missing"}" ); + await context.Run( Programs.Cargo, ["update", CommandLineOptions.Workspace, "--offline", "--dry-run"], capture: true ); } - } - - private static void CheckTemplateChecksum( List errors, string label, string value ) - { - var values = Regex.Matches( value, @"[A-Za-z0-9]+" ).Select( match => match.Value ).ToArray( ); - if ( !values.SequenceEqual( [EnvironmentVariables.Skip] ) ) + catch ( ToolException error ) when ( error.ExitCode != ExitCodes.CommandNotFound ) { - errors.Add( $"{label}: expected SKIP template checksum, got {(values.Length == 0 ? "missing" : string.Join( ", ", values ))}" ); + throw new ToolException( OfflineResolutionFailure ); } } } diff --git a/tools/csharp/Commands/VersionConsistency.cs b/tools/csharp/Commands/VersionConsistency.cs new file mode 100644 index 000000000..237d517dd --- /dev/null +++ b/tools/csharp/Commands/VersionConsistency.cs @@ -0,0 +1,366 @@ +using System.Text.Json; +using System.Text.RegularExpressions; + +namespace Wayscriber.Tools; + +// Release metadata rules shared by `version check`, `version bump`, and the release-tag commands. +internal static partial class VersionCommands +{ + // Repository-relative paths, `/`-separated as `Validate` reads them. + private const string ConfiguratorPackageConfig = + RepositoryPaths.PackagingDirectory + "/" + RepositoryNames.ConfiguratorPackageConfigFile; + private const string PackageBuildPath = RepositoryPaths.PackagingDirectory + "/" + RepositoryNames.PackageBuildFile; + private const string SourceInfoPath = RepositoryPaths.PackagingDirectory + "/" + RepositoryNames.SourceInfoFile; + private const string ShellAurUpdater = "tools/update-aur-from-manifest.sh"; + private const string ReleaseWorkflow = ".github/workflows/build-packages.yml"; + private const string ReleasePackageJob = "package"; + private const string ValueGroup = "value"; + private const string OtherGroup = "other"; + private const string ItemsGroup = "items"; + private const string TomlStringArrayToken = @"""(?[^""\\\n]*)""|'(?[^'\n]*)'|#[^\n]*|[,\s]+|(?.)"; + + // A floor raise must add a reviewed runner contract instead of inheriting the + // previous Ubuntu base image by accident. + private static readonly IReadOnlyDictionary ReleaseRunnerLibadwaitaFloors = + new Dictionary( StringComparer.Ordinal ) { [PackagingPlatform.UbuntuRunner] = "1.4" }; + + private static readonly (string Label, string Path, string Pattern)[] LibadwaitaFloorSurfaces = + [ + ("configurator deb libadwaita floor", ConfiguratorPackageConfig, @"^\s*-\s*libadwaita-1-0 \(>= ([0-9]+\.[0-9]+)\)\s*$"), + ("configurator rpm libadwaita floor", ConfiguratorPackageConfig, @"^\s*-\s*libadwaita >= ([0-9]+\.[0-9]+)\s*$"), + ("packaging/PKGBUILD libadwaita floor", PackageBuildPath, @"^\s*'libadwaita>=([0-9]+\.[0-9]+)'\s*$"), + ("packaging/.SRCINFO libadwaita floor", SourceInfoPath, @"^\s*depends = libadwaita>=([0-9]+\.[0-9]+)\s*$"), + ("AUR updater generated libadwaita floor", ShellAurUpdater, + @"^\s*ensure_runtime_dependency 'libadwaita>=([0-9]+\.[0-9]+)' gcc-libs\s*$"), + ("AUR updater PKGBUILD validation floor", ShellAurUpdater, + @"^\s*&& grep -Eq ""[^""\n]*libadwaita>=([0-9]+\.[0-9]+)[^""\n]*"" PKGBUILD"), + ("AUR updater .SRCINFO validation floor", ShellAurUpdater, + @"^\s*&& grep -Fxq .*depends = libadwaita>=([0-9]+\.[0-9]+).*\.SRCINFO"), + ]; + + // Install examples that pin a concrete tag are stale one release later. + private static readonly (string Pattern, string Label)[] ReadmePinPatterns = + [ + (@"wayscriber\?ref=v?\d+\.\d+\.\d+(?:\.\d+)?", "pinned flake ref"), + (@"/releases/(?:tag|download)/v?\d+\.\d+\.\d+(?:\.\d+)?", "pinned release URL"), + ]; + + internal static void EnsureConsistent( string root, string? releaseText ) + { + var errors = Validate( root, releaseText ); + if ( errors.Count > 0 ) + { + throw new ToolException( "Version consistency check failed:\n" + string.Join( '\n', errors.Select( error => $"- {error}" ) ) ); + } + } + + // Commands always use SupportedLibadwaitaFloor; another floor models a coordinated + // floor raise, which must still be rejected until the release runner contract changes. + internal static List Validate( string root, string? releaseText = null, + string supportedLibadwaitaFloor = SupportedLibadwaitaFloor ) + { + string PathFor( string relativePath ) => Path.Combine( [root, .. relativePath.Split( '/' )] ); + string Read( string relativePath ) => Files.Read( PathFor( relativePath ) ); + + var errors = new List( ); + var cargo = ReadCargoVersion( PathFor( RepositoryPaths.CargoManifest ) ); + var configurator = ReadCargoVersion( PathFor( RepositoryPaths.ConfiguratorCargoManifest ) ); + + RequireEqual( errors, RepositoryPaths.ConfiguratorCargoManifest, configurator, cargo ); + RequireEqual( errors, $"{RepositoryNames.GlobalJsonFile} SDK", ReadToolSdk( Read, errors ), ToolSdkVersion ); + ValidateLibadwaitaFloors( Read, supportedLibadwaitaFloor, errors ); + ValidateReleaseRunner( Read( ReleaseWorkflow ), supportedLibadwaitaFloor, errors ); + ValidatePackageVersions( Read, cargo, releaseText, errors ); + ValidateFlake( Read( RepositoryNames.FlakeFile ), errors ); + ValidateReadme( Read( RepositoryNames.ReadmeFile ), errors ); + + return errors; + } + + private static string? ReadToolSdk( Func read, List errors ) + { + JsonElement sdk; + JsonElement version; + try + { + using var document = JsonDocument.Parse( read( RepositoryNames.GlobalJsonFile ) ); + sdk = document.RootElement.GetProperty( "sdk" ).Clone( ); + version = sdk.GetProperty( "version" ); + } + catch ( Exception error ) when ( error is IOException or JsonException or KeyNotFoundException or InvalidOperationException ) + { + errors.Add( $"{RepositoryNames.GlobalJsonFile} SDK metadata is invalid: {error.Message}" ); + return null; + } + + if ( version.ValueKind != JsonValueKind.String || string.IsNullOrWhiteSpace( version.GetString( ) ) ) + { + errors.Add( $"{RepositoryNames.GlobalJsonFile} SDK metadata is invalid: sdk.version must be a non-empty string" ); + return null; + } + + if ( !sdk.TryGetProperty( "rollForward", out var rollForward ) || rollForward.ValueKind != JsonValueKind.String || + rollForward.GetString( ) != ToolSdkRollForward ) + { + errors.Add( $"{RepositoryNames.GlobalJsonFile} SDK rollForward must be {ToolSdkRollForward}" ); + } + if ( !sdk.TryGetProperty( "allowPrerelease", out var allowPrerelease ) || allowPrerelease.ValueKind != JsonValueKind.True ) + { + errors.Add( $"{RepositoryNames.GlobalJsonFile} SDK allowPrerelease must be true" ); + } + + return version.GetString( )!.Trim( ); + } + + private static void ValidateLibadwaitaFloors( Func read, string floor, List errors ) + { + string[] expectedFeatures = ["v" + floor.Replace( '.', '_' )]; + var features = ReadLibadwaitaFeatures( read( RepositoryPaths.ConfiguratorCargoManifest ), errors ); + if ( !features.SequenceEqual( expectedFeatures ) ) + { + errors.Add( $"{RepositoryPaths.ConfiguratorCargoManifest} libadwaita features: expected {FormatList( expectedFeatures )}, " + + $"got {FormatList( features )}" ); + } + + foreach ( var (label, path, pattern) in LibadwaitaFloorSurfaces ) + { + var matches = Regex.Matches( read( path ), pattern, RegexOptions.Multiline ); + if ( matches.Count != 1 ) + { + errors.Add( $"{label}: expected one libadwaita floor, found {matches.Count}" ); + continue; + } + + RequireEqual( errors, label, matches[0].Groups[1].Value, floor ); + } + } + + private static string[] ReadLibadwaitaFeatures( string manifest, List errors ) + { + var dependency = LibadwaitaDependencyTable( manifest ); + if ( dependency is null ) + { + errors.Add( $"{RepositoryPaths.ConfiguratorCargoManifest}: missing structured libadwaita dependency" ); + return []; + } + + var features = Regex.Match( dependency, $@"(?[^\]]*)\]" ); + if ( !features.Success ) + { + return []; + } + + var values = new List( ); + foreach ( Match token in Regex.Matches( features.Groups[ItemsGroup].Value, TomlStringArrayToken ) ) + { + if ( token.Groups[OtherGroup].Success ) + { + errors.Add( $"{RepositoryPaths.ConfiguratorCargoManifest}: libadwaita features must be a string list" ); + return []; + } + if ( token.Groups[ValueGroup].Success ) + { + values.Add( token.Groups[ValueGroup].Value ); + } + } + + return [.. values]; + } + + // Accept the inline `libadwaita = { ... }` form in [dependencies] and the + // [dependencies.libadwaita] table form; a plain version string has no features. + private static string? LibadwaitaDependencyTable( string manifest ) + { + var dependencies = TomlTableBody( manifest, "dependencies" ); + var inline = dependencies is null + ? Match.Empty + : Regex.Match( dependencies, $@"(?m)^[ \t]*libadwaita[ \t]*=[ \t]*\{{(?<{ValueGroup}>[^\n]*)\}}[ \t]*(?:#[^\n]*)?$" ); + return inline.Success ? inline.Groups[ValueGroup].Value : TomlTableBody( manifest, "dependencies.libadwaita" ); + } + + private static string? TomlTableBody( string manifest, string header ) + { + var table = Regex.Match( manifest, + $@"(?ms)^[ \t]*\[[ \t]*{Regex.Escape( header )}[ \t]*\][ \t]*(?:#[^\n]*)?$(?<{ValueGroup}>.*?)(?=^[ \t]*\[|\z)" ); + return table.Success ? table.Groups[ValueGroup].Value : null; + } + + private static void ValidateReleaseRunner( string workflow, string floor, List errors ) + { + var runner = ReadWorkflowJobRunner( workflow, errors ); + if ( runner is null ) + { + return; + } + + if ( !ReleaseRunnerLibadwaitaFloors.TryGetValue( runner, out var runnerFloor ) ) + { + errors.Add( $"release package runner libadwaita floor: no reviewed contract for {runner}" ); + return; + } + + RequireEqual( errors, $"release package runner {runner} libadwaita floor", floor, runnerFloor ); + } + + private static string? ReadWorkflowJobRunner( string workflow, List errors ) + { + var start = Regex.Match( workflow, $@"(?m)^ {Regex.Escape( ReleasePackageJob )}:[ \t]*$" ); + if ( !start.Success ) + { + errors.Add( $"{ReleaseWorkflow}: missing {ReleasePackageJob} job" ); + return null; + } + + var remaining = workflow[(start.Index + start.Length)..]; + var nextJob = Regex.Match( remaining, @"(?m)^ [A-Za-z0-9_-]+:[ \t]*$" ); + var job = nextJob.Success ? remaining[..nextJob.Index] : remaining; + var runners = Regex.Matches( job, @"(?m)^ runs-on:[ \t]*([^#\n]+?)[ \t]*$" ); + if ( runners.Count != 1 ) + { + errors.Add( $"{ReleaseWorkflow} {ReleasePackageJob} job: expected one literal runs-on value, found {runners.Count}" ); + return null; + } + + return runners[0].Groups[1].Value; + } + + private static void ValidatePackageVersions( Func read, string cargo, string? releaseText, List errors ) + { + var cargoLock = read( RepositoryPaths.CargoLock ); + RequireEqual( errors, "Cargo.lock wayscriber", LockVersion( cargoLock, RepositoryNames.MainPackage ), cargo ); + RequireEqual( errors, "Cargo.lock wayscriber-configurator", LockVersion( cargoLock, RepositoryNames.ConfiguratorPackage ), cargo ); + if ( !IsReleaseVersion( cargo ) ) + { + errors.Add( $"Cargo.toml version has unsupported format: {cargo}" ); + } + + var packageBuild = read( PackageBuildPath ); + var sourceInfo = read( SourceInfoPath ); + var packageBuildVersion = ReadAssignment( packageBuild, "pkgver" ); + var sourceInfoVersion = Regex.Match( sourceInfo, @"(?m)^\s*pkgver = (.+)$" ); + var expected = ExpectedPackageVersion( cargo, releaseText, packageBuildVersion, errors ); + RequireEqual( errors, "packaging/PKGBUILD pkgver", packageBuildVersion, expected ); + RequireEqual( errors, "packaging/.SRCINFO pkgver", sourceInfoVersion.Success ? sourceInfoVersion.Groups[1].Value.Trim( ) : null, + expected ); + RequireTemplateChecksums( errors, "packaging/PKGBUILD sha256sums", PackageBuildChecksums( packageBuild ) ); + RequireTemplateChecksums( errors, "packaging/.SRCINFO sha256sums", + Regex.Matches( sourceInfo, @"(?m)^\s*sha256sums = (.+)$" ).Select( match => match.Groups[1].Value.Trim( ) ).ToArray( ) ); + } + + // A packaging hotfix such as 0.9.19.1 ships Cargo 0.9.19; release automation + // names the hotfix explicitly, while a plain check accepts the recipe's hotfix. + private static string ExpectedPackageVersion( string cargo, string? releaseText, string? packageBuildVersion, List errors ) + { + if ( releaseText is null ) + { + return packageBuildVersion is not null && IsHotfixOf( packageBuildVersion, cargo ) ? packageBuildVersion : cargo; + } + + if ( !IsReleaseVersion( releaseText ) ) + { + errors.Add( $"release version has unsupported format: {releaseText}" ); + } + else if ( releaseText != cargo && !IsHotfixOf( releaseText, cargo ) ) + { + errors.Add( $"release version {releaseText} must equal Cargo version {cargo} or be a hotfix of it, such as {cargo}.1" ); + } + + return releaseText; + } + + private static string[] PackageBuildChecksums( string packageBuild ) + { + var array = Regex.Match( packageBuild, @"(?ms)^sha256sums=\((.*?)\)" ); + if ( !array.Success ) + { + return []; + } + + return Regex.Matches( array.Groups[1].Value, @"'([^']*)'|""([^""]*)""|(\S+)" ) + .Select( match => match.Groups.Cast( ).Skip( 1 ).First( group => group.Success ).Value.Trim( ) ) + .ToArray( ); + } + + // Repo packaging metadata is a release template; release/AUR automation writes + // the real source archive checksum after the tag exists. + private static void RequireTemplateChecksums( List errors, string label, string[] values ) + { + if ( values is [EnvironmentVariables.Skip] ) + { + return; + } + + if ( values.Length == 0 ) + { + errors.Add( $"{label}: expected SKIP template checksum, got missing" ); + return; + } + + var actual = string.Join( ", ", values ); + if ( values.Any( value => Regex.IsMatch( value, HashingConstants.Sha256HexPattern ) ) ) + { + errors.Add( $"{label}: expected SKIP template checksum, got fixed SHA {actual}; " + + "release/AUR automation writes the real checksum after the tag exists" ); + return; + } + + errors.Add( $"{label}: expected SKIP template checksum, got {actual}" ); + } + + private static void ValidateFlake( string flake, List errors ) + { + if ( !flake.Contains( "builtins.fromTOML (builtins.readFile ./Cargo.toml)", StringComparison.Ordinal ) ) + { + errors.Add( "flake.nix package version should be derived from Cargo.toml" ); + } + + string[] rustVersionGate = ["package.rust-version", "rustToolchain.version", "versionAtLeast"]; + if ( !rustVersionGate.All( token => flake.Contains( token, StringComparison.Ordinal ) ) ) + { + errors.Add( "flake.nix should compare the selected rustc against Cargo.toml rust-version" ); + } + } + + private static void ValidateReadme( string readme, List errors ) + { + foreach ( var (pattern, label) in ReadmePinPatterns ) + { + var pins = Regex.Matches( readme, pattern ).Select( match => match.Value ).Distinct( StringComparer.Ordinal ) + .Order( StringComparer.Ordinal ); + foreach ( var pin in pins ) + { + errors.Add( $"README.md: {label} '{pin}' goes stale on the next release; " + + "use a RELEASE_TAG placeholder or link to /releases/latest" ); + } + } + } + + private static bool IsReleaseVersion( string value ) => ReleaseVersion.TryParse( value, out _ ); + + private static bool IsHotfixOf( string value, string cargo ) => + ReleaseVersion.TryParse( value, out var version ) && version.IsHotfix && version.CargoVersion == cargo; + + private static string FormatList( IEnumerable values ) => + "[" + string.Join( ", ", values.Select( value => $"'{value}'" ) ) + "]"; + + private static string? LockVersion( string text, string package ) + { + var match = Regex.Match( text, + "(?ms)\\[\\[package\\]\\]\\s+name\\s*=\\s*\"" + Regex.Escape( package ) + "\"\\s+version\\s*=\\s*\"([^\"]+)\"" ); + return match.Success ? match.Groups[1].Value : null; + } + + private static string? ReadAssignment( string text, string name ) + { + var match = Regex.Match( text, $@"(?m)^{Regex.Escape( name )}=(.+)$" ); + return match.Success ? match.Groups[1].Value.Trim( ) : null; + } + + private static void RequireEqual( List errors, string label, string? actual, string expected ) + { + if ( actual != expected ) + { + errors.Add( $"{label}: expected {expected}, got {actual ?? "missing"}" ); + } + } +} diff --git a/tools/csharp/Infrastructure/RepositoryNames.cs b/tools/csharp/Infrastructure/RepositoryNames.cs index fcda8ca48..2e7d2cb36 100644 --- a/tools/csharp/Infrastructure/RepositoryNames.cs +++ b/tools/csharp/Infrastructure/RepositoryNames.cs @@ -6,9 +6,24 @@ internal static class RepositoryNames public const string BinaryPackage = "wayscriber-bin"; public const string ConfiguratorPackage = "wayscriber-configurator"; public const string Gtk4LayerShell = "gtk4-layer-shell"; + public const string FlakeFile = "flake.nix"; + public const string GlobalJsonFile = "global.json"; + public const string ReadmeFile = "README.md"; + public const string MainPackageConfigFile = "package.wayscriber.yaml"; + public const string ConfiguratorPackageConfigFile = "package.configurator.yaml"; public const string PackageBuildFile = "PKGBUILD"; public const string SourceInfoFile = ".SRCINFO"; public const string UserServiceFile = "wayscriber.service"; public const string ToolEntryFile = "wayscriber.cs"; public const string ToolTestEntryFile = "wayscriber.tests.cs"; + public const string ReleaseArchiveVersionPrefix = "-v"; + public const string ReleaseArchiveSuffix = "-linux-x86_64"; + public const string ReleaseArchiveExtension = ".tar.gz"; + + // `-v-linux-x86_64`: a release archive's top directory, and its name without the extension. + public static string ReleaseArchiveRoot( string package, string version ) => + $"{package}{ReleaseArchiveVersionPrefix}{version}{ReleaseArchiveSuffix}"; + + public static string ReleaseArchive( string package, string version ) => + ReleaseArchiveRoot( package, version ) + ReleaseArchiveExtension; } diff --git a/tools/csharp/Infrastructure/RustSource.cs b/tools/csharp/Infrastructure/RustSource.cs new file mode 100644 index 000000000..0c36827b3 --- /dev/null +++ b/tools/csharp/Infrastructure/RustSource.cs @@ -0,0 +1,141 @@ +using System.Text.RegularExpressions; + +namespace Wayscriber.Tools; + +// Masks Rust source for line-oriented reports: comments and literals become +// spaces, and `#[cfg(test)]` items can be removed, keeping every offset. +internal static class RustSource +{ + internal static string RemoveCfgTestBlocks( string text ) + { + var chars = text.ToCharArray( ); + foreach ( Match marker in Regex.Matches( text, @"#\[cfg\((?!\s*not\s*\(\s*test\s*\))(?=[^]]*\btest\b)[^]]*\)\]" ) ) + { + var opening = text.IndexOfAny( ['{', ';'], marker.Index + marker.Length ); + if ( opening < 0 ) + { + continue; + } + var end = opening + 1; + if ( text[opening] == '{' ) + { + var depth = 1; + while ( end < text.Length && depth > 0 ) + { + if ( text[end] == '{' ) + { + depth++; + } + else if ( text[end] == '}' ) + { + depth--; + } + + end++; + } + } + for ( var index = marker.Index; index < end; index++ ) + { + if ( chars[index] != '\n' ) + { + chars[index] = ' '; + } + } + } + return new string( chars ); + } + + internal static string StripRustCommentsAndStrings( string text ) + { + var output = text.ToCharArray( ); + var index = 0; + var blockDepth = 0; + while ( index < text.Length ) + { + if ( blockDepth == 0 && StartsWith( text, index, '/', '/' ) ) + { + MaskLineComment( text, output, ref index ); + } + else if ( StartsWith( text, index, '/', '*' ) ) + { + MaskPair( output, ref index ); + blockDepth++; + } + else if ( blockDepth > 0 ) + { + MaskBlockCommentCharacter( text, output, ref index, ref blockDepth ); + } + else if ( text[index] == '"' ) + { + MaskString( text, output, ref index ); + } + else + { + index++; + } + } + return new string( output ); + } + + private static bool StartsWith( string text, int index, char first, char second ) => + index + 1 < text.Length && text[index] == first && text[index + 1] == second; + + private static void MaskLineComment( string text, char[] output, ref int index ) + { + while ( index < text.Length && text[index] != '\n' ) + { + output[index++] = ' '; + } + } + + private static void MaskBlockCommentCharacter( string text, char[] output, ref int index, ref int blockDepth ) + { + if ( StartsWith( text, index, '*', '/' ) ) + { + MaskPair( output, ref index ); + blockDepth--; + return; + } + + if ( text[index] != '\n' ) + { + output[index] = ' '; + } + index++; + } + + private static void MaskPair( char[] output, ref int index ) + { + output[index++] = ' '; + output[index++] = ' '; + } + + private static void MaskString( string text, char[] output, ref int index ) + { + output[index++] = ' '; + while ( index < text.Length ) + { + var character = text[index]; + if ( character != '\n' ) + { + output[index] = ' '; + } + index++; + + if ( character == '\\' && index < text.Length ) + { + if ( text[index] != '\n' ) + { + output[index] = ' '; + } + index++; + continue; + } + + if ( character == '"' ) + { + return; + } + } + } +} diff --git a/tools/csharp/Infrastructure/SharedDependencyGuard.cs b/tools/csharp/Infrastructure/SharedDependencyGuard.cs deleted file mode 100644 index d63a81a35..000000000 --- a/tools/csharp/Infrastructure/SharedDependencyGuard.cs +++ /dev/null @@ -1,136 +0,0 @@ -using System.Text; -using System.Text.RegularExpressions; - -namespace Wayscriber.Tools; - -// Partial source guard: no alias resolution, macro expansion, or dependency graph. -internal static class SharedDependencyGuard -{ - private static readonly Regex NonCode = new( - "r(?#{0,16})\".*?\"\\k|\"(?:\\\\.|[^\"\\\\])*\"|'(?:\\\\.|[^'\\\\\\n])'|//[^\\n]*|/\\*", - RegexOptions.Singleline ); - private static readonly Regex Tokens = new( @"r#[A-Za-z_][A-Za-z_0-9]*|[A-Za-z_][A-Za-z_0-9]*|::|[{},;*]" ); - private static readonly Regex CommentMarkers = new( @"/\*|\*/" ); - - public static bool HasUpwardPath( string source, string relativePath, IReadOnlySet forbidden ) - { - var tokens = Tokens.Matches( StripNonCode( source ) ) - .Select( match => match.Value.StartsWith( "r#", StringComparison.Ordinal ) ? match.Value[2..] : match.Value ).ToArray( ); - var module = relativePath[..^3].Split( '/' ).Skip( 1 ).ToList( ); - if ( module[^1] == "mod" ) - { - module.RemoveAt( module.Count - 1 ); - } - - for ( var index = 0; index + 1 < tokens.Length; index++ ) - { - if ( tokens[index] is "crate" or "super" or "self" && tokens[index + 1] == "::" && - Traverse( tokens, index, module, forbidden ).Rejected ) - { - return true; - } - } - return false; - } - - private static (bool Rejected, int Index) Traverse( - string[] tokens, int index, List prefix, IReadOnlySet forbidden ) - { - var path = new List( prefix ); - while ( index < tokens.Length ) - { - var token = tokens[index]; - if ( token is "," or ";" or "}" or "as" ) - { - break; - } - if ( token == "{" ) - { - return TraverseGroup( tokens, index + 1, path, forbidden ); - } - if ( token == "crate" ) - { - path.Clear( ); - } - else if ( token == "super" && path.Count > 0 ) - { - path.RemoveAt( path.Count - 1 ); - } - else if ( token is not ("super" or "self" or "::" or "*") ) - { - path.Add( token ); - } - if ( path.Count > 0 && forbidden.Contains( path[0] ) ) - { - return (true, index); - } - index++; - if ( index >= tokens.Length || tokens[index] != "::" ) - { - break; - } - index++; - } - return (false, index); - } - - private static (bool Rejected, int Index) TraverseGroup( - string[] tokens, int index, List prefix, IReadOnlySet forbidden ) - { - while ( index < tokens.Length && tokens[index] != "}" ) - { - var result = Traverse( tokens, index, prefix, forbidden ); - if ( result.Rejected ) - { - return result; - } - index = result.Index; - if ( index < tokens.Length && tokens[index] == "as" ) - { - index += 2; - } - if ( index < tokens.Length && tokens[index] == "," ) - { - index++; - } - else if ( index < tokens.Length && tokens[index] != "}" ) - { - break; - } - } - return (false, index + 1); - } - - private static string StripNonCode( string source ) - { - var pieces = new StringBuilder( ); - var position = 0; - var match = NonCode.Match( source, position ); - while ( match.Success ) - { - pieces.Append( source, position, match.Index - position ); - position = match.Index + match.Length; - if ( match.Value == "/*" ) - { - position = SkipBlockComment( source, position ); - } - pieces.Append( ' ' ); - match = NonCode.Match( source, position ); - } - pieces.Append( source, position, source.Length - position ); - return pieces.ToString( ); - } - - private static int SkipBlockComment( string source, int position ) - { - var depth = 1; - var marker = CommentMarkers.Match( source, position ); - while ( depth > 0 && marker.Success ) - { - depth += marker.Value == "/*" ? 1 : -1; - position = marker.Index + marker.Length; - marker = CommentMarkers.Match( source, position ); - } - return depth > 0 ? source.Length : position; - } -} diff --git a/tools/csharp/Infrastructure/ToolConstants.cs b/tools/csharp/Infrastructure/ToolConstants.cs index 03d09b582..a9a186885 100644 --- a/tools/csharp/Infrastructure/ToolConstants.cs +++ b/tools/csharp/Infrastructure/ToolConstants.cs @@ -39,7 +39,6 @@ internal static class CommandNames public const string CheckLiveArchInstaller = "check-live-arch-installer"; public const string Clone = "clone"; public const string CodeHealth = "code-health"; - public const string ConfigWriters = "config-writers"; public const string Configurator = "configurator"; public const string ConfigureGit = "configure-git"; public const string CreateTag = "create-tag"; @@ -59,14 +58,12 @@ internal static class CommandNames public const string NixVersions = "nix-versions"; public const string PrepareGtk4LayerShell = "prepare-gtk4-layer-shell"; public const string PrepareSsh = "prepare-ssh"; - public const string ProcessSites = "process-sites"; public const string PublishTag = "publish-tag"; public const string ReloadDaemon = "reload-daemon"; public const string RequireEnvironment = "require-environment"; public const string ResolveVersion = "resolve-version"; public const string RustSourceCoverage = "rust-source-coverage"; public const string SetPortalShortcut = "set-portal-shortcut"; - public const string SharedDependencies = "shared-dependencies"; public const string SmokeUbuntu = "smoke-ubuntu"; public const string SourceChecksum = "source-checksum"; public const string Test = "test"; diff --git a/tools/csharp/includes.cs b/tools/csharp/includes.cs index 307e21582..72d40bf50 100644 --- a/tools/csharp/includes.cs +++ b/tools/csharp/includes.cs @@ -8,9 +8,10 @@ #:include Commands/DevelopmentCommands.cs #:include Commands/ChecksCommand.cs #:include Commands/VersionCommands.cs +#:include Commands/VersionConsistency.cs #:include Commands/NativeDesktopCommands.cs #:include Commands/PackagingCommands.cs #:include Commands/ReleaseAurCommands.cs #:include Commands/ReportCommands.cs -#:include Infrastructure/SharedDependencyGuard.cs +#:include Infrastructure/RustSource.cs diff --git a/tools/lint-and-test.sh b/tools/lint-and-test.sh index eaf298f6c..e0b388be1 100755 --- a/tools/lint-and-test.sh +++ b/tools/lint-and-test.sh @@ -12,29 +12,33 @@ run_check() { "$@" } -run_check bash tools/check-version-consistency.sh -run_check bash tools/test-package-repo-layout.sh -run_check bash tools/test-release-packaging.sh -run_check bash tools/test-aur-desktop-assets.sh - -if command -v dotnet >/dev/null 2>&1 && dotnet --version >/dev/null 2>&1; then - run_check dotnet build tools/wayscriber.cs --disable-build-servers --verbosity quiet - run_check dotnet build tools/install.cs --disable-build-servers --verbosity quiet - run_check dotnet build tools/wayscriber.tests.cs --disable-build-servers --verbosity quiet - for file_app in tools/wayscriber.cs tools/install.cs tools/wayscriber.tests.cs; do - run_check dotnet format style "$file_app" --no-restore --verify-no-changes - run_check dotnet format whitespace "$file_app" --no-restore --verify-no-changes - done - run_check dotnet run tools/wayscriber.tests.cs --no-build --verbosity quiet -else - printf '\nSkipping C# repository-tool checks: the SDK selected by global.json is unavailable.\n' +# Source invariants also run with Cargo alone, in `tests/repository_guards`. +# The release, packaging, and build-metadata checks exist once, in the C# tool +# CI runs, so the complete gate needs the .NET SDK that global.json selects. +if ! { command -v dotnet >/dev/null 2>&1 && dotnet --version >/dev/null 2>&1; }; then + printf 'error: the complete gate needs the .NET SDK selected by global.json.\n' >&2 + printf 'Without it, cargo test still runs the Rust source guards; that is Cargo validation only.\n' >&2 + exit 1 fi -run_check ./tools/check-nixpkgs-recipe.py -run_check ./tools/check-rust-source-coverage.py -run_check ./tools/check-process-sites.py -run_check ./tools/check-config-writers.py -run_check ./tools/check-shared-dependencies.py -run_check ./tools/test-shared-dependencies.py + +run_check dotnet build tools/wayscriber.cs --disable-build-servers --verbosity quiet +run_check dotnet build tools/install.cs --disable-build-servers --verbosity quiet +run_check dotnet build tools/wayscriber.tests.cs --disable-build-servers --verbosity quiet + +# From here on, the same steps in the same order as `ci lint-and-test`. The C# +# tests also run the retained shell contracts (`test-package-repo-layout.sh`, +# `test-release-packaging.sh`). +for tool_check in "assets check" "version check" "check nixpkgs-recipe" \ + "check rust-source-coverage" "check legacy-tools"; do + # shellcheck disable=SC2086 # Each entry is an area and a command. + run_check dotnet run tools/wayscriber.cs --no-build -- $tool_check +done +for file_app in tools/wayscriber.cs tools/install.cs tools/wayscriber.tests.cs; do + run_check dotnet format style "$file_app" --no-restore --verify-no-changes + run_check dotnet format whitespace "$file_app" --no-restore --verify-no-changes +done +run_check dotnet run tools/wayscriber.tests.cs --no-build --verbosity quiet + run_check cargo fmt --all -- --check run_check cargo clippy --locked --workspace --all-targets --all-features -- -D warnings run_check cargo build --locked --workspace --all-features --bins diff --git a/tools/publish-release-tag.sh b/tools/publish-release-tag.sh deleted file mode 100755 index d86111318..000000000 --- a/tools/publish-release-tag.sh +++ /dev/null @@ -1,69 +0,0 @@ -#!/usr/bin/env bash -# Create and push a release tag (vX.Y.Z) using the wayscriber crate version by default. - -set -euo pipefail - -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)" - -VERSION="" -DRY_RUN=0 - -usage() { - cat <<'EOF' -publish-release-tag.sh [--version X.Y.Z[.N]] [--dry-run] - -Creates an annotated git tag "v" and pushes it to origin. -If --version is omitted, uses the wayscriber crate version from Cargo metadata. -EOF -} - -while [[ $# -gt 0 ]]; do - case "$1" in - --version) VERSION="$2"; shift 2 ;; - --dry-run) DRY_RUN=1; shift ;; - -h|--help) usage; exit 0 ;; - *) echo "Unknown arg: $1" >&2; usage; exit 1 ;; - esac -done - -cd "$REPO_ROOT" - -if [[ -z "$VERSION" ]]; then - VERSION="$(cargo metadata --no-deps --format-version 1 | jq -r '.packages[] | select(.name=="wayscriber") | .version')" -fi - -if ! [[ "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+(\.[0-9]+)?$ ]]; then - echo "error: invalid version format: $VERSION (expected MAJOR.MINOR.PATCH[.HOTFIX])" >&2 - exit 1 -fi - -TAG="v${VERSION}" - -# Safety checks -if [[ -n "$(git status --porcelain)" ]]; then - echo "Working tree is dirty; aborting." >&2 - exit 1 -fi - -bash tools/check-version-consistency.sh --release-version "$VERSION" - -if git rev-parse "$TAG" >/dev/null 2>&1; then - echo "Tag $TAG already exists locally; aborting." >&2 - exit 1 -fi - -if git ls-remote --tags origin "$TAG" | grep -q "$TAG"; then - echo "Tag $TAG already exists on origin; aborting." >&2 - exit 1 -fi - -echo "Creating annotated tag ${TAG}" -if [[ "$DRY_RUN" -eq 0 ]]; then - git tag -a "$TAG" -m "Release $TAG" - echo "Pushing ${TAG} to origin" - git push origin "$TAG" -else - echo "[dry-run] git tag -a \"$TAG\" -m \"Release $TAG\"" - echo "[dry-run] git push origin \"$TAG\"" -fi diff --git a/tools/test-aur-desktop-assets.sh b/tools/test-aur-desktop-assets.sh deleted file mode 100755 index fa7bdb5f3..000000000 --- a/tools/test-aur-desktop-assets.sh +++ /dev/null @@ -1,81 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" -bash "$SCRIPT_DIR/aur-desktop-assets.sh" --check -WORK_DIR="$(mktemp -d)" -trap 'rm -rf "$WORK_DIR"' EXIT -mkdir -p "$WORK_DIR/tools" -cp "$SCRIPT_DIR/aur-desktop-assets.sh" "$SCRIPT_DIR/aur-desktop-assets.py" "$WORK_DIR/tools/" -cp -a "$REPO_ROOT/packaging" "$WORK_DIR/" - -# All concurrent standalone callers must receive complete, identical JSON. -pids=() -for run in 1 2 3; do - bash "$WORK_DIR/tools/aur-desktop-assets.sh" "$WORK_DIR" > "$WORK_DIR/$run.json" 2> "$WORK_DIR/$run.log" & - pids+=("$!") -done -for index in "${!pids[@]}"; do - if ! wait "${pids[$index]}"; then - echo "Concurrent cold asset generation failed for caller $((index + 1))" >&2 - cat "$WORK_DIR/$((index + 1)).log" >&2 - exit 1 - fi -done -for run in 1 2 3; do - jq -e '.source.lines | length > 0' "$WORK_DIR/$run.json" >/dev/null - cmp "$WORK_DIR/1.json" "$WORK_DIR/$run.json" || { - echo "Concurrent asset output differs for caller $run" >&2 - exit 1 - } -done - -write_manifest() { - cat > "$WORK_DIR/packaging/package.wayscriber.yaml" < "$WORK_DIR/failure.json" 2> "$WORK_DIR/failure.log"; then - echo "Expected asset generator failure: $1" >&2 - exit 1 - fi - grep -Fq "$1" "$WORK_DIR/failure.log" || { - echo "Expected diagnostic: $1" >&2 - cat "$WORK_DIR/failure.log" >&2 - exit 1 - } - [[ ! -s "$WORK_DIR/failure.json" ]] || { - echo 'Failed generation wrote partial JSON' >&2 - exit 1 - } -} - -for mode in 0644 420 0o644; do - write_manifest "$mode" - bash "$WORK_DIR/tools/aur-desktop-assets.sh" "$WORK_DIR" > "$WORK_DIR/mode-$mode.json" - cmp "$WORK_DIR/mode-0644.json" "$WORK_DIR/mode-$mode.json" || { - echo "Equivalent YAML integer mode $mode changed the recipe" >&2 - exit 1 - } -done -for mode in nonsense 0oBAD 999999999999999999999999 '"0644"' '!!str 0644' 644; do - write_manifest "$mode" - expect_failure 'package.wayscriber.yaml: desktop asset /usr/share/applications/wayscriber.desktop must have mode 0644' -done -write_manifest 0644 -sed -i '/file_info:/,+1d' "$WORK_DIR/packaging/package.wayscriber.yaml" -expect_failure 'package.wayscriber.yaml: desktop asset /usr/share/applications/wayscriber.desktop has no file_info mapping' -printf '{}\n' > "$WORK_DIR/packaging/package.wayscriber.yaml" -expect_failure 'package.wayscriber.yaml: expected one package contents sequence' - -echo 'AUR standalone asset concurrency and parser regression checks passed.' diff --git a/tools/test-fixtures/gtk_popup_wayland.py b/tools/test-fixtures/gtk_popup_wayland.py deleted file mode 100644 index e559ff5f0..000000000 --- a/tools/test-fixtures/gtk_popup_wayland.py +++ /dev/null @@ -1,277 +0,0 @@ -"""Run a real GTK popup behind a private Weston with one frame event withheld. - -All protocol messages and SCM_RIGHTS descriptors pass through unchanged, except -the selected popup callback and its delete_id event. The XML schemas provide -opcodes and new-object types; this fixture neither renders nor invents feedback. -""" - -import array -import json -import os -from pathlib import Path -import signal -import socket -import struct -import subprocess -import sys -import tempfile -import threading -import time -import xml.etree.ElementTree as ET - - -def schemas(): - interfaces = {} - for package, relative in (("wayland-client", "wayland.xml"), - ("wayland-protocols", "stable/xdg-shell/xdg-shell.xml")): - directory = subprocess.check_output( - ["pkg-config", "--variable=pkgdatadir", package], text=True - ).strip() - for interface in ET.parse(Path(directory) / relative).getroot().findall("interface"): - interfaces[interface.attrib["name"]] = { - direction: interface.findall(tag) - for direction, tag in (("request", "request"), ("event", "event")) - } - return interfaces - - -def arguments(message, payload): - values = {} - offset = 0 - for argument in message.findall("arg"): - kind = argument.attrib["type"] - if kind == "fd": - continue - value = struct.unpack_from("=I", payload, offset)[0] - offset += 4 - if kind in ("string", "array"): - raw = payload[offset:offset + value] - offset += (value + 3) & ~3 - value = raw.rstrip(b"\0").decode() if kind == "string" else raw - elif kind == "new_id" and "interface" not in argument.attrib: - # A dynamic new_id carries its interface name, version, then ID. - interface = payload[offset:offset + value].rstrip(b"\0").decode() - offset += (value + 3) & ~3 - version, value = struct.unpack_from("=II", payload, offset) - offset += 8 - values["interface"] = interface - values["version"] = version - values[argument.attrib["name"]] = value - return values - - -def receive(sock): - data = bytearray() - fds = array.array("i") - size = 8 - while len(data) < size: - chunk, ancillary, flags, _ = sock.recvmsg( - size - len(data), socket.CMSG_SPACE(256 * fds.itemsize) - ) - if not chunk: - for fd in fds: - os.close(fd) - return None - if flags & socket.MSG_CTRUNC: - raise RuntimeError("truncated Wayland descriptors") - data.extend(chunk) - for level, kind, raw in ancillary: - if level == socket.SOL_SOCKET and kind == socket.SCM_RIGHTS: - fds.frombytes(raw) - if len(data) == 8: - size = struct.unpack_from("=I", data, 4)[0] >> 16 - if size < 8 or size % 4: - raise RuntimeError("invalid Wayland message length") - return bytes(data), fds - - -def send(sock, packet): - data, fds = packet - try: - ancillary = [(socket.SOL_SOCKET, socket.SCM_RIGHTS, fds)] if fds else [] - sent = sock.sendmsg([data], ancillary) - sock.sendall(data[sent:]) - finally: - for fd in fds: - os.close(fd) - - -class Connection: - def __init__(self, client, upstream, interfaces): - self.client = client - self.server = socket.socket(socket.AF_UNIX) - self.server.connect(upstream) - self.interfaces = interfaces - self.objects = {1: "wl_display"} - self.xdg_surfaces = {} - self.popups = set() - self.callbacks = {} - self.lock = threading.RLock() - self.armed = False - self.selected = None - self.held = [] - self.commits = 0 - self.delivered = 0 - self.error = None - - def start(self): - for source, destination, direction in ( - (self.client, self.server, "request"), (self.server, self.client, "event") - ): - threading.Thread(target=self.forward, args=(source, destination, direction), - daemon=True).start() - - def inspect(self, packet, direction): - data, _ = packet - object_id, header = struct.unpack_from("=II", data) - interface = self.objects.get(object_id) - if interface not in self.interfaces: - return False - message = self.interfaces[interface][direction][header & 0xffff] - values = arguments(message, data[8:]) - for argument in message.findall("arg"): - if argument.attrib["type"] == "new_id": - self.objects[values[argument.attrib["name"]]] = argument.attrib.get( - "interface", values.get("interface") - ) - name = message.attrib["name"] - - if direction == "request": - if interface == "xdg_wm_base" and name == "get_xdg_surface": - self.xdg_surfaces[values["id"]] = values["surface"] - elif interface == "xdg_surface" and name == "get_popup": - self.popups.add(self.xdg_surfaces[object_id]) - elif interface == "wl_surface" and name == "frame": - self.callbacks[values["callback"]] = object_id - if self.armed and object_id in self.popups: - self.selected = values["callback"] - self.armed = False - elif interface == "wl_surface" and name == "commit" and object_id in self.popups: - self.commits += 1 - return False - - if interface == "wl_callback" and name == "done": - if object_id == self.selected: - return True - if self.callbacks.pop(object_id, None) in self.popups: - self.delivered += 1 - elif interface == "wl_display" and name == "delete_id": - if values["id"] == self.selected: - return True - self.objects.pop(values["id"], None) - return False - - def forward(self, source, destination, direction): - try: - while (packet := receive(source)) is not None: - with self.lock: - if self.inspect(packet, direction): - self.held.append(packet) - else: - send(destination, packet) - except Exception as error: - with self.lock: - self.error = f"{type(error).__name__}: {error}" - - def control(self, command): - with self.lock: - if self.error: - return {"error": self.error} - if command == "arm": - self.armed = True - self.commits = 0 - self.delivered = 0 - elif command == "release": - if not self.held: - return {"error": "no popup callback held"} - self.selected = None - for packet in self.held: - self.inspect(packet, "event") - send(self.client, packet) - self.held.clear() - elif command != "status": - return {"error": "unknown control command"} - return {"held": bool(self.held), "popup_commits": self.commits, - "popup_callbacks_delivered": self.delivered} - - -def listener(path): - sock = socket.socket(socket.AF_UNIX) - sock.bind(str(path)) - sock.listen() - return sock - - -def run_fixture(directory, executable, test_name): - environment = dict(os.environ, XDG_RUNTIME_DIR=directory, WAYLAND_DISPLAY="upstream", - GDK_BACKEND="wayland", GSK_RENDERER="gl", LIBGL_ALWAYS_SOFTWARE="1", - GTK_A11Y="test", GDK_DEBUG="no-portals") - environment.pop("DISPLAY", None) - environment.pop("WAYLAND_SOCKET", None) - environment.pop("DBUS_SESSION_BUS_ADDRESS", None) - with open(Path(directory) / "weston.log", "w+") as log: - weston = subprocess.Popen( - ["weston", "--backend=headless-backend.so", "--renderer=pixman", "--no-config", - "--socket=upstream", "--idle-time=0"], env=environment, stdout=log, stderr=log - ) - try: - upstream = str(Path(directory) / "upstream") - deadline = time.monotonic() + 10 - while not Path(upstream).is_socket(): - if weston.poll() is not None or time.monotonic() >= deadline: - log.seek(0) - raise RuntimeError(log.read()) - time.sleep(0.01) - interfaces = schemas() - proxy = listener(Path(directory) / "proxy") - control = listener(Path(directory) / "control") - connections = [] - - def accept_clients(): - while True: - client, _ = proxy.accept() - connection = Connection(client, upstream, interfaces) - connections.append(connection) - connection.start() - - def accept_control(): - while True: - client, _ = control.accept() - with client, client.makefile("r") as reader: - command = reader.readline().strip() - popup_connections = [connection for connection in connections - if connection.popups] - response = popup_connections[-1].control(command) if popup_connections else { - "error": "no GTK popup connection" - } - client.sendall((json.dumps(response) + "\n").encode()) - - threading.Thread(target=accept_clients, daemon=True).start() - threading.Thread(target=accept_control, daemon=True).start() - environment.update(WAYLAND_DISPLAY="proxy", G_DEBUG="fatal-criticals", - WAYSCRIBER_GTK_WAYLAND_CONTROL=str(Path(directory) / "control")) - child = subprocess.Popen( - ["dbus-run-session", "--", executable, test_name, "--exact", "--test-threads=1", - "--nocapture"], env=environment, start_new_session=True - ) - try: - return child.wait(timeout=20) - finally: - # Includes the private bus and any services it activated. - try: - os.killpg(child.pid, signal.SIGKILL) - except ProcessLookupError: - pass - child.wait() - finally: - weston.terminate() - try: - weston.wait(timeout=3) - except subprocess.TimeoutExpired: - weston.kill() - weston.wait() - - -if __name__ == "__main__": - with tempfile.TemporaryDirectory(prefix="wayscriber-popup-test-") as directory: - raise SystemExit(run_fixture(directory, *sys.argv[1:])) diff --git a/tools/test-release-packaging.sh b/tools/test-release-packaging.sh index 1beff5634..774415893 100755 --- a/tools/test-release-packaging.sh +++ b/tools/test-release-packaging.sh @@ -12,7 +12,6 @@ ARCH_INSTALLER_CHECKER="${REPO_ROOT}/tools/check-arch-installer-manifest.sh" INSTALL_SCRIPT="${REPO_ROOT}/tools/install-gtk4-layer-shell.sh" CONFIGURATOR_INSTALL_SCRIPT="${REPO_ROOT}/tools/install-configurator.sh" STATIC_LINK_VERIFIER="${REPO_ROOT}/tools/verify-static-gtk4-layer-shell.sh" -VERSION_CHECKER="${REPO_ROOT}/tools/check-version-consistency.sh" WORK_DIR="$(mktemp -d)" cleanup() { @@ -133,27 +132,6 @@ assert_contains "${CONFIG_INSTALL_SUDO_LOG}" "${CONFIG_INSTALL_PRIVILEGED_DATA}" assert_not_contains "${CONFIG_INSTALL_SUDO_LOG}" "${CONFIG_INSTALL_MIXED_BIN}" test -x "${CONFIG_INSTALL_MIXED_BIN}/wayscriber-configurator" -VERSION_FAILURE_COUNT=0 -expect_version_consistency_failure() { - local expected="$1" fixture_root="$2" - VERSION_FAILURE_COUNT=$((VERSION_FAILURE_COUNT + 1)) - local output="${WORK_DIR}/version-failure-${VERSION_FAILURE_COUNT}.log" - - set +e - ( - cd "${fixture_root}" - bash tools/check-version-consistency.sh - ) >"${output}" 2>&1 - local status=$? - set -e - - if [[ ${status} -eq 0 ]]; then - echo "Expected version consistency check to fail: ${expected}" >&2 - exit 1 - fi - assert_contains "${output}" "${expected}" -} - # Trimming trailing separators must not turn the filesystem root into the # invocation directory. Discard the deliberately privileged writes after sudo # records their normalized destinations. @@ -206,151 +184,6 @@ assert_contains "${CONFIGURATOR_PACKAGE_CONFIG}" "- glibc >= 2.39" assert_contains "${CONFIGURATOR_PACKAGE_CONFIG}" "- libadwaita-1-0 (>= 1.4)" assert_contains "${CONFIGURATOR_PACKAGE_CONFIG}" "- libadwaita >= 1.4" -# One supported libadwaita floor governs compilation and every package -# channel. Exercise the real consistency checker, then mutation-check each -# independently editable metadata surface so a partial floor bump cannot pass. -LIBADWAITA_FLOOR_REPO="${WORK_DIR}/libadwaita-floor-repo" -mkdir -p \ - "${LIBADWAITA_FLOOR_REPO}/.github/workflows" \ - "${LIBADWAITA_FLOOR_REPO}/configurator" \ - "${LIBADWAITA_FLOOR_REPO}/packaging" \ - "${LIBADWAITA_FLOOR_REPO}/tools" -cp "${REPO_ROOT}/Cargo.toml" \ - "${REPO_ROOT}/Cargo.lock" \ - "${REPO_ROOT}/README.md" \ - "${REPO_ROOT}/flake.nix" \ - "${REPO_ROOT}/global.json" \ - "${LIBADWAITA_FLOOR_REPO}/" -cp "${REPO_ROOT}/configurator/Cargo.toml" \ - "${LIBADWAITA_FLOOR_REPO}/configurator/Cargo.toml" -cp "${REPO_ROOT}/packaging/PKGBUILD" \ - "${REPO_ROOT}/packaging/.SRCINFO" \ - "${REPO_ROOT}/packaging/package.configurator.yaml" \ - "${LIBADWAITA_FLOOR_REPO}/packaging/" -cp "${VERSION_CHECKER}" \ - "${LIBADWAITA_FLOOR_REPO}/tools/check-version-consistency.sh" -cp "${RELEASE_WORKFLOW}" \ - "${LIBADWAITA_FLOOR_REPO}/.github/workflows/build-packages.yml" -cp "${REPO_ROOT}/tools/update-aur-from-manifest.sh" \ - "${LIBADWAITA_FLOOR_REPO}/tools/update-aur-from-manifest.sh" -( - cd "${LIBADWAITA_FLOOR_REPO}" - bash tools/check-version-consistency.sh >/dev/null -) - -printf '%s\n' '{"sdk":{"version":"11.0.100-rc.1.26425.128","rollForward":"disable","allowPrerelease":true}}' \ - > "${LIBADWAITA_FLOOR_REPO}/global.json" -( - cd "${LIBADWAITA_FLOOR_REPO}" - bash tools/check-version-consistency.sh >/dev/null -) - -printf '%s\n' '{"sdk":{}}' > "${LIBADWAITA_FLOOR_REPO}/global.json" -expect_version_consistency_failure \ - "global.json SDK metadata is invalid" \ - "${LIBADWAITA_FLOOR_REPO}" - -printf '%s\n' '{"sdk":{"version":"11.0.100-rc.1.26425.127","rollForward":"disable","allowPrerelease":true}}' \ - > "${LIBADWAITA_FLOOR_REPO}/global.json" -expect_version_consistency_failure \ - "global.json SDK: expected 11.0.100-rc.1.26425.128, got 11.0.100-rc.1.26425.127" \ - "${LIBADWAITA_FLOOR_REPO}" - -printf '%s\n' '{"sdk":{"version":"11.0.100-rc.1.26425.128","rollForward":"latestPatch","allowPrerelease":false}}' \ - > "${LIBADWAITA_FLOOR_REPO}/global.json" -expect_version_consistency_failure \ - "global.json SDK rollForward must be disable" \ - "${LIBADWAITA_FLOOR_REPO}" -expect_version_consistency_failure \ - "global.json SDK allowPrerelease must be true" \ - "${LIBADWAITA_FLOOR_REPO}" -cp "${REPO_ROOT}/global.json" "${LIBADWAITA_FLOOR_REPO}/global.json" - -sed -i 's/features = \["v1_4"\]/features = ["v1_5"]/' \ - "${LIBADWAITA_FLOOR_REPO}/configurator/Cargo.toml" -expect_version_consistency_failure \ - "configurator/Cargo.toml libadwaita features: expected ['v1_4'], got ['v1_5']" \ - "${LIBADWAITA_FLOOR_REPO}" -cp "${REPO_ROOT}/configurator/Cargo.toml" \ - "${LIBADWAITA_FLOOR_REPO}/configurator/Cargo.toml" - -sed -i 's/libadwaita-1-0 (>= 1.4)/libadwaita-1-0 (>= 1.5)/' \ - "${LIBADWAITA_FLOOR_REPO}/packaging/package.configurator.yaml" -expect_version_consistency_failure \ - "configurator deb libadwaita floor: expected 1.4, got 1.5" \ - "${LIBADWAITA_FLOOR_REPO}" -cp "${REPO_ROOT}/packaging/package.configurator.yaml" \ - "${LIBADWAITA_FLOOR_REPO}/packaging/package.configurator.yaml" - -sed -i 's/libadwaita >= 1.4/libadwaita >= 1.5/' \ - "${LIBADWAITA_FLOOR_REPO}/packaging/package.configurator.yaml" -expect_version_consistency_failure \ - "configurator rpm libadwaita floor: expected 1.4, got 1.5" \ - "${LIBADWAITA_FLOOR_REPO}" -cp "${REPO_ROOT}/packaging/package.configurator.yaml" \ - "${LIBADWAITA_FLOOR_REPO}/packaging/package.configurator.yaml" - -sed -i 's/libadwaita>=1.4/libadwaita>=1.5/' \ - "${LIBADWAITA_FLOOR_REPO}/packaging/PKGBUILD" -expect_version_consistency_failure \ - "packaging/PKGBUILD libadwaita floor: expected 1.4, got 1.5" \ - "${LIBADWAITA_FLOOR_REPO}" -cp "${REPO_ROOT}/packaging/PKGBUILD" \ - "${LIBADWAITA_FLOOR_REPO}/packaging/PKGBUILD" - -sed -i 's/libadwaita>=1.4/libadwaita>=1.5/' \ - "${LIBADWAITA_FLOOR_REPO}/packaging/.SRCINFO" -expect_version_consistency_failure \ - "packaging/.SRCINFO libadwaita floor: expected 1.4, got 1.5" \ - "${LIBADWAITA_FLOOR_REPO}" - -cp "${REPO_ROOT}/packaging/.SRCINFO" \ - "${LIBADWAITA_FLOOR_REPO}/packaging/.SRCINFO" - -sed -i \ - "s/ensure_runtime_dependency 'libadwaita>=1.4'/ensure_runtime_dependency 'libadwaita>=1.5'/" \ - "${LIBADWAITA_FLOOR_REPO}/tools/update-aur-from-manifest.sh" -expect_version_consistency_failure \ - "AUR updater generated libadwaita floor: expected 1.4, got 1.5" \ - "${LIBADWAITA_FLOOR_REPO}" -cp "${REPO_ROOT}/tools/update-aur-from-manifest.sh" \ - "${LIBADWAITA_FLOOR_REPO}/tools/update-aur-from-manifest.sh" - -sed -i '/grep -Eq.*libadwaita/ s/libadwaita>=1.4/libadwaita>=1.5/' \ - "${LIBADWAITA_FLOOR_REPO}/tools/update-aur-from-manifest.sh" -expect_version_consistency_failure \ - "AUR updater PKGBUILD validation floor: expected 1.4, got 1.5" \ - "${LIBADWAITA_FLOOR_REPO}" -cp "${REPO_ROOT}/tools/update-aur-from-manifest.sh" \ - "${LIBADWAITA_FLOOR_REPO}/tools/update-aur-from-manifest.sh" - -sed -i '/grep -Fxq.*depends = libadwaita/ s/libadwaita>=1.4/libadwaita>=1.5/' \ - "${LIBADWAITA_FLOOR_REPO}/tools/update-aur-from-manifest.sh" -expect_version_consistency_failure \ - "AUR updater .SRCINFO validation floor: expected 1.4, got 1.5" \ - "${LIBADWAITA_FLOOR_REPO}" - -# Updating every metadata floor is still incomplete until the package runner -# moves to a reviewed release-platform contract that supports the new ABI. -cp "${REPO_ROOT}/packaging/.SRCINFO" \ - "${LIBADWAITA_FLOOR_REPO}/packaging/.SRCINFO" -sed -i 's/supported_libadwaita_floor = "1.4"/supported_libadwaita_floor = "1.7"/' \ - "${LIBADWAITA_FLOOR_REPO}/tools/check-version-consistency.sh" -sed -i 's/features = \["v1_4"\]/features = ["v1_7"]/' \ - "${LIBADWAITA_FLOOR_REPO}/configurator/Cargo.toml" -sed -i 's/1\.4/1.7/g' \ - "${LIBADWAITA_FLOOR_REPO}/packaging/PKGBUILD" \ - "${LIBADWAITA_FLOOR_REPO}/packaging/.SRCINFO" \ - "${LIBADWAITA_FLOOR_REPO}/packaging/package.configurator.yaml" \ - "${LIBADWAITA_FLOOR_REPO}/.github/workflows/build-packages.yml" -cp "${REPO_ROOT}/tools/update-aur-from-manifest.sh" \ - "${LIBADWAITA_FLOOR_REPO}/tools/update-aur-from-manifest.sh" -sed -i 's/1\.4/1.7/g' \ - "${LIBADWAITA_FLOOR_REPO}/tools/update-aur-from-manifest.sh" -expect_version_consistency_failure \ - "release package runner ubuntu-24.04 libadwaita floor: expected 1.4, got 1.7" \ - "${LIBADWAITA_FLOOR_REPO}" - # Release jobs must not silently raise the glibc floor when ubuntu-latest # changes; the package job is the binary floor's defining runner. sed -n '/^ package:/,/^ package-repos:/p' "${RELEASE_WORKFLOW}" \ @@ -1694,84 +1527,4 @@ run_aur_updater "${WORK_DIR}" \ exit 1 } -# A version bump must retain a locked registry dependency even when a newer -# compatible release exists. A local directory source makes this fully offline. -BUMP_REPO="${WORK_DIR}/bump-repo" -BUMP_FAKE_BIN="${WORK_DIR}/bump-fake-bin" -mkdir -p "${BUMP_REPO}/tools" "${BUMP_REPO}/src" \ - "${BUMP_REPO}/configurator/src" "${BUMP_REPO}/packaging" \ - "${BUMP_REPO}/.cargo" "${BUMP_FAKE_BIN}" -cp "${REPO_ROOT}/tools/bump-version.sh" "${BUMP_REPO}/tools/" -# This test isolates the bump's Cargo operation; real metadata consistency is -# checked separately by the canonical gate on this repository. -printf '#!/usr/bin/env bash\nexit 0\n' > "${BUMP_REPO}/tools/check-version-consistency.sh" -printf '#!/usr/bin/env bash\nprintf "pkgbase = wayscriber\\n"\n' > "${BUMP_FAKE_BIN}/makepkg" -chmod +x "${BUMP_FAKE_BIN}/makepkg" -cat > "${BUMP_REPO}/Cargo.toml" <<'EOF' -[package] -name = "wayscriber" -version = "1.0.0" -edition = "2024" -[workspace] -members = ["configurator"] -[dependencies] -release-fixture = "1" -EOF -cat > "${BUMP_REPO}/configurator/Cargo.toml" <<'EOF' -[package] -name = "wayscriber-configurator" -version = "1.0.0" -edition = "2024" -[dependencies] -wayscriber = { path = ".." } -EOF -touch "${BUMP_REPO}/src/lib.rs" "${BUMP_REPO}/configurator/src/lib.rs" -printf "pkgver=1.0.0\nsha256sums=('SKIP')\n" > "${BUMP_REPO}/packaging/PKGBUILD" -cat > "${BUMP_REPO}/.cargo/config.toml" < "${fixture_dir}/Cargo.toml" - touch "${fixture_dir}/src/lib.rs" - printf '{"files":{},"package":"%s"}\n' "${AUR_SOURCE_SHA}" > "${fixture_dir}/.cargo-checksum.json" - if [[ "$fixture_version" == 1.0.0 ]]; then - (cd "${BUMP_REPO}" && cargo generate-lockfile --offline) - fi -done -BUMP_FAIL_BIN="${WORK_DIR}/bump-fail-bin" -BUMP_DRY_RUN_OUTPUT="${WORK_DIR}/bump-dry-run-output" -mkdir -p "${BUMP_FAIL_BIN}" -cat > "${BUMP_FAIL_BIN}/cargo" <<'EOF' -#!/usr/bin/env bash -if [[ "$*" == "update --workspace --offline --dry-run" ]]; then - exit 1 -fi -exec "${BUMP_REAL_CARGO:?}" "$@" -EOF -chmod +x "${BUMP_FAIL_BIN}/cargo" -if BUMP_REAL_CARGO="$(command -v cargo)" PATH="${BUMP_FAIL_BIN}:${BUMP_FAKE_BIN}:${PATH}" \ - bash "${BUMP_REPO}/tools/bump-version.sh" --dry-run 1.0.1 >"${BUMP_DRY_RUN_OUTPUT}" 2>&1; then - echo 'Expected the version-bump dry run to fail when offline resolution fails' >&2 - exit 1 -fi -assert_contains "${BUMP_DRY_RUN_OUTPUT}" 'cannot resolve locked dependencies offline' - -cp "${BUMP_REPO}/Cargo.lock" "${WORK_DIR}/before-bump.lock" -for version in 1.0.1 1.0.1.1; do - PATH="${BUMP_FAKE_BIN}:${PATH}" bash "${BUMP_REPO}/tools/bump-version.sh" "$version" >/dev/null - # Both workspace packages change, but all remaining bytes must stay put. - sed 's/version = "1.0.1"/version = "1.0.0"/g' "${BUMP_REPO}/Cargo.lock" \ - > "${WORK_DIR}/normalized-bump.lock" - cmp "${WORK_DIR}/before-bump.lock" "${WORK_DIR}/normalized-bump.lock" -done -# Prove the fixture exposes the original bug: resolving afresh selects 1.1.0. -(cd "${BUMP_REPO}" && cargo generate-lockfile --offline) -assert_contains "${BUMP_REPO}/Cargo.lock" 'version = "1.1.0"' - echo "Release packaging contract checks passed." diff --git a/tools/test-shared-dependencies.py b/tools/test-shared-dependencies.py deleted file mode 100755 index 00d17fc03..000000000 --- a/tools/test-shared-dependencies.py +++ /dev/null @@ -1,23 +0,0 @@ -#!/usr/bin/env python3 -"""Exercise the standalone guard's actual command against shared syntax fixtures.""" -import json -from pathlib import Path -import shutil -import subprocess -import tempfile - -TOOLS = Path(__file__).resolve().parent -fixtures = json.loads((TOOLS / "shared-dependency-fixtures.json").read_text()) -for fixture in fixtures: - with tempfile.TemporaryDirectory(prefix="wayscriber-shared-dependencies-") as directory: - root = Path(directory) - (root / "tools").mkdir() - checker = root / "tools/check-shared-dependencies.py" - shutil.copyfile(TOOLS / checker.name, checker) - source = root / fixture["path"] - source.parent.mkdir(parents=True) - source.write_text(fixture["source"]) - result = subprocess.run(["python3", str(checker)], capture_output=True, text=True) - expected = 1 if fixture["reject"] else 0 - assert result.returncode == expected, (fixture["name"], result.returncode, result.stderr) -print(f"Standalone shared-dependency syntax fixtures passed ({len(fixtures)}).")