Skip to content

testsuite, ci: rework the test matrix and expand coverage - #1122

Merged
steadytao merged 193 commits into
masterfrom
rework-workflows
Nov 14, 2026
Merged

steadytao merged 193 commits into
masterfrom
rework-workflows

Conversation

@steadytao

@steadytao steadytao commented Oct 8, 2026 •

Copy link
Copy Markdown
Member

Bluntly. This PR is way too large for someone to review, been working on it for a while. I assume no one will care enough to review either 🥲 But nonetheless. Proper description.

Summary

Rebuilds the testsuite around a Python 3.7 standard-library-only harness and replaces the fragmented workflows with one CI entry point. Tests now declare their transports, protocol limits, peer-version limits and required capabilities. Platform profiles distinguish tests which do not apply from capabilities a system lacks. Every run produces a machine-readable receipt which records the selected tests, outcomes, environment and tested binaries.

Test harness

  • Organises behavioural tests under build, daemon, metadata, path, protocol, rrsync, selftests and transfer groups.
  • Moves reusable process, filesystem, daemon, protocol, metadata and fixture handling into the harness.
  • Removes Fleettest, obsolete wrappers, duplicated tests and the old root-level test layout.
  • Runs tests in parallel with bounded process cleanup and retained failure evidence.
  • Checks unsupported results against explicit platform capabilities rather than broad skip lists.
  • Compiles every tracked Python source and runs the helper and harness regressions under Python 3.7.
  • Verifies historical peer binaries against recorded hashes, release commits, build inputs and version banners.
  • Exercises releases from 2.6.0 through 3.5.2 in both executable roles over pipe and TCP.

CI

  • Consolidates the previous independent workflows under CI / Required.
  • Covers Ubuntu, AlmaLinux, Alpine, FreeBSD, NetBSD, OpenBSD, macOS, Solaris, Cygwin and Android.
  • Runs native X86-32, X86-64, ARM64 and ARMv7 coverage where applicable.
  • Executes Android binaries under Bionic, an Android emulator and the Termux application sandbox.
  • Runs ASan, UBSan, Clang 21 static analysis, coverage and Valgrind.
  • Exercises installed rsync, restricted SSH and local TLS with certificate and hostname verification.
  • Qualifies encrypted storage, mount boundaries, space exhaustion, device transfers and --copy-as using disposable filesystems.
  • Builds and tests the release source archive without Git or developer generators.
  • Retains logs, receipts and platform evidence as workflow artefacts.

Pull requests select checks from their changed paths. Documentation-only changes run Changes and Required. Testsuite and CI changes run their owning platforms and diagnostics. Production, build, packaging and unknown paths run the complete matrix. Pushes to master and manual runs also select the complete matrix.

Required rejects a failed selector, a missing family, an unexpected skip, a failed child job or a selected job which did not run.

Interoperability

Seven native endpoints cover Linux, FreeBSD, NetBSD, OpenBSD, macOS, Solaris and Cygwin.

Every ordered platform pair is exercised through push and pull operations over SSH and daemon transports, giving 168 directed data-transfer cases. The same system also checks updates, deletion, ACL and xattr handling, unsupported metadata, fake-super backup restoration and network isolation.

Endpoint jobs have read-only GitHub permissions and no federated identity. A trusted controller issues encrypted single-use Tailscale enrolment keys, admits only the recorded cohort then revokes its grant, keys and devices. Cleanup is independently recoverable from an attested lease.

Startup waits account for hosted-runner queueing. Interoperability SSH uses bounded connection retries and Cygwin restores Windows Git before checkout cleanup.

Release integration

Release archive construction now has one implementation shared by the release process and archive qualification. Generated files are validated before packaging and archived peer binaries remain excluded from release tarballs. make check, protocol-specific checks, coverage and installed-binary checks use the new runner directly.

@steadytao
steadytao force-pushed the rework-workflows branch 5 times, most recently from 3ae6b62 to 1cfbbe8 Compare October 8, 2026 17:11
@steadytao steadytao changed the title testsuite, ci: expand coverage and rework the test matrix testsuite, ci: test matrix rework and coverage expansion Nov 10, 2026
@steadytao
steadytao force-pushed the rework-workflows branch 2 times, most recently from a684dc5 to 4932fcf Compare November 13, 2026 13:00
@steadytao steadytao changed the title testsuite, ci: test matrix rework and coverage expansion testsuite, ci: rework the test matrix and expand coverage Nov 13, 2026
@steadytao
steadytao merged commit 570af7a into master Nov 14, 2026
42 checks passed
@steadytao
steadytao deleted the rework-workflows branch November 14, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant