Skip to content

test: keep copied-image daemon tests CPU-independent under sanitizers - #2987

Open
DeusData wants to merge 2 commits into
mainfrom
fix/test-sha256-portable-hash-budget
Open

DeusData wants to merge 2 commits into
mainfrom
fix/test-sha256-portable-hash-budget

Conversation

@DeusData

Copy link
Copy Markdown
Owner

Summary

Makes the copied-image daemon tests independent of which CPU a hosted runner
draws. This is a test/CI-behaviour change. Release binaries are
unaffected, and the daemon's admission logic is not touched.

  1. src/foundation/sha256.c: the portable block transform
    (sha256_transform, and only that function) is compiled without
    ASan/UBSan instrumentation via no_sanitize("address", "undefined").
    It carries a WHY comment with the measured cost. cbm_sha256_update /
    cbm_sha256_final, the portable block loop that calls the transform, and
    the x86 SHA-NI / ARMv8 SHA2 backends all stay instrumented. An
    unsanitized (release) build compiles to the same code.
  2. tests/test_main.c: when the daemon never answered within the
    exchange budget, the copied-image HELLO probe now exits 30 and the
    activation probe exits 31. Exit 26 / 29 therefore mean only
    that the daemon answered with a refusal. The daemon_runtime assertions
    keep their expected values (0 for the accept cases, 26 for the rejection
    cases), and a comment at each rejection assertion now records that a
    timeout can no longer satisfy it.

Why

  • src/foundation/sha256.c hashes on x86 SHA-NI or ARMv8 SHA2 when the CPU
    has them, and on its portable transform otherwise. Some hosted x86 runners
    have no SHA-NI (CPUID leaf 7, EBX bit 29).
  • The sanitized test runner is about 785 MB. For every copied-image peer,
    the daemon fingerprints the peer's executable before it answers. On a
    runner without SHA-NI, one such fingerprint took about 72-92 s on the
    instrumented portable transform, against under 2 s on SHA-NI runners.
  • The probes' exchange budget (TF_RUNTIME_IMAGE_EXCHANGE_TIMEOUT_MS) is
    60 s. On those runners the identical-copy HELLO test therefore exited 26
    and the foreign activation test exited 29. The cli suite (66 full-image
    fingerprints) ran into its 3600 s suite ceiling. The runner image version
    did not decide the outcome; the CPU the job drew did.
  • The same timeout let the changed-copy rejection assertion pass for the
    wrong reason: "no answer yet" and "answered: rejected" both exited 26.
    With this change a slow fingerprint fails loudly as itself.

Measured cost of the instrumentation (one full hash, same input, digests identical)

toolchain / CPU build origin/main this PR
GCC 13.3 x86-64 (no SHA-NI; Rosetta) ASan+UBSan -O1 15.6 s 3.0 s (5.3x)
GCC 13.3 aarch64, portable forced ASan+UBSan -O1 5.33 s 2.18 s (2.4x)
Apple clang 21 arm64, portable forced (456 MB input) ASan+UBSan -O1 2.48 s 1.15 s (2.2x)
GCC 13.3 x86-64 / aarch64 release -O2 2.59 s / 3.70 s 2.60 s / 3.69 s (unchanged)
hardware backend (ARMv8 SHA2) ASan+UBSan -O1 0.66 s 0.66 s (unchanged)

The input was the 786 MB sanitized Linux test runner, or the 456 MB macOS
runner on the clang row. Applying the x86-64 ratio to the hosted
observation puts a no-SHA-NI fingerprint at roughly 14-18 s, inside the
60 s budget.

Verification

RED/GREEN under the failing condition, reproduced locally. Setup: Linux
arm64, GCC 13.3, the repository's test image, and the 786 MB sanitized test
runner. That hardware has SHA2, so a proof-only patch (never committed)
recreates a no-SHA-NI runner: it forces the portable backend and scales the
exchange budget from 60 s to 4 s. One local portable fingerprint costs 5.3 s
against 4 s, the same ratio as hosted 72-92 s against 60 s. Line numbers are
origin/main's.

tree backend, 4 s budget daemon_runtime
origin/main hardware 55/55 pass (25 s)
origin/main portable RED: identical_exit == 26 (:5088), foreign_exit == 29 (:2400)
probe change only portable RED, now self-describing: identical_exit == 30, foreign_exit == 31
this PR portable 55/55 pass (43 s)
this PR hardware 55/55 pass (28 s)
this PR portable, full cli suite 410 passed, 1 skipped (Windows-only case), 282 s

In the passing portable run, the daemon's fingerprint verdicts match the
hardware run exactly. In particular, the changed copy receives an answered
rejection rather than a timeout.

Final tree (no proof patch), natural backend:

  • Linux arm64, GCC 13.3, test image:
    • daemon_bootstrap 27/27, daemon_runtime 55/55, daemon_frontend 18/18;
    • cli 410 passed, 1 skipped (the Windows-only case);
    • cli_sha256_hardware_matches_portable_issue2441 passes.
  • macOS arm64, Apple clang 21, sanitized runner:
    • daemon_bootstrap 28/28, daemon_runtime 56/56, daemon_frontend 18/18;
    • cli_sha256_hardware_matches_portable_issue2441 passes (it also
      asserts that the backend is arm-sha2);
    • cli: 408 passed, 1 skipped (Windows-only). The one failure,
      cli_vscode_only_installs_copilot_durable_context, is specific to the
      machine it ran on: that machine has the Copilot CLI installed under
      /opt/homebrew/bin, which the CLI's agent detection always probes, so
      a test that assumes no Copilot CLI fails there. It fails the same way
      when run alone, does not involve any code this PR touches, and passes
      on the Linux leg.
  • sha256.c compiles warning-free with -Wall -Wextra -Werror as a
    release, TSan and ASan+UBSan build (GCC 13.3 and Apple clang 21).
  • clang-format (Homebrew LLVM 23) is clean on sha256.c and on the changed
    test lines; cppcheck 2.20.0 (the CI lint image) is clean on sha256.c;
    lint-memory-core passes (no file grew).

Notes

  • No change to production behaviour: the daemon's admission, rejection and
    fingerprint logic are untouched. Only the sanitizer instrumentation of one
    pure arithmetic function changes, and only in sanitized builds.
  • Equality of the portable transform with the hardware backends stays
    covered by cli_sha256_hardware_matches_portable_issue2441 (every length
    0-299 plus 4 KiB to 1 MiB+13, eight chunkings each) and the NIST vectors.

Hosted x86 runners without SHA-NI hash with sha256.c's portable transform.
The daemon fingerprints each copied-image peer (the ~785 MB sanitized test
runner) before answering, and with ASan + UBSan instrumentation that took
~72-92 s per fingerprint on such runners, past the probes' 60 s exchange
budget: daemon_runtime's identical-copy HELLO and foreign activation tests
went red (exit 26 / 29) and the cli suite hit its 3600 s ceiling, depending
only on which CPU the job drew.

- sha256_transform (the portable block transform only) is compiled with
  no_sanitize("address", "undefined"). Measured per full hash, same input:
  GCC 13 x86-64 15.6 s -> 3.0 s, GCC 13 arm64 5.33 s -> 2.18 s, Apple clang
  arm64 2.2x; release builds and the hardware backends are unchanged, and
  cbm_sha256_update/final plus the block loop stay instrumented.

- The copied-image probes now exit 30 (HELLO) / 31 (activation) when the
  daemon never answered inside the budget, so 26 / 29 mean only an answered
  refusal. A slow fingerprint now fails as itself and can no longer satisfy
  the changed-copy rejection assertion by timing out.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData DeusData mentioned this pull request Nov 14, 2026
4 tasks done
…macro

CI's clang-format 20 lays out an attribute on its own line between
preprocessor guards differently from Homebrew's newer release, so the
lint job failed on sha256.c:85 while the local format check passed.
SHA256_NO_SANITIZE keeps the attribute on the signature line, as
NO_BUILTIN_MEM does in mem_override_libc.c. No behaviour change: the
macro expands to the same attribute (and to nothing on other compilers).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant