XTS Streaming .S and _c.c implementations, streaming callers & SVR brackets for aarch64

[kaleb-himes]

Description

https://github.com/wolfSSL/scripts/pull/670
taught the aarch64 crypto-extension generator to emit the AES-XTS streaming routines, but that PR regenerated only the MASM (armv8-aes-asm.asm). This PR regenerates the .S and _c.c twins from that same, already-merged generator so all three outputs agree. armv8-aes-asm.asm is untouched here and there is no partner scripts/ PR because 670 already did all the work.

In addition to the regenerated files:

aes.c / aes.h: The streaming callers dispatch into the crypto-extension asm (47 + 8 lines)
SAVE/RESTORE_VECTOR_REGISTERS claims around the aarch64 NEON and crypto lanes: aes.c, sha256.c, sha512.c, wc_mlkem.c, wc_mlkem_poly.c
SHA-3 aarch64 crypto lane claims the vector registers (sha3.c)
linuxkm arm64 vector-register glue on kernel_neon_begin, enabling ARM SIMD in the kernel module (linuxkm/arm64_vector_register_glue.c, plus Kbuild/PIE-table/configure plumbing)

NOTE: There is a known NEON gap for any aarch64 device that does NOT have the crypto extensions and I am happy to add that as a followup PR but this PR was already just over 5k lines of additions and adding that would have swelled it to 16-17k lines + a scripts/ generator update to go with it and that felt like too much to take on in a single PR.

Fixes zd#

Testing

• x86_64 host: fips-ready, fips-v7, C89, linuxkm build
• aarch64 cross-build, run under qemu-system on a 6.6.99 kernel
• aarch64 kernel module serving real dm-crypt xts(aes): ext4 + fio + stress, e2fsck clean
• cross-implementation readback: wolfSSL encrypts, the kernel's own xts decrypts, bad=0
• both lanes: crypto-extension and NEON
• both builds: FIPS v7 and non-FIPS (non-FIPS covers XTS-384 / AES-192)
• both forms: assembled .S and the inline _c.c twin
• streaming-off build (the Windows ARM64 config): asserts no branch to an uncompiled streaming symbol
• no-asm build: a software regression can't hide behind the asm
• new multi-block streaming tests in testwolfcrypt: chunked Update() vs one-shot across three chunking patterns, both rejection paths, and a proof the 32-bit per-stream counter can't corrupt data or run backwards
• generated asm reproduces byte-for-byte from merged scripts master, with a negative control
• benchmark builds cleanly both ways: default 64 KiB chunk and BENCH_EMBEDDED 256-byte chunk, with the chunk clamped at runtime down to a single AES block
• 15/15 configurations PASS
• throughput measured on Graviton3 spot hardware, us-east-2c (benchmark only; correctness is the qemu and kernel cells above)
• KATs and throughput on Graviton3 spot hardware, us-east-2c: testwolfcrypt passes on real silicon for both the crypto-extension and NEON lanes

Graviton3, c7g.xlarge spot, us-east-2c, median of 3 runs, MiB/s
testwolfcrypt KATs pass on both lanes on this hardware (rc=0)

Crypto-extension lane
                             master    PR 1KiB   PR 64KiB     64K vs
                                                              master
--------------------------------------------------------------------
AES-XTS-stream-enc           640.00    5228.64    6138.71       9.6x
AES-XTS-stream-dec           641.16    5319.33    6282.51       9.8x
AES-XTS-enc (one-shot)      6197.28    6202.64    6223.24      +0.4%
AES-XTS-dec (one-shot)      6257.80    6235.37    6256.67      -0.0%

NEON lane  (control: no streaming asm in this PR, expected flat)
                             master    PR 1KiB   PR 64KiB     64K vs
                                                              master
--------------------------------------------------------------------
AES-XTS-stream-enc           177.88     179.25     178.65      +0.4%
AES-XTS-stream-dec           186.87     186.60     186.23      -0.3%
AES-XTS-enc                  277.40     277.39     277.37      -0.0%
AES-XTS-dec                  227.93     228.23     228.23      +0.1%

PR 1KiB / PR 64KiB = benchmark chunk size, not a code difference;
both are AESINSNS 6838, identical binaries apart from one #define.
master and PR 1KiB were measured together; PR 64KiB comes from a later
instance whose 1KiB re-measure landed within 0.9%, which is what makes
the columns comparable.

Linux 6.16.12 under KVM, AES-NI
                   | 1 KiB |      | 64 KiB       | 
-------------------|-------|------|-------|------|
                   | MiB/s | cpb  | MiB/s | cpb  |
AES-XTS-enc        | 7614  | 0.46 | 11546 | 0.30 |
AES-XTS-stream-enc | 4478  | 0.79 | 11759 | 0.30 |
AES-XTS-dec        | 7660  | 0.46 | 12063 | 0.29 |
AES-XTS-stream-dec | 4411  | 0.80 | 11765 | 0.30 |
-------------------|-------|------|-------|------|

Linux 6.6.99, qemu-system-arm -cpu max, C-only build:

AES-XTS-enc           27.8 MiB/s
AES-XTS-stream-enc    27.4 MiB/s
AES-XTS-dec           26.0 MiB/s
AES-XTS-stream-dec    21.8 MiB/s

Functional verdict: PASS
The numbers are emulation speed (~28 MiB/s vs 11,500 native x86_64) and mean nothing as far as performance goes.

Checklist

• added tests
• updated/added doxygen
• updated appropriate READMEs
• Updated manual and documentation

[github-actions]

MemBrowse Memory Report

gcc-arm-cortex-m4-dtls13

• FLASH: .text +64 B (+0.0%, 187,260 B / 1,048,576 B, total: 18% used)
  No memory changes detected for:
• gcc-arm-cortex-m0plus
• gcc-arm-cortex-m3
• gcc-arm-cortex-m4
• gcc-arm-cortex-m4-baremetal
• gcc-arm-cortex-m4-crypto-only
• gcc-arm-cortex-m4-min-ecc
• gcc-arm-cortex-m4-openssl-compat
• gcc-arm-cortex-m4-pkcs7
• gcc-arm-cortex-m4-pq
• gcc-arm-cortex-m4-rsa-only
• gcc-arm-cortex-m4-sp-math
• gcc-arm-cortex-m4-tls12
• gcc-arm-cortex-m4-tls13
• gcc-arm-cortex-m7
• gcc-arm-cortex-m7-pq
• gcc-arm-cortex-m7-tls13
• linuxkm-pie
• linuxkm-standard
• stm32-sim-stm32h753

[kaleb-himes]

retest this please.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11423

Scan targets checked: wolfcrypt-src, wolfcrypt-bugs, wolfssl-src, wolfssl-bugs
Unchanged since last review (not re-run): linuxkm-src, linuxkm-bugs

Findings: 3
3 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11423

Scan targets checked: wolfcrypt-src, wolfcrypt-bugs, wolfssl-src, wolfssl-bugs
Unchanged since last review (not re-run): linuxkm-src, linuxkm-bugs

Findings: 2
2 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11423

Scan targets checked: wolfcrypt-src, wolfcrypt-bugs, wolfssl-src, wolfssl-bugs
Unchanged since last review (not re-run): linuxkm-src, linuxkm-bugs

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11423

Scan targets checked: wolfcrypt-src, wolfcrypt-bugs, wolfssl-src, wolfssl-bugs
Unchanged since last review (not re-run): linuxkm-src, linuxkm-bugs

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11423

Scan targets checked: linuxkm-src, linuxkm-bugs, wolfcrypt-src, wolfcrypt-bugs, wolfssl-src, wolfssl-bugs

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11423

Scan targets checked: linuxkm-src, linuxkm-bugs
Unchanged since last review (not re-run): wolfcrypt-src, wolfcrypt-bugs, wolfssl-src, wolfssl-bugs

Findings: 2
2 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

[kaleb-himes]

@wolfSSL-Fenrir-bot review force

Dismiss to re-request

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11423

Scan targets checked: none
Failed targets: linuxkm-src, linuxkm-bugs, wolfcrypt-src, wolfcrypt-bugs, wolfssl-src, wolfssl-bugs

⚠️ Review incomplete — one or more scan targets failed before findings could be produced. See the Fenrir PR review detail page for logs.

[kaleb-himes]

Fenrir just keeps err'ing out so I checked the Fenrir PR review page, it says 0 new findings and the other 14 have either been fixed or determined to be pre-existing items that are out of scope of what this PR is doing.

https://fenrir.wolfssl.com/pr-reviews/2534

[kaleb-himes]

NOTE: This is being addressed/fixed in branch N (#11413) but branch N is on top of this PR (branch L) so L has to merge before N can be considered for merge to plug the 32-bit gaps.