TLS Extensions: add more extensions and improve current

[SparkiDev]

Description

Added support for record_size_limit in TLS 1.3 and TLS 1.2.
Added compress_certificate support for TLS 1.3.
Added signed_certificate_timestamp TLS 1.2 and 1.3.
Added handshake-message fragmentation to EncryptedExtension, CertificateRequest
and NewSessionTicket.
Accepts server_name in CertificateRequest.
Added API for setting signature algorithms for signature_algorithms_cert.

SHA-1 certificate chain fix: a peer advertising ecdsa_sha1 against a chain
with an RSA-SHA1 intermediates will see a failed handshake.

Tests added and interop performed where possible.

Testing

Regression tested TLS.

[github-actions]

MemBrowse Memory Report

gcc-arm-cortex-m4-dtls13

• FLASH: .text +256 B (+0.1%, 186,876 B / 1,048,576 B, total: 18% used)

gcc-arm-cortex-m4-openssl-compat

• FLASH: .text +576 B (+0.1%, 783,452 B / 1,048,576 B, total: 75% used)

gcc-arm-cortex-m4-pq

• FLASH: .text +512 B (+0.2%, 302,152 B / 1,048,576 B, total: 29% used)

gcc-arm-cortex-m4-rsa-only

• FLASH: .text +448 B (+0.1%, 332,896 B / 1,048,576 B, total: 32% used)

gcc-arm-cortex-m4-tls13

• FLASH: .text +512 B (+0.2%, 242,639 B / 262,144 B, total: 93% used)

gcc-arm-cortex-m7-pq

• FLASH: .text +512 B (+0.2%, 303,112 B / 1,048,576 B, total: 29% used)

gcc-arm-cortex-m7-tls13

• FLASH: .text +512 B (+0.2%, 242,703 B / 262,144 B, total: 93% used)

linuxkm-standard

• Data: __patchable_function_entries +104 B (+0.2%, 49,888 B)
  No memory changes detected for:
• gcc-arm-cortex-m0plus
• gcc-arm-cortex-m3
• gcc-arm-cortex-m4
• gcc-arm-cortex-m4-baremetal
• gcc-arm-cortex-m4-crypto-only
• gcc-arm-cortex-m4-min-ecc
• gcc-arm-cortex-m4-pkcs7
• gcc-arm-cortex-m4-sp-math
• gcc-arm-cortex-m4-tls12
• gcc-arm-cortex-m7
• linuxkm-pie
• stm32-sim-stm32h753

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11362

Scan targets checked: wolfcrypt-rs-bugs, wolfssl-bugs, wolfssl-src

Findings: 3
3 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11362

Scan targets checked: wolfcrypt-rs-bugs, wolfssl-bugs, wolfssl-src

Findings: 3
3 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11362

Scan targets checked: wolfcrypt-rs-bugs, wolfssl-bugs, wolfssl-src

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11362

Scan targets checked: wolfcrypt-rs-bugs, wolfssl-bugs, wolfssl-src

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11362

Scan targets checked: wolfcrypt-rs-bugs, wolfssl-bugs, wolfssl-src

Findings: 2
2 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11362

Scan targets checked: wolfcrypt-rs-bugs, wolfssl-bugs, wolfssl-src
Findings: 2
1 finding(s) posted as inline comments (see file-level comments below)

Required changes (1)

Certificate-entry extension index not advanced in SCT-only builds

File: src/tls13.c:11240
Function: SendTls13Certificate
Category: Logic errors

WriteCertEntryExts() is now reached under HAVE_CERTIFICATE_STATUS_REQUEST || HAVE_SIGNED_CERT_TIMESTAMP and inflates extSz[0] with the SCT list, but the chain loop's extIdx++ is still guarded on HAVE_CERTIFICATE_STATUS_REQUEST alone. With --enable-sct and no OCSP stapling, each chain entry reuses extSz[0] and certExts[0], already freed at line 11223, so AddCertExt() dereferences NULL at line 10096.

Recommendation: Widen the extIdx++ and FreeDer() guards at lines 11104, 11119 and 11240 to also include HAVE_SIGNED_CERT_TIMESTAMP, matching the call site at line 11017.

Referenced code: src/tls13.c:11240-11244 (5 lines)

---

This review was generated automatically by Fenrir. Reported findings require changes before merge.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11362

Scan targets checked: wolfcrypt-rs-bugs, wolfssl-bugs, wolfssl-src

Fenrir result: Approved ✅

No new issues found in the changed files.

Advisory only — this automated result does not count as a GitHub approval.

Fenrir's latest completed scan found no issues; clearing the prior automated change request.