Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,9 @@
64-bit target, and `Max`/`Min` on every 64-bit target, bit-identical to its
scalar loops. Its `MatMul` runs on a native float32 GEMM on arm64 (NEON),
amd64 with FMA (AVX2), ppc64le (VSX) and s390x (z14 vector), accumulating in float32 like NumPy's sgemm, and
elsewhere on the float64 GEMM (accumulated in float64, rounded once)
elsewhere on the float64 GEMM (accumulated in float64, rounded once).
Its `Exp` and `Log` are **correctly rounded**, proven by an exhaustive test
over all 2³² float32 inputs
([docs/design-dtypes.md](docs/design-dtypes.md)).
- **Creation** — `New`/`Zeros`/`Ones`/`Full`/`FromData`/`Arange`/`Linspace`/
`Eye`/`Identity`.
Expand Down
7 changes: 7 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,13 @@ standard library's known vulnerabilities; `govulncheck ./...` reports none.

## Audit log

**2026-11-01 (v0.32.0).** float32 `Log` checks each result for a float32
rounding midpoint with one integer comparison. On a hit it looks the input
up by binary search in a fixed 24-entry table, compiled in and read only.
No input can make that search read outside the table or allocate. The
exhaustive test that derives and checks the table is
`TestLog32CorrectlyRounded`.

**2026-11-01 (v0.31.0).** float32 `Exp` and `Log` widen their input 256
elements at a time into a fixed stack buffer and call the float64 kernels in
place. There is no allocation proportional to the input beyond the result,
Expand Down
40 changes: 40 additions & 0 deletions docs/perf.md
Original file line number Diff line number Diff line change
Expand Up @@ -1441,6 +1441,46 @@ equal on at least 99.9% of 100 000 inputs. They also check the edges
(±0, ±Inf, NaN, overflow, the subnormals, log of a negative), every length
across the chunk boundaries, in-place use and the parallel split.

#### Correctly rounded, proven over every float32 (2026-11-01)

A float32 function has 2³² inputs, so correct rounding can be checked
exhaustively instead of by sampling. `TestExp32CorrectlyRounded` and
`TestLog32CorrectlyRounded` (set off by `NDARRAY_EXHAUSTIVE=1`; CI samples
every 65 537th input) run every bit pattern. Each result is certified one of
three ways:

- the float64 intermediate, within 0.504 float64 ULP of the exact value, is
more than 2 float64 ULP from both float32 rounding midpoints;
- or it is far outside float32's range;
- or a 300-bit reference decides the rounding.

The certifier is itself tested. Planted values on a midpoint, one ULP from
one, and at the overflow boundary are not certified; that last one was a bug,
caught by its test. A kernel sabotaged to round one ULP up fails 76% of exp
and 50% of log inputs.

Results, identical on darwin/arm64, linux/amd64 (Zen 3), linux/ppc64le
(POWER9), linux/s390x (IBM Z) and linux/riscv64 (Spacemit X60), each run on
the real machine (loong64: CI's sampled run only, no hardware access):

| | inputs | decided by the reference | not correctly rounded |
|---|---:|---:|---:|
| `Exp` | 4 294 967 296 | 2 | **0** |
| `Log` before the table | 4 294 967 296 | 24 | 5 |
| `Log` with the table | 4 294 967 296 | 24 | **0** |

`Log`'s double rounding was wrong for 5 inputs, e.g. log(9.47263622):
`0x400fe5e8` instead of `0x400fe5e7`. The fix is the one correctly-rounded
libraries use. A result within 8 float64 ULP of a float32 midpoint (the 29
dropped bits are 0x10000000 ± 8) is looked up in a table of the 24 inputs
that come within 2 ULP, each with its correctly rounded result from the
300-bit reference. The check costs nothing measurable (−4%, p = 0.07, on a
host at load 15).

So float32 `Exp` and `Log` are **correctly rounded**: every result is the
float32 nearest the exact value. NumPy's float32 SIMD kernels are documented
at 2.52 and 3.83 ULP in their sources.

**Open defect: still 3× slower than NumPy.** NumPy computes float32 exp and
log in float32 SIMD (`simd_exp_f32`/`simd_log_f32` in
`loops_exponent_log.dispatch.c.src`). Porting those kernels per architecture
Expand Down
52 changes: 52 additions & 0 deletions internal/kernels/via64.go
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
package kernels

import "math"

// float32 transcendental functions on the float64 kernels.
//
// float32 Exp and Log went through math.Exp and math.Log per element, 6-14x
Expand Down Expand Up @@ -42,12 +44,62 @@ func via64(op via64Op, dst, src []float32) {
Log(b, b)
}
d := dst[i : i+len(b)]
if op == via64Log {
for j, v := range b {
f := float32(v)
// Within 8 float64 ULP of a float32 rounding midpoint (the
// 29 bits float32 drops are 0x10000000 +- 8): the double
// rounding may be wrong, so take the table's answer.
if math.Float64bits(v)&(1<<29-1)-(1<<28-8) < 16 {
f = log32Exception(s[j], f)
}
d[j] = f
}
continue
}
for j, v := range b {
d[j] = float32(v)
}
}
}

// log32Exceptions are the float32 inputs whose float64 log lies within 2
// float64 ULP of a float32 rounding midpoint, sorted, each with its correctly
// rounded log decided by a 300-bit reference. The exhaustive test lists them
// (NDARRAY_EXHAUSTIVE=1 NDARRAY_DUMP_HARD=1, TestLog32CorrectlyRounded): 24
// inputs, 5 of which rounded to the wrong float32 without this table. Log's
// float64 arithmetic is the same on every architecture (its multiply-add is
// math.FMA), so the list is too.
var log32Exceptions = [...][2]uint32{
{0x0dc8bba4, 0xc289bac4}, {0x111c87f8, 0xc28085df}, {0x1a8446cb, 0xc24d0a45},
{0x1f116ab8, 0xc233b53e}, {0x28e3fa26, 0xc1fa75e0}, {0x29e6126b, 0xc1ef4c02},
{0x29fd22f8, 0xc1ee8859}, {0x2c4c24b7, 0xc1d48710}, {0x2e492984, 0xc1be76ee},
{0x38dcbe38, 0xc1128ba4}, {0x3bf86ef0, 0xc09c399e}, {0x3c413d3a, 0xc08e158f},
{0x41178feb, 0x400fe5e7}, {0x464d5b2b, 0x4117bcf1}, {0x4665a9a6, 0x41198725},
{0x4c5d65a5, 0x418f034b}, {0x4d604ebe, 0x419a352c}, {0x4e85f412, 0x41a6b811},
{0x5ee8984e, 0x422e4a21}, {0x65d890d3, 0x4254d1f9}, {0x66a8c860, 0x42595e46},
{0x66abbd63, 0x4259700e}, {0x6f31a8ec, 0x42845a89}, {0x79e7ec37, 0x42a1ffb7},
}

// log32Exception returns the table's correctly rounded log of x if x is in
// log32Exceptions, and f otherwise.
func log32Exception(x, f float32) float32 {
bits := math.Float32bits(x)
lo, hi := 0, len(log32Exceptions)
for lo < hi {
m := (lo + hi) / 2
switch e := log32Exceptions[m]; {
case e[0] == bits:
return math.Float32frombits(e[1])
case e[0] < bits:
lo = m + 1
default:
hi = m
}
}
return f
}

// Exp32 writes exp(src[i]) into dst[i] for float32 elements.
func Exp32(dst, src []float32) { via64(via64Exp, dst, src) }

Expand Down
255 changes: 255 additions & 0 deletions internal/kernels/via64_exhaustive_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,255 @@
package kernels

import (
"math"
"math/big"
"os"
"runtime"
"sync"
"sync/atomic"
"testing"
)

// Exp32 and Log32 are correctly rounded: proven over every float32.
//
// A float32 function has 2^32 inputs, so correct rounding can be checked
// exhaustively instead of sampled. For each input the float64 intermediate y
// (Exp or Log of the widened input, the value Exp32 and Log32 round) has an
// error of at most 0.504 float64 ULP (TestExpULP, TestLogULP). If y is more
// than 2 float64 ULP away from both float32 rounding midpoints around
// float32(y), the exact value rounds to the same float32: certified. The few
// inputs closer than that are decided by the 300-bit references refExp and
// refLog, rounded by big.Float.Float32 (round to nearest, ties to even).
// Results far beyond float32's range are certified by bounds: |y| > 2^129
// rounds to ±Inf, |y| < 2^-152 to ±0, and so does the exact value.
//
// The full run (2^32 inputs per function) takes seconds on all cores and is
// set off by NDARRAY_EXHAUSTIVE=1; without it every 65537th bit pattern is
// checked, about 65 000 inputs, so CI stays fast.

// certified32 rounds the float64 intermediate y to float32 and reports
// whether that rounding is provably the correct rounding of the exact value.
func certified32(y float64) (float32, bool) {
f := float32(y)
ay := math.Abs(y)
if math.IsNaN(y) || math.IsInf(y, 0) || ay > 0x1p129 || ay < 0x1p-152 {
return f, true
}
eps := 2 * (math.Nextafter(ay, math.Inf(1)) - ay)
if math.IsInf(float64(f), 0) {
// y rounded to ±Inf although |y| <= 2^129: the only boundary is
// half an ULP past MaxFloat32, where ties-to-even goes to Inf.
// (TestCertified32 plants exactly this value.)
return f, ay-(math.MaxFloat32+0x1p103) > eps
}
ff := float64(f)
near := func(neighbour float32) bool {
var m float64
if math.IsInf(float64(neighbour), 0) { // f is ±MaxFloat32: the boundary is half an ULP past it
m = ff + math.Copysign(0x1p103, ff)
} else {
m = (ff + float64(neighbour)) / 2 // exact: adjacent float32s have 24-bit mantissas
}
return math.Abs(y-m) <= eps
}
if near(math.Nextafter32(f, float32(math.Inf(-1)))) || near(math.Nextafter32(f, float32(math.Inf(1)))) {
return f, false
}
return f, true
}

type exhaustiveCase struct {
name string
k32 func(dst, src []float32)
k64 func(dst, src []float64)
ref func(float64) *big.Float
// special returns the expected result for inputs the reference does not
// take (NaN, ±Inf, and for log 0 and negatives), and ok=false otherwise.
special func(x float32) (want float32, isNaN, ok bool)
}

func exp32Special(x float32) (float32, bool, bool) {
switch {
case x != x:
return 0, true, true
case math.IsInf(float64(x), 1):
return float32(math.Inf(1)), false, true
case math.IsInf(float64(x), -1):
return 0, false, true
}
return 0, false, false
}

func log32Special(x float32) (float32, bool, bool) {
switch {
case x != x || x < 0:
return 0, true, true
case x == 0:
return float32(math.Inf(-1)), false, true
case math.IsInf(float64(x), 1):
return float32(math.Inf(1)), false, true
}
return 0, false, false
}

func runExhaustive(t *testing.T, c exhaustiveCase, stride uint64) {
const batch = 4096
workers := runtime.GOMAXPROCS(0)
var next atomic.Uint64
var bad, hard, checked atomic.Int64
var mu sync.Mutex
var firstBad []string
var wg sync.WaitGroup
for range workers {
wg.Add(1)
go func() {
defer wg.Done()
xs := make([]float32, 0, batch)
got := make([]float32, batch)
w64 := make([]float64, batch)
y64 := make([]float64, batch)
for {
start := next.Add(batch*stride) - batch*stride
if start >= 1<<32 {
return
}
xs = xs[:0]
for b := start; b < start+batch*stride && b < 1<<32; b += stride {
xs = append(xs, math.Float32frombits(uint32(b)))
}
n := len(xs)
c.k32(got[:n], xs)
for i, x := range xs {
w64[i] = float64(x)
}
c.k64(y64[:n], w64[:n])
for i, x := range xs {
checked.Add(1)
var want float32
if w, isNaN, ok := c.special(x); ok {
if isNaN {
if got[i] == got[i] {
bad.Add(1)
mu.Lock()
if len(firstBad) < 8 {
firstBad = append(firstBad, c.name+": NaN wanted")
}
mu.Unlock()
}
continue
}
want = w
} else {
f, sure := certified32(y64[i])
if sure {
want = f
} else {
hard.Add(1)
want, _ = c.ref(float64(x)).Float32()
if os.Getenv("NDARRAY_DUMP_HARD") != "" {
mu.Lock()
t.Logf("hard %s 0x%s -> 0x%s", c.name, hex32(math.Float32bits(x)), hex32(math.Float32bits(want)))
mu.Unlock()
}
}
}
if math.Float32bits(got[i]) != math.Float32bits(want) {
bad.Add(1)
mu.Lock()
if len(firstBad) < 8 {
firstBad = append(firstBad, c.name+"("+formatF32(x)+") = "+formatF32(got[i])+", correctly rounded "+formatF32(want))
}
mu.Unlock()
}
}
}
}()
}
wg.Wait()
t.Logf("%s: %d inputs checked, %d decided by the 300-bit reference, %d not correctly rounded",
c.name, checked.Load(), hard.Load(), bad.Load())
for _, s := range firstBad {
t.Error(s)
}
}

func formatF32(x float32) string {
return big.NewFloat(float64(x)).Text('g', 9) + " (0x" + hex32(math.Float32bits(x)) + ")"
}

func hex32(u uint32) string {
const digits = "0123456789abcdef"
var b [8]byte
for i := 7; i >= 0; i-- {
b[i] = digits[u&15]
u >>= 4
}
return string(b[:])
}

func exhaustiveStride() uint64 {
if os.Getenv("NDARRAY_EXHAUSTIVE") != "" {
return 1
}
return 65537
}

func TestExp32CorrectlyRounded(t *testing.T) {
runExhaustive(t, exhaustiveCase{"exp", Exp32, Exp, refExp, exp32Special}, exhaustiveStride())
}

func TestLog32CorrectlyRounded(t *testing.T) {
runExhaustive(t, exhaustiveCase{"log", Log32, Log, refLog, log32Special}, exhaustiveStride())
}

// TestLog32Exceptions runs Log32 on every input of the exception table (the
// sampled run rarely meets one): each result must be the table's, and the
// table's must be the 300-bit reference rounded to float32. An input outside
// the table keeps the computed value.
func TestLog32Exceptions(t *testing.T) {
src := make([]float32, len(log32Exceptions))
for i, e := range log32Exceptions {
src[i] = math.Float32frombits(e[0])
}
got := make([]float32, len(src))
Log32(got, src)
for i, e := range log32Exceptions {
want, _ := refLog(float64(src[i])).Float32()
if math.Float32bits(want) != e[1] {
t.Errorf("table entry 0x%s: 0x%s, the reference rounds to 0x%s", hex32(e[0]), hex32(e[1]), hex32(math.Float32bits(want)))
}
if math.Float32bits(got[i]) != e[1] {
t.Errorf("Log32(0x%s) = 0x%s, want the table's 0x%s", hex32(e[0]), hex32(math.Float32bits(got[i])), hex32(e[1]))
}
}
if f := log32Exception(2, 0.5); f != 0.5 {
t.Errorf("an input outside the table changed: %v", f)
}
}

// TestCertified32 checks the certifier itself on planted values: a float64
// exactly on a float32 midpoint, and one ULP from it, must be sent to the
// reference; a value in the middle of a float32 interval is certified.
func TestCertified32(t *testing.T) {
one, next := float64(float32(1)), float64(math.Nextafter32(1, 2))
mid := (one + next) / 2
if _, sure := certified32(mid); sure {
t.Error("a float32 midpoint was certified")
}
if _, sure := certified32(math.Nextafter(mid, 2)); sure {
t.Error("one float64 ULP above a midpoint was certified")
}
if f, sure := certified32(one + (next-one)/8); !sure || f != 1 {
t.Errorf("1 + ulp/8: %v, %v", f, sure)
}
if f, sure := certified32(0x1p130); !sure || !math.IsInf(float64(f), 1) {
t.Errorf("2^130: %v, %v", f, sure)
}
if f, sure := certified32(0x1p-160); !sure || f != 0 {
t.Errorf("2^-160: %v, %v", f, sure)
}
maxMid := float64(math.MaxFloat32) + 0x1p103
if _, sure := certified32(maxMid); sure {
t.Error("the overflow boundary was certified")
}
}
Loading