diff --git a/README.md b/README.md index 8a821ef..4438caf 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ describes**: | Source | Published under | | --- | --- | -| branch `main`, with `params.ndarray.version: v0.41.3` in `hugo.yaml` | `https://go-ndarray.github.io/docs//`, here `/docs/0.41/` | +| branch `main`, with `params.ndarray.version: v0.42.0` in `hugo.yaml` | `https://go-ndarray.github.io/docs//`, here `/docs/0.42/` | | the newest version | also `https://go-ndarray.github.io/docs/latest/` | `https://go-ndarray.github.io/docs/` redirects to `latest/`. Each push to diff --git a/content/en/_index.md b/content/en/_index.md index 6b17f20..3f8d3c3 100644 --- a/content/en/_index.md +++ b/content/en/_index.md @@ -97,10 +97,13 @@ Large elementwise ops, reductions and products run **multicore** (across loops are go-asmgen SIMD kernels for `float64`: sum, add/sub/mul/div, sqrt and the dot product (SSE2 or AVX2/FMA on amd64, NEON on arm64, VSX on ppc64le, LASX on loong64, RVV on riscv64 and the vector facility on s390x, the last three -when the CPU has them), max/min too on amd64, arm64 and s390x (z14 and later), -and a **panel-packed, cache-blocked GEMM** with an SIMD-FMA micro-kernel (NEON +when the CPU has them), max/min too on amd64, arm64 and s390x (z14 and later). +The **panel-packed, cache-blocked GEMM** with its SIMD-FMA micro-kernels (NEON 4×8; AVX2/FMA 6×8 chosen at run time by a CPUID probe, SSE2 fallback; VSX 8×8; -LASX 8×8; RVV 6×8; s390x 4×8). The 32-bit targets run the same pure-Go code +LASX 8×8; RVV 6×8; s390x 4×8) no longer lives here: `MatMul`, `Dot` and the +linalg products call [go-blas/blas](https://github.com/go-blas/blas) v0.2.1 +(`DgemmStrided`/`SgemmStrided`), the one implementation of that kernel; the GEMM code moved +there byte for byte. The 32-bit targets run the same pure-Go code those kernels are tested against. `Exp` and `Log` are ports of Arm's optimized-routines (worst 0.504 and 0.508 ULP measured, against up to 0.88 and 0.72 for Go's `math.Exp`/`math.Log` on arm64), and they are correct where amd64's `math.Exp` returns +Inf @@ -112,7 +115,7 @@ is wrong on subnormals ([Performance](performance.md)): one row per public function (or a named exemption), against the pinned NumPy 2.5.4, `GOMAXPROCS=1` and one BLAS thread, interleaved rounds, refused at a 1-minute load of 8 or more, with the -allocation count of every row checked in CI. At v0.41.3 the measured table has +allocation count of every row checked in CI. The table measured at v0.41.0 has 188 of 256 rows slower than NumPy and 54 faster ([ndarray#118](https://github.com/go-ndarray/ndarray/issues/118) tracks the slower rows by cause). The earlier multicore comparison with single-threaded NumPy is kept as [Performance history](performance-history.md) diff --git a/content/en/accuracy.md b/content/en/accuracy.md index 689e8d7..4bf2387 100644 --- a/content/en/accuracy.md +++ b/content/en/accuracy.md @@ -8,7 +8,7 @@ description: "What go-ndarray promises about its results: a stated error bound, tags: [accuracy, numpy, math/big] --- -> Mirrored from [`README.md`](https://github.com/go-ndarray/ndarray/blob/v0.41.3/README.md#accuracy-contract) at v0.41.3. +> Mirrored from [`README.md`](https://github.com/go-ndarray/ndarray/blob/v0.42.0/README.md#accuracy-contract) at v0.42.0. The contract is **correct, and as fast as possible**. "Correct" means a stated error bound, checked against an exact oracle. Equality with NumPy's bits is not the contract: wherever it holds today, it is how a function was ported, and a faster default that stays within its bound may replace it. diff --git a/content/en/dtypes.md b/content/en/dtypes.md index 7e6871f..2538b90 100644 --- a/content/en/dtypes.md +++ b/content/en/dtypes.md @@ -8,7 +8,7 @@ description: "Why the dtype is a run-time property of one Array type, how arrays tags: [dtypes, numpy] --- -> Mirrored from [`docs/design-dtypes.md`](https://github.com/go-ndarray/ndarray/blob/v0.41.3/docs/design-dtypes.md) at v0.41.3. +> Mirrored from [`docs/design-dtypes.md`](https://github.com/go-ndarray/ndarray/blob/v0.42.0/docs/design-dtypes.md) at v0.42.0. > Status: implemented (Phase 2). Supersedes decision 4 of > [plan-ndarray.md](roadmap.md) ("`float64` first"). diff --git a/content/en/linalg.md b/content/en/linalg.md index ebb91ef..837fabf 100644 --- a/content/en/linalg.md +++ b/content/en/linalg.md @@ -8,9 +8,9 @@ description: "Solve, Inv, Det, Cholesky, QR, SVD, Eig and the rest of numpy.lina tags: [linalg, numpy] --- -> Mirrored from [`docs/linalg.md`](https://github.com/go-ndarray/ndarray/blob/v0.41.3/docs/linalg.md) at v0.41.3. +> Mirrored from [`docs/linalg.md`](https://github.com/go-ndarray/ndarray/blob/v0.42.0/docs/linalg.md) at v0.42.0. -Package [`linalg`](https://pkg.go.dev/github.com/go-ndarray/ndarray@v0.41.3/linalg) gives ndarray arrays the functions of +Package [`linalg`](https://pkg.go.dev/github.com/go-ndarray/ndarray@v0.42.0/linalg) gives ndarray arrays the functions of `numpy.linalg`, written in Go without cgo or assembly of its own: the matrix products it relies on are the ndarray package's packed SIMD GEMM. diff --git a/content/en/performance.md b/content/en/performance.md index 8326176..5857d65 100644 --- a/content/en/performance.md +++ b/content/en/performance.md @@ -8,11 +8,11 @@ description: "go-ndarray against the pinned NumPy on one core, both sides: the m tags: [performance, numpy, refbench] --- -> Mirrored from [`docs/perf.md`](https://github.com/go-ndarray/ndarray/blob/v0.41.3/docs/perf.md) at v0.41.3. +> Mirrored from [`docs/perf.md`](https://github.com/go-ndarray/ndarray/blob/v0.42.0/docs/perf.md) at v0.42.0. -The table is [perf-table.md](https://github.com/go-ndarray/ndarray/blob/v0.41.3/docs/perf-table.md), generated by +The table is [perf-table.md](https://github.com/go-ndarray/ndarray/blob/v0.42.0/docs/perf-table.md), generated by [refbench](https://github.com/go-scikit/refbench) (go-scikit/refbench v0.3.0) -from [perf.json](https://github.com/go-ndarray/ndarray/blob/v0.41.3/docs/perf.json); nothing in it is edited by hand. The rules are +from [perf.json](https://github.com/go-ndarray/ndarray/blob/v0.42.0/docs/perf.json); nothing in it is edited by hand. The rules are those of the go-scikit performance campaign ([go-scikit/roadmap#27](https://github.com/go-scikit/roadmap/issues/27)): @@ -36,7 +36,7 @@ those of the go-scikit performance campaign `TestRefSmoke` runs every NumPy call once. - **One case per public function** (a function with several shapes of use has several rows, `Array.SumAxis/axis0` and `/axis1`), or a named exemption with - its reason in [perf-exempt.md](https://github.com/go-ndarray/ndarray/blob/v0.41.3/docs/perf-exempt.md). CI fails on a public function + its reason in [perf-exempt.md](https://github.com/go-ndarray/ndarray/blob/v0.42.0/docs/perf-exempt.md). CI fails on a public function with neither. ## What a row times @@ -97,10 +97,10 @@ rounds: - The scalar forms (`AddScalar` … `DivScalar`, `AddInto` … `DivInto`, and any Float64 operation with a one-element operand against a contiguous array of the result's shape) no longer copy the scalar to the result's - size. A strided operand, such as a transpose, still takes the general path. - They stream it through the same SIMD kernel, so the result is bit for bit - the same. `AddScalar` went from 183–203 µs to 95–110 µs, and from 1.6 MB - to 0.8 MB per call. + size. A strided operand, such as a transpose, still takes the general path. They + stream it through the same SIMD kernel, so the result is bit for bit the + same. `AddScalar` went from 183–203 µs to 95–110 µs, and from 1.6 MB to + 0.8 MB per call. - `Full`, `Ones` and `FullOf` fill by doubling `copy` instead of storing one element at a time. `Full` went from 131–135 µs to 84–104 µs. `Ones` calls `Full`. `FullOf`, which fills the other dtypes, was not timed. diff --git a/content/en/random.md b/content/en/random.md index 0e2b5c9..6502842 100644 --- a/content/en/random.md +++ b/content/en/random.md @@ -8,9 +8,9 @@ description: "numpy.random with NumPy's numbers for the same seed: the bit gener tags: [random, numpy] --- -> Mirrored from [`docs/random.md`](https://github.com/go-ndarray/ndarray/blob/v0.41.3/docs/random.md) at v0.41.3. +> Mirrored from [`docs/random.md`](https://github.com/go-ndarray/ndarray/blob/v0.42.0/docs/random.md) at v0.42.0. -Package [`random`](https://pkg.go.dev/github.com/go-ndarray/ndarray@v0.41.3/random) is `numpy.random` over go-ndarray arrays. Its goal +Package [`random`](https://pkg.go.dev/github.com/go-ndarray/ndarray@v0.42.0/random) is `numpy.random` over go-ndarray arrays. Its goal is migration: a notebook moved from NumPy to Go keeps its numbers for the same seed. @@ -24,7 +24,7 @@ u, _ := random.Rand(3) // np.random.rand(3) ``` `Example`, `ExampleSeed` and `ExampleGenerator_ChoiceN` in -[`example_test.go`](https://github.com/go-ndarray/ndarray/blob/v0.41.3/random/example_test.go) print NumPy's own numbers. +[`example_test.go`](https://github.com/go-ndarray/ndarray/blob/v0.42.0/random/example_test.go) print NumPy's own numbers. ## What is implemented @@ -142,7 +142,7 @@ random arguments. Used as-is, that made 3% of `np.random.randn` values and 10% of `lognormal` values one ULP off. So `random` rounds `log`, `log1p`, `exp`, `expm1` and `pow` **correctly** -([`libm.go`](https://github.com/go-ndarray/ndarray/blob/v0.41.3/random/libm.go)): a fast evaluation to about 2⁻⁶⁸ with Ziv's +([`libm.go`](https://github.com/go-ndarray/ndarray/blob/v0.42.0/random/libm.go)): a fast evaluation to about 2⁻⁶⁸ with Ziv's rounding test, and a double-double fallback (about 2⁻¹⁰⁰) for the one argument in ~2¹⁵ that needs it. Checked against mpmath at 300 bits on 63,000 arguments (0 misrounded; the committed `libm.json` keeps 8,400), and the fast paths diff --git a/content/en/roadmap.md b/content/en/roadmap.md index eea7867..7521937 100644 --- a/content/en/roadmap.md +++ b/content/en/roadmap.md @@ -8,7 +8,7 @@ description: "The plan for go-ndarray and what ships today." tags: [roadmap] --- -> Mirrored from [`docs/plan-ndarray.md`](https://github.com/go-ndarray/ndarray/blob/v0.41.3/docs/plan-ndarray.md) at v0.41.3. +> Mirrored from [`docs/plan-ndarray.md`](https://github.com/go-ndarray/ndarray/blob/v0.42.0/docs/plan-ndarray.md) at v0.42.0. > Goal: a **pure-Go (CGO=0)** NumPy-style N-dimensional array library, as a > **standalone, reusable** module with correct scalar kernels now and a clean @@ -44,7 +44,8 @@ ndarray/ internal/kernels/ portable scalar inner loops over flat []float64 (Add/Sub/Mul/Div, Map, comparisons, Maximum/Minimum, Sum/Prod/Max/Min, Abs, {Sum,Prod,Max,Min}Axis over - [outer][axisLen][inner], MatMul GEMM, Dot1D) + [outer][axisLen][inner], mat-vec, Dot1D; the GEMM is + github.com/go-blas/blas, through internal/blasmm) docs/plan-ndarray.md this roadmap ``` @@ -115,7 +116,7 @@ suite (plus the per-arch CI jobs already wired in `.github/workflows/ci.yml`). NumPy. What follows is the history of the levers behind the kernel seam, measured then against NumPy 2.2.4 (see [docs/perf.md](performance.md) and the repo's - [BENCHMARKS.md](https://github.com/go-ndarray/ndarray/blob/v0.41.3/BENCHMARKS.md)): + [BENCHMARKS.md](https://github.com/go-ndarray/ndarray/blob/v0.42.0/BENCHMARKS.md)): - **Multicore.** `Add/Sub/Mul/Div/Map`, `Sum/Max/Min`, the axis reductions, `Concatenate`/`Stack`, and the GEMM fan out across `GOMAXPROCS` above a size threshold (small arrays stay serial). Plus an allocation fix: the diff --git a/content/en/security.md b/content/en/security.md index 64f1fe3..ac6c0f0 100644 --- a/content/en/security.md +++ b/content/en/security.md @@ -8,7 +8,7 @@ description: "How to report a vulnerability, what the library guarantees, and th tags: [security] --- -> Mirrored from [`SECURITY.md`](https://github.com/go-ndarray/ndarray/blob/main/SECURITY.md) at v0.41.3. +> Mirrored from [`SECURITY.md`](https://github.com/go-ndarray/ndarray/blob/main/SECURITY.md) at v0.42.0. ## Reporting @@ -25,15 +25,19 @@ bounds checks do not cover, so they are held to the following: - **Every wrapper checks the reach of its kernel.** Before an assembly call, the Go wrapper indexes the last element of every operand the kernel reads or - writes (`internal/kernels/{reduce,gemm}_.go`). A caller's mistake is an - index-out-of-range panic in Go, never an access outside a slice. + writes (`internal/kernels/{reduce,dot}_.go`). A caller's mistake is an + index-out-of-range panic in Go, never an access outside a slice. The GEMM + micro-kernels, their wrappers and their fence tests live in + [go-blas/blas](https://github.com/go-blas/blas), which owns that audit; the + entries below about GEMM kernels (v0.26.0 to v0.38.0) are history of code now + there. `TestWrappersRefuseShortOperands` passes each wrapper an operand one element short and requires the panic. - **The kernels stay inside their slices.** `TestKernelsStayInsideTheirSlices` - and `TestGemmMicroStaysInsideItsOperands` ([`internal/kernels/fence_test.go`](https://github.com/go-ndarray/ndarray/blob/main/internal/kernels/fence_test.go), - Linux and macOS) map each operand flush against a `PROT_NONE` page, once - ending at it and once starting after one, for every length from 1 to 70 and - every GEMM depth from 1 to 9. A read or write one element past either end is a + ([`internal/kernels/fence_test.go`](https://github.com/go-ndarray/ndarray/blob/main/internal/kernels/fence_test.go), Linux and macOS) maps each operand flush + against a `PROT_NONE` page, once ending at it and once starting after one, for + every length from 1 to 70 (go-blas/blas fences its GEMM micro-kernels at every + depth from 1 to 9 the same way). A read or write one element past either end is a fault. The fence itself was checked by a kernel deliberately reading one element too far, which faults on amd64 (4 KiB pages), arm64 (16 KiB), loong64 (16 KiB) and ppc64le (64 KiB). Prefetch instructions may point past the end: @@ -76,7 +80,19 @@ standard library's known vulnerabilities; `govulncheck ./...` reports none. ## Audit log -**2026-11-12 (v0.40.2, #103).** The gather of `random`'s `Choice` +**2026-11-15 (unreleased).** The GEMM moved to +[go-blas/blas](https://github.com/go-blas/blas) v0.2.1 (the micro-kernels, their +wrappers, the packing and their fence tests, byte for byte; the audit of that +code is that repository's). What ndarray adds is `internal/blasmm`, which hands +an operand's row and column strides to `DgemmStrided`/`SgemmStrided`. go-blas +checks every stride and slice length (overflow-safe, `*ArgError` on refusal), so +a view ndarray builds wrongly is an error there, which `blasmm` turns into a +panic naming the package, not a read outside a slice. A zero or negative stride +on an axis longer than one (a broadcast or reversed view) is copied to a +contiguous matrix first. ndarray keeps the dot kernels (`dot_.go`), the +mat-vec and the thread pool, with their wrappers and fences. + +**2026-11-12 (unreleased, #103).** The gather of `random`'s `Choice` (`takeAxis`, random/arrays.go) now copies bytes through `bytesOf`, an `unsafe.Slice` view of a slice's backing array of exactly `Len()·size` bytes. Every dtype the package stores is a fixed-size value without pointers diff --git a/hugo.yaml b/hugo.yaml index 01835a2..c86e8b6 100644 --- a/hugo.yaml +++ b/hugo.yaml @@ -6,7 +6,7 @@ # have used since the site was built with MkDocs and mike. The CI sets it # through --baseURL, from params.ndarray.version below. This default is the # same build. -baseURL: "https://go-ndarray.github.io/docs/0.41/" +baseURL: "https://go-ndarray.github.io/docs/0.42/" title: "go-ndarray" # themes/hextra is the tannevaled/hextra fork, pinned at a release tag: # upstream Hextra plus opt-in features (partial menu items, combos, @@ -100,7 +100,7 @@ params: # The ndarray release these pages describe. Its minor version is the # documentation's version label (v0.9.1 -> 0.9): the CI reads it here. ndarray: - version: "v0.41.3" + version: "v0.42.0" navbar: displayTitle: true