Skip to content

perf: fold contiguous reductions directly; NumPy's pairwise sum for float32/complex - #51

Merged
tannevaled merged 1 commit into
mainfrom
pairwise-sum
Oct 18, 2026
Merged

tannevaled merged 1 commit into
mainfrom
pairwise-sum

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

The problem. A float32 Sum of 2²⁰ elements took 6.85 ms on POWER9 against 0.23 ms for float64: 6.8 ns per element. Yet Add on float32 was only 1.5× slower than on float64, and a missing SIMD kernel explains a factor of 4–8, not 30.

The cause. Every non-float64 reduction (reduceNum) walked the [outer][axisLen][inner] layout with the reduced axis outside and the kept elements inside. A whole-array sum (inner == 1) therefore took a slice and ran a one-iteration loop for every element.

The fix. A contiguous run is now folded directly, as NumPy's add and multiply reductions do: the result starts as the first element and the rest is added to it.

  • float32 and complex: the rest is summed pairwise, transcribed from NumPy's FLOAT/CFLOAT_pairwise_sum (numpy/_core/src/umath/loops_utils.h.src, v2.5.3): eight accumulators, blocks of 128, halved above that. The complex variant counts floats as NumPy does.
  • Integer sums and all products: the rest is folded in order. Integer sums are exact in any order, and NumPy multiplies in order.
  • The inner > 1 path is unchanged.

Measured. On POWER9 (cfarm29), one thread pinned to the idlest CPU, the host 3% busy before and after. Six runs each with old and new binaries alternating, compared with benchstat, p = 0.002:

n = 2²⁰ before after
float32 Sum 6.850 ms 1.637 ms −76%
int64 Sum 5.195 ms 1.870 ms −64%
int32 Sum 6.170 ms 2.868 ms −54%
float64 Sum (own kernels, control) 225.7 µs 225.7 µs ~

More accurate too. The expectations come from exact arithmetic, not from the code:

  • 2²⁴ followed by 1000 ones gives 2²⁴ + 1000. An in-order float32 sum saturates at 2²⁴. Checked whole-array, row by row, and for complex64.
  • On 2¹⁶ values of mixed sign and magnitude, the error against the exact math/big sum falls from 75 to 0.87.
  • A sum of only −0 stays −0, as NumPy guarantees.
  • Every length class is covered (1, 7, 8, 9, 127, 128, 129, 4099).

The new tests fail on the old code (saturation and error). Coverage stays at 100%, with a new test for the product along a non-contiguous axis, which the fast path no longer exercises incidentally. The tests compile for 386 and arm.

Also in this PR:

  • BenchmarkDTypeSum and BenchmarkDTypeAdd.
  • A docs/perf.md section.
  • The roadmap is brought up to date: fft, linalg and the Ruby binding have shipped. What is open is SIMD for float32 (still 7× float64 for Sum) and the integer sums' extra conversion pass.

Not claimed: bit-identity with NumPy for large float32 sums. The algorithm is NumPy's, but whether NumPy's iterator feeds the loop in one call for every size was not verified by running NumPy.

🤖 Generated with Claude Code

…loat32/complex

A float32 Sum of 2^20 elements took 6.85 ms on POWER9 against 0.23 ms
for float64: 6.8 ns an element, while Add was only 1.5x slower. Every
non-float64 reduction walked [outer][axisLen][inner] with the reduced
axis outside, so a whole-array sum (inner == 1) took a slice and ran a
one-iteration loop per element.

A contiguous run is now folded directly, as NumPy's add and multiply
reductions do: the first element, then the rest added to it, pairwise
for float32 and complex (NumPy's FLOAT/CFLOAT_pairwise_sum,
loops_utils.h.src v2.5.3), in order for integers and products.

POWER9, 1 thread, n=2^20, benchstat over 6 interleaved runs (p=0.002):
float32 Sum -76%, int64 -64%, int32 -54%, float64 (control) ~.
More accurate too: error vs the exact sum 75 -> 0.87 on 2^16 mixed
values, and 2^24 + 1000 ones no longer saturates at 2^24; the new tests
fail on the old code. Coverage 100%; compiles for 386 and arm.

Also: BenchmarkDTypeSum/Add, a perf.md section, and the roadmap brought
up to date (fft, linalg and the Ruby binding shipped; float32 SIMD is
what is open).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit 0659ebc into main Oct 18, 2026
12 checks passed
@tannevaled
tannevaled deleted the pairwise-sum branch October 18, 2026 07:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant