Repository navigation
Description
Why
go-embedded-ruby, go-embedded-julia, go-embedded-r and go-embedded-c each need many of the same capabilities: regexes, Unicode normalisation, compression, YAML, time zones, digests, random bit generators, linear algebra, and access to files and the network with a sandbox mode. Today the most complete implementations sit in go-ruby-* orgs, and each one describes itself as "Pure-Go MRI-compatible Ruby X". The others would otherwise rewrite them.
The direction, agreed by the maintainer on 2026-11-10: each language should gain the capabilities that used to be intrinsic to one language.
Rule
- If an engine's reference behaviour is a C library or a standard that several languages share, the engine lives in a neutral org named for that reference.
go-ruby-Xkeeps only the layer faithful to MRI: API shape, messages, output formatting quirks. - What Ruby itself defines stays in
go-ruby-*. - Extraction happens on need, when a second language needs the engine, not as one sweep.
Moving a module costs:
- a new module path and a release;
- updating its importers, including rbgo;
- a
// Deprecated:notice on the old path; - carrying the CI it had: fuzz targets, the 6 architectures, the documentation's promises.
First inventory of the 178 go-ruby-* modules rbgo binds
| engine (today) | shared reference | other languages that need it |
|---|---|---|
go-ruby-regexp (Onigmo) |
Onigmo | any Go program (the fleet already recommends it for PCRE/Ruby/JS regexes); a natural neighbour of go-regexp/pcre2 |
go-ruby-unicode-normalize |
Unicode UAX #15 | julia now (Base's normalize calls utf8proc); go-embedded-julia also has its own tables (internal/unitab) |
go-ruby-digest, go-ruby-securerandom |
the standard algorithms; the OS entropy source | julia SHA and Random, R digest; entropy belongs to the shared host/sandbox layer |
go-ruby-zlib |
zlib (byte-identical streams) | R (memCompress, gzcon), julia |
go-ruby-yaml |
libyaml (Psych, and R's yaml package) |
R |
go-ruby-tzinfo |
the IANA tz database | julia, R |
go-ruby-csv, -json, -toml, -msgpack, -protobuf, -arrow, -parquet |
the formats | all of them, each with its own output layer; Arrow also gives notebook cells one in-memory table format across languages |
go-ruby-pg, -mysql, -sqlite3, -redis, -http, -net-*, -resolv, -ldap… |
the protocols | all of them, once each VM has a host mode |
go-ruby-format, go-ruby-scanf |
C's printf/scanf core |
C, R sprintf, julia Printf, each with its own variants on top |
Stays Ruby: erb, haml, slim, erubi, marshal, ostruct, zeitwerk, bundler, rubygems, irb, and the Rails, RSpec and Puppet stacks.
Also shared, outside go-ruby-*
- Linear algebra:
go-ndarray/ndarray/linalg(BLAS, LU, QR, Cholesky, eig, SVD, lstsq) becomes a standalone library that julia's LinearAlgebra and R'ssolve/qr/choluse. - Random: the bit generators in
go-ndarray/ndarray/random(MT19937, PCG64, Philox) become a shared library, with Xoshiro256++ and dSFMT added for julia. Distributions stay per language, for bit parity. - libm: a bit-exact libm (openlibm for julia, the system libm for R and C).
- Host/sandbox layer: files, sockets, environment, clock, entropy and processes, designed once on WASI's capability model. go-embedded-julia's first version is in progress (M3: julia's seven stdlibs in the image, and a host mode beside the sandbox go-embedded-julia/julia#32).
First extractions (needed by go-embedded-julia now)
unicode-normalize, reconciled with go-embedded-julia'sinternal/unitab.digest, for julia's SHA.securerandom, into the host/sandbox layer.- Onigmo, moved to
go-regexp.
Prior art: priorart "cross-language" at 2026-11-10 17:15 CET reported prior art; checked: priorart 'neutral' and 'cross-language' matched only unrelated VM fixes (#108-#121, #366-#367, #674, #870) and spec-shim issues on the words; no issue covers extracting engines or neutral naming
Activity
tannevaled commentedon Nov 10, 2026
go-ndarray's answer for
linalgandrandom(2026-11-10)State: neither is mid-change. The only open PRs are go-ndarray/ndarray#106 (Renovate) and #81 (loong64 float32 GEMM in
internal/kernels, blocked on hardware access). There is no reason against the extraction, under four constraints:linalgimportsndarray/internal/kernels(the GEMM micro-kernels). Either the neutral library carries the GEMM too, or the kernels move down first into go-simd/floats, where the K1 kernels already go (K1 remainder: float32 kernels, Dot1D and the dot inside GEMM, Maximum/Minimum move to go-simd/floats go-ndarray/ndarray#98). There is one implementation, in the lowest shared library.linalgworks on*ndarray.Array. The neutral library takes raw slices with strides and leading dimensions, LAPACK-style, and ndarray keeps a thin wrapper. Its oracle is NumPy 2.5.3 on 1,040 cases. That corpus moves with it, as do the CI jobs: fuzz, govulncheck, cross-arch.ndarray/linalgandndarray/randomstay as wrappers, so import paths do not change; below v1 this is a MINOR bump.The allocation fixes land once, in the new library:
Choiceallocates per draw.Also relevant to julia: go-ndarray/ndarray#76 (
Wrap, a zero-copy array with strides over a caller's buffer) was filed for julia's column-major arrays. The LAPACK-style API of point 2 serves julia'sMatrixdirectly, with no copy.docs(shared-engines): julia's three sandbox modes, and random's depen…
docs(shared-engines): retract an overreach, and measure what replaces it
docs: shared engines — the rule, the inventory, and what each languag…
m3-plan: shared engines (go-numerics, linear algebra, regex, zlib, ho…
tannevaled commentedon Nov 15, 2026
julia needs the linear-algebra extraction now, and it fixes the API's shape
go-embedded-julia v0.17.0 runs julia's real
LinearAlgebrastdlib, butA * xfails: julia sends every dense product, solve and factorisation through libblastrampoline (LBT). Our interpreter has LBT in Go (interp/lbt.go:lbt_forward,lbt_get_config), and it forwards to nothing. go-notebook ranks this first, because it is the first thing a scientific user hits.go-ndarray's answer above, and a reread of a v0.41.2 clone today, show that the fleet already has what is needed, but not in importable form:
ndarray/internal/kernels:MatMulStridedP,MatVecStridedP,Dot1DP, and float32). It is row-major and internal.ndarray/linalg). These are unexported, and checked against NumPy 2.5.3 on 1,040 cases.So binding julia means doing the extraction this issue describes. No second GEMM gets written, per go-ndarray's constraint 1.
Proposal: the neutral library's surface is Fortran BLAS and LAPACK
This is constraint 2 of go-ndarray's answer, made concrete, because julia pins the shape exactly:
Signatures follow the reference BLAS and LAPACK:
trans/uplo/diag/sideflags,alpha/beta, and leading dimensionslda/ldb/ldc;inforesult convention.julia calls exactly that through LBT, in its ILP64 variant (
dgemm_64_…), with 64-bit integers. R'ssolve,qrandcholcall the same names, and C programs can call them unchanged. ndarray's numpy-shaped API becomes a thin wrapper over it, as constraint 4 says.Column-major needs no new kernel. A column-major
C = op(A)·op(B)is the row-major productCᵀ = op(B)ᵀ·op(A)ᵀover the same buffers, with the operands swapped and the transpose flags flipped. The existing strided micro-kernel serves both layouts. Wrap: an array over a caller's buffer with given strides (zero-copy, for Julia column-major arrays) go-ndarray/ndarray#76 (Wrap) gives julia'sMatrixwith no copy.The oracle is the reference netlib BLAS and LAPACK, plus NumPy's 1,040 cases, with a stated error bound per routine (CLAUDE.md, the bit-parity decision of Accuracy contract: a stated error bound against an extended-precision oracle; bit identity becomes an option go-fft/fft#121). The kernel's FMA and blocking stay. Bit identity with OpenBLAS is not the contract.
What julia needs first:
\andlu), potrf/potrs (cholesky), geqrf/ormqr (qr), syevd/heevd (eigenof a symmetric matrix), gesdd (svd).That is close to what
ndarray/linalgalready holds.Placement
This issue's rule names the org for the shared reference, so something like
go-blas, orgo-lapackfor both. The GEMM micro-kernel goes where go-ndarray's constraint 1 puts it, either in go-simd/floats or in that library. go-ndarray holds the kernels and decides that part; I would rather not guess.Who does what
dgemm_64_and friends to the new library, plus julia's ownLinearAlgebratest files as a second oracle.The julia side waits for the library's first release.
The same goes for
unicode-normalize(first extraction 1 above): julia'suppercaseandnormalizecall utf8proc, and fail today (go-embedded-julia/julia#65).tannevaled commentedon Nov 15, 2026
go-ndarray agrees: the Fortran BLAS/LAPACK surface, and where GEMM goes (2026-11-15)
Prior art: this layer is already designed
priorartturned up go-ndarray/ndarray#92 (docs/design-linalg.md, merged). Its §9 describes exactly the layer proposed above, aslinalg/internal/lapack:refblas.go, level 1–3 in netlib loop order (dot, axpy, nrm2, gemv, ger, trmv, trsv, gemm, trmm, trsm, syrk);None of it has started: a v0.41.2 tree has no
linalg/internal/lapack. So the extraction does not move existing column-major code. It builds that design in the neutral module instead of underlinalg/internal, and ndarray consumes it from there.One amendment to that design
go-ndarray/ndarray#92 makes bit identity with netlib the contract: a
refgemmwithout FMA (§7.3), and fused or k-reordered GEMM only as an exception (§7.4). It was written before the 2026-11-11 decision on go-fft/fft#121. Under that decision, the netlib-exact port is a porting step and the oracle, not the default. The library ships:Placement (constraint 1)
The whole GEMM moves into the new library: packing, blocking and the go-asmgen micro-kernels of all 6 architectures. It does not go into go-simd/floats.
inc = 1instead of duplicating it. The "dot inside GEMM" item of K1 remainder: float32 kernels, Dot1D and the dot inside GEMM, Maximum/Minimum move to go-simd/floats go-ndarray/ndarray#98 stays there.ndarray/internal/kernelsgoes away, and ndarray imports the library.Shape of the module
blasandlapack. Reference-LAPACK ships its reference BLAS in the same repository, julia's LBT needs both, and a single version avoids a lockstep pair.go-lapackexists but is not ours: it was created 2018-08-27, has 0 public repositories, and we have no readable membership.go-blasdoes not exist. Only the maintainer can create an org; I am asking.int. That is already ILP64 on 64-bit GOARCH. Mappingdgemm_64_and friends stays in julia's LBT layer.info < 0(the xerbla convention). Slice lengths are checked againstlda·(n−1)+mwith overflow-safe arithmetic, for 386 and arm.Order (one PR each, once the org exists)
Solve/Inv/Det/Cholesky/QRmove onto them, and the 1,040 cases still pass. The allocation fixes of perf: linalg SVD, Pinv, Lstsq, Eigh, QR, Eig allocate 90-276 times per 64x64 call go-ndarray/ndarray#104 land here.go-ndarray does the extraction. go-embedded-julia/julia#67 can forward to the library from step 2 for
A * x, and from step 3 for\.README: the BLAS limit is #67, not #33 (#68)