Skip to content

Parallel solving: bundled MKL PARDISO in a separate program, on every platform - #2

Merged
kylebme merged 17 commits into
workbench-uifrom
workbench-ui-4w5t2s
Oct 8, 2026
Merged

kylebme merged 17 commits into
workbench-uifrom
workbench-ui-4w5t2s

Conversation

@kylebme

@kylebme kylebme commented Oct 7, 2026 •

Copy link
Copy Markdown
Owner

What changes

  • CalculiX 2.23 is built from source on macOS (Apple silicon), Linux x64 and Windows x64. scripts/build-solver.py builds it from a conda-forge environment per platform (native/calculix/solver-env-*.yml) and leaves a self-contained solver/ folder (scripts/solverlibs.py).
  • Licensing structure. CalculiX's source files are GPL version 2 only, so nothing under GPL-3, LGPL-3 or Intel's license is linked into ccx:
    • PaStiX and Scotch are removed. PaStiX is LGPL-3.0.
    • PARDISO on macOS stays Apple Accelerate, a system library.
    • PARDISO on Linux and Windows runs in sprung-solve (native/sprung-solve, MIT), a separate program that links Intel oneMKL statically.
      • ccx's pardiso_ (native/calculix/pardiso_client.c) starts it and sends each factorization and solve through a pipe, by the documented protocol in docs/solver-protocol.md.
      • sprung-solve also solves Matrix Market files on its own.
      • It ends with ccx, even mid-factorization: on Linux through the parent-death signal, on Windows through a job object.
      • It keeps PARDISO's analysis while the matrix structure is unchanged, so nonlinear iterations only refactor.
    • Build check: the build refuses a CalculiX release whose license notices are not GPL-2-only.
  • MKL ships with the app. Packaged builds carry runtime/solver/sprung-solve/ with:
    • sprung-solve itself;
    • its OpenMP runtime (GNU on Linux, LLVM libiomp5md.dll on Windows);
    • its MIT license, Intel's license and Intel's third-party notices.
  • Size and build: statically linked, sprung-solve is about 105 MB (15 MB compressed). It is built by GCC on Linux and by MSVC on Windows, since MKL's static libraries are made for MSVC, not MinGW. build-solver.py --without-mkl builds a SPOOLES-only solver.
  • Engine:
    • It offers PARDISO only when sprung-solve --version solves a test system on this computer. Otherwise it falls back to SPOOLES.
    • Eigenvalue analyses now use PARDISO too.
    • SPRUNG_FEA_DIRECT_SOLVER=pardiso|spooles forces one solver.
  • PARDISO accuracy fix. pardisoinit's two refinement steps per solve corrupted a free part's frequency analysis: the first elastic mode came out as a spurious seventh rigid mode. CalculiX linked to MKL directly has the same problem. sprung-solve now refines only after perturbed pivots, and tests/test_solvers.py compares a free part's modes across solvers.
  • CI:
    • Every platform builds ccx and sprung-solve and runs tests/test_solvers.py: static, frequency, buckling, a free part, a nonlinear frictional bolted joint, the SPOOLES fallback, and sprung-solve on Matrix Market systems.
    • The packaged app then has to solve static and frequency studies with PARDISO from what it ships, with no MKL installed on the runner.
  • Notices. THIRD_PARTY_NOTICES.md, README.md and IMPLEMENTATION.md describe the mixed-license distribution:
    • The Sprung FEA source is GPL-3.0-or-later.
    • CalculiX is GPL-2.0-only and runs as a separate program.
    • sprung-solve is MIT, with oneMKL under the Intel Simplified Software License.
    • License texts for components no longer shipped are removed.

Results

Static beam solves on a 4-core Linux machine:

Equations SPOOLES (threaded) PARDISO via sprung-solve
136k 12.7 s 5.7 s
417k 94 s 27 s

MKL loaded inside ccx took 6.0 s on the 136k case, so the pipe adds no measurable time.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HCMo1BnyzJo8oewdckpPUd

kylebme and others added 17 commits October 6, 2026 11:15
build-solver.py applied whatever .patch files it found, so a missing or
untracked patch still produced a solver that starts but whose threaded
SPOOLES solves silently return wrong answers. It now checks the patch
folder against the expected list and stops, naming any missing or
unexpected patch, before downloading or building anything.

Also adds the SOLIDWORKS Simulation gap audit (docs/solidworks-fea-gap-audit.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCMo1BnyzJo8oewdckpPUd
CalculiX is now built from source on Apple silicon, Linux x64 and Windows
x64 by one script, scripts/build-solver.py, from a conda-forge environment
per platform (native/calculix/solver-env-*.yml), with three multithreaded
direct solvers:

- PaStiX 6.4 through native/calculix/pastix_ccx.c, which replaces
  CalculiX's pastix.c (written for a PaStiX fork) with the same entry
  points on released PaStiX: LDLT/LU, Scotch ordering, the ordering kept
  across a nonlinear step's refactorizations, and refinement only after
  static pivoting. conda-forge ships PaStiX for macOS and Linux; on
  Windows the script builds it from source.
- PARDISO: Apple Accelerate on macOS as before; on Windows and Linux,
  Intel MKL's, loaded at run time from an MKL the user has installed
  (native/calculix/mkl_pardiso.c). MKL is proprietary and is never linked
  or bundled with the GPL solver.
- SPOOLES with the thread-safety patches, now compiled portably.

The engine picks the fastest available (PARDISO, PaStiX, SPOOLES), finds
an installed MKL (calculix.find_mkl), and keeps PaStiX out of eigenvalue
analyses, which CalculiX sends to PARDISO when both are built in.
scripts/solverlibs.py leaves solver/ self-contained (one copy of each
library, references renamed to it) and is shared with bundle-runtime.py.
CI builds the solver on every platform, compares all direct solvers
(tests/test_solvers.py, now including a nonlinear frictional bolted joint),
and checks that the packaged app solves with PaStiX or PARDISO.

On a 4-core Linux machine a 137k-equation static beam solves in 18.0 s
with threaded SPOOLES, 8.9 s with PaStiX and 6.0 s with MKL PARDISO.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCMo1BnyzJo8oewdckpPUd
Keep the SPOOLES patches byte-exact on Windows checkouts (CRLF broke them),
give conda's macOS gfortran the clang it assembles with, and skip a source
file CalculiX 2.23's Makefile.inc lists but the release lacks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCMo1BnyzJo8oewdckpPUd
conda's MinGW gcc loads its DLLs from the environment, which CI does not
activate, so it exited without output. Every tool now runs with the
environment's folders first on PATH, and a failed compile reports its
exit code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCMo1BnyzJo8oewdckpPUd
conda-forge's Windows BLAS packages hold only DLLs, so PaStiX's CMake found
no BLAS. Use the openblas package's import library and headers, link the
MinGW-built ARPACK from Library/mingw-w64, pin 64-bit-index Scotch to match
PASTIX_INT64, and pin the PaStiX 6.4.0 archive's SHA-256.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCMo1BnyzJo8oewdckpPUd
…crashes

Windows: conda-forge's Scotch is a static library built by Microsoft's
compiler, which PaStiX's MinGW build could not link. The build now wraps
it into a MinGW scotch.dll exporting Scotch's API, with
native/calculix/msvc_compat.c supplying the few runtime pieces MSVC
objects expect (the /GS cookie and handler, the UCRT's inline stdio
functions, __isa_available; __chkstk is MinGW's ___chkstk_ms). Checked
locally with conda-forge's Linux-hosted MinGW cross compiler.

The engine now says when CalculiX stopped without reporting an error (a
signal, or an exit or Windows exception code) instead of only suggesting
the model is at fault. CI's solver comparison turns on PaStiX's verbose
output while PaStiX on macOS stops during its analysis.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCMo1BnyzJo8oewdckpPUd
macOS: conda-forge's PaStiX pairs it with Scotch 7.0.5, whose threaded
ordering aborts there (CI: SIGABRT in the Scotch ordering), and pulls in
hwloc with libxml2 and ICU. Build PaStiX from its release archive on macOS
as on Windows, with 64-bit indices like conda-forge's Linux build,
conda's current 64-bit Scotch, single-threaded Scotch ordering and no
hwloc.

Windows: PaStiX's code without hwloc calls an undeclared function, which
GCC 14 rejects; build it with the same narrow leniency flags as CalculiX.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCMo1BnyzJo8oewdckpPUd
PaStiX 6.4 without hwloc calls a pastix_warning that does not exist (the
Windows link failed) and, on macOS, sysctl without its header. The build
now patches that one file: the notice goes to stderr through fprintf, and
macOS gets the sysctl header.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCMo1BnyzJo8oewdckpPUd
…output

PaStiX's test helper library uses pthreads without linking them, which
MinGW does not do by itself; the library and its kernels already linked.
macOS's PaStiX now passes, so CI no longer turns on its verbose output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCMo1BnyzJo8oewdckpPUd
SPM's install rules name some headers with backslashes, which CMake's
generated cmake_install.cmake then reads as escapes ('\s'), so the
Windows install step failed after a complete build. CMake writes every
other path there with '/', so the Windows build rewrites them before
installing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCMo1BnyzJo8oewdckpPUd
ccx_2.23.c and ccx_2.23step.c call _set_output_format outside any
function under __WIN32, which GCC 14 rejects. It chose two-digit
exponents in the old msvcrt; the UCRT prints them anyway.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCMo1BnyzJo8oewdckpPUd
On Windows, CalculiX writes to a pipe, which buffers C output whole: the
'Factoring ... PaStiX' lines now reach the engine as they happen, and
with PASTIX_VERBOSE set stdout is unbuffered so PaStiX's own trace
survives a crash. CI turns that on for Windows, where PaStiX stops with
an access violation while MKL's PARDISO and SPOOLES pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCMo1BnyzJo8oewdckpPUd
Windows: PaStiX died with heap corruption (0xc0000374) inside Scotch's
ordering, in conda-forge's Scotch built by Microsoft's compiler and
linked into MinGW code. The build now compiles Scotch 7.0.11 from its
release archive with the MinGW compilers (static, 64-bit indices, one
thread, no compressed graph files) and drops native/calculix/msvc_compat.c
and conda's Scotch, compression libraries and hwloc from the Windows
environment.

Tests: buckling factors from MKL's PARDISO and SPOOLES differed by a few
units in the 7th digit, ARPACK's own tolerance with factorizations that
differ by round-off; compare eigenvalues to 1e-6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCMo1BnyzJo8oewdckpPUd
Without threads Scotch handles compressed graph files by forking, which
Windows lacks (its tools failed to link). With COMMON_PTHREAD_FILE it uses
a thread; checked locally that the library then needs no fork or waitpid.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCMo1BnyzJo8oewdckpPUd
PaStiX passes on Windows now. The Windows solver bundles vcruntime140.dll
for conda-forge's OpenBLAS; the notices say so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCMo1BnyzJo8oewdckpPUd
CalculiX's source files are GPL version 2 only, so neither PaStiX
(LGPL-3.0) nor Intel oneMKL can be linked into it. Both are now out of
ccx:

- PaStiX, Scotch and their builds are removed. The engine's direct
  solvers are PARDISO, then SPOOLES.
- sprung-solve (native/sprung-solve, MIT) is a separate program that links
  oneMKL statically and solves sparse systems sent through its standard
  input. The protocol is documented in docs/solver-protocol.md, and the
  program also solves Matrix Market files from the command line.
  native/calculix/pardiso_client.c replaces mkl_pardiso.c: CalculiX's
  pardiso_ starts sprung-solve and sends it each factorization and solve.
  The helper ends with CalculiX (Linux parent-death signal, Windows job
  object) and keeps PARDISO's analysis while the structure is unchanged.
- PARDISO refines only after perturbed pivots. pardisoinit's two refinement
  steps per solve turned a free part's first elastic mode into a seventh
  rigid one; CalculiX linked to MKL directly has the same problem.
  tests/test_solvers.py now compares a free part's modes across solvers.
- build-solver.py builds sprung-solve with GCC on Linux and Microsoft's
  compiler on Windows, from conda-forge's mkl-static, into
  solver/sprung-solve with its MIT license and Intel's license and
  third-party notices. It also refuses a CalculiX release whose license
  notices are not GPL version 2 only. --without-mkl builds SPOOLES only.
  bundle-runtime.py ships the helper folder.
- The engine offers PARDISO only when `sprung-solve --version` solves on
  this computer, and otherwise falls back to SPOOLES.
- CI no longer installs a user MKL. The packaged app must solve static and
  frequency studies with PARDISO from what it ships.
- THIRD_PARTY_NOTICES.md, README.md and IMPLEMENTATION.md describe the
  mixed-license distribution: GPL-3.0-or-later Sprung FEA source,
  GPL-2.0-only CalculiX as a separate program, and MIT sprung-solve with
  oneMKL under Intel's license. License texts for the components no longer
  shipped are removed.

On a 4-core Linux machine, static beam solves take 5.7 s at 136k equations
(threaded SPOOLES 12.7 s) and 27 s at 417k (SPOOLES 94 s). The pipe adds
no measurable time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCMo1BnyzJo8oewdckpPUd
@kylebme kylebme changed the title Parallel solving: PaStiX and optional MKL PARDISO on every platform Parallel solving: bundled MKL PARDISO in a separate program, on every platform Oct 8, 2026
@kylebme
kylebme marked this pull request as ready for review October 8, 2026 02:58
@kylebme
kylebme merged commit 7fe05ce into workbench-ui Oct 8, 2026
3 checks passed
@kylebme
kylebme deleted the workbench-ui-4w5t2s branch October 8, 2026 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants