Replace uBLAS storage with vector/mdspan matrices and evaluate performance

.github/workflows/build.yml

@@ -22,14 +22,17 @@ jobs:
             os: ubuntu-24.04
             compiler: gcc
             configure_debug: --disable-debug
+            std_mdspan: 0
           - name: GCC 14 debug
             os: ubuntu-24.04
             compiler: gcc
             configure_debug: --enable-debug
+            std_mdspan: 0
           - name: Apple Clang release
             os: macos-15
             compiler: apple-clang
             configure_debug: --disable-debug
+            std_mdspan: 1
 
     steps:
       - name: Check out sources
@@ -93,6 +96,8 @@ jobs:
             ${{ matrix.configure_debug }}
           grep -Eq '^CXX = .* -std=c\+\+23([[:space:]]|$)' Makefile
           grep -q '^#define INTARNA_MULITHREADING 1' src/config.h
+          grep -q '^#define INTARNA_USE_STD_MDSPAN ${{ matrix.std_mdspan }}$' src/config.h
+          grep -q '^#define INTARNA_USE_STD_MDSPAN ${{ matrix.std_mdspan }}$' src/IntaRNA/intarna_config.h
 
       - name: Build, install, and run full test suite
         run: |
@@ -115,7 +120,9 @@ jobs:
       - name: Compile and link an installed pkg-config consumer
         run: |
           consumer="$RUNNER_TEMP/intarna-consumer.cpp"
-          printf '#include <IntaRNA/RnaSequence.h>\nint main() { IntaRNA::RnaSequence rna("test", "ACGU"); return rna.size() == 4 ? 0 : 1; }\n' > "$consumer"
+          printf '#include <IntaRNA/RnaSequence.h>\n#include <IntaRNA/Matrix.h>\nint main() { IntaRNA::RnaSequence rna("test", "ACGU"); IntaRNA::Matrix<int> matrix(2, 3); matrix(1, 2) = 42; return rna.size() == 4 && matrix(1, 2) == 42 ? 0 : 1; }\n' > "$consumer"
+          test -f "$RUNNER_TEMP/intarna-install/include/IntaRNA/mdspan/LICENSE"
+          test -f "$RUNNER_TEMP/intarna-install/include/IntaRNA/experimental/LICENSE"
           export PKG_CONFIG_PATH="$RUNNER_TEMP/intarna-install/lib/pkgconfig"
           "$CXX" -std=c++23 ${CPPFLAGS:-} ${CXXFLAGS:-} "$consumer" \
             ${LDFLAGS:-} $(pkg-config --cflags --libs IntaRNA) \

AGENTS.md

@@ -33,6 +33,12 @@ pkg-config and Autotools; OpenMP is used by the default multithreaded build.
 [conda-build-env.yml](conda-build-env.yml) supplies library/build dependencies,
 but does not select the C++ compiler. Use the platform setup in CI when needed.
 
+Configure prefers native `std::mdspan` and otherwise uses the bundled Kokkos
+headers. `INTARNA_USE_STD_MDSPAN` in the installed public configuration records
+the choice; `--with-mdspan=std|kokkos` can select a backend explicitly. Preserve
+the upstream headers and licenses in `src/mdspan` and `src/experimental` when
+editing project code; their provenance is recorded in `doc/mdspan-storage.md`.
+
 From the repository root, with dependencies available in standard locations:
 
 ```bash

ChangeLog

@@ -24,6 +24,11 @@
 ## Technical changes and Optimizations
 
 - add repository-specific AI coding guidance in AGENTS.md (issue #247)
+- replace uBLAS storage matrices with std::vector and std::mdspan, retaining
+  compact upper-band and triangular storage; configure selects native mdspan
+  or bundled Kokkos headers while preserving GCC 14 support; document the
+  public matrix operations and align their layout with repository guidance
+
 - reduce temporary allocations and fix cleanup on error paths
 - remove redundant accessibility checks from predictor, helix and seed handling
 - C++23 required; build and installation checks with GCC 14 and Apple Clang
@@ -52,6 +57,38 @@
 ################################################################################  
 ################################################################################
 
+260929 Alexander Mitrofanov
+ * configure.ac, IntaRNA/Matrix.h, IntaRNA/intarna_config.h.in:
+   + prefer native mdspan, falling back to bundled Kokkos with a public
+     INTARNA_USE_STD_MDSPAN define; allow explicit --with-mdspan selection
+ * src/mdspan, src/experimental:
+   + vendor only the Kokkos headers and both copies of its license from
+     stable revision 8989f70749e28f337e6f7aa210db88659dba6f2f
+   + distribute and install the headers with IntaRNA for external consumers
+ * CI, README.md, doc/mdspan-storage.md:
+   + restore GCC 14 release/debug builds and document automatic fallback
+   + record a clean GCC 14/Kokkos comparison against uBLAS with all outputs
+     matching across 160 executions; retain the original native measurements
+ * IntaRNA/Matrix.h, AGENTS.md:
+   + document public matrix operations and place explicit inline template
+     definitions after class declarations, following the repository guidance
+   + keep the guidance aligned with native/bundled mdspan selection (issue #246)
+
+260928
+ * IntaRNA/Matrix.h and storage aliases:
+   + replace uBLAS dense, upper-band and upper-triangular storage with owned
+     std::vector data accessed through C++23 std::mdspan
+   + preserve logical resize, copy/move ownership and compact storage
+ * configure.ac, README.md:
+   + probe native mdspan library support and document the toolchain requirement
+ * tests/Matrix_test.cpp, tests/benchmark/compare-matrix-storage.py:
+   + add storage regression tests and reproducible before/after measurements
+   + terminate the timed process group on timeout or interruption, with
+     regression checks
+   + include the biological benchmark inputs in source distributions
+ * doc/mdspan-storage.md:
+   + record validation and performance evaluation for issue #246
+
 260928 Alexander Mitrofanov
  * AGENTS.md :
    + repository-wide AI coding guidance based on the current code and build setup

README.md

@@ -180,7 +180,10 @@ you with an encapsulated IntaRNA installation.
 If you are going to compile IntaRNA from source, ensure you meet the following
 dependencies:
 
-- compiler supporting C++23 and OpenMP (GCC 14 or Apple Clang)
+- compiler supporting C++23 and OpenMP (GCC 14 or Apple Clang). Configure
+  prefers native `std::mdspan` when available and otherwise uses the bundled
+  header-only [Kokkos mdspan implementation](https://github.com/kokkos/mdspan).
+  No separate mdspan installation or compiler upgrade is required.
 - [boost C++ library](http://www.boost.org/) version >= 1.50.0
   (ensure the following libraries are installed for development (not just runtime libraries!); or install all e.g. in Ubuntu via package `libboost-all-dev`)
     - libboost_regex
@@ -2191,6 +2194,16 @@ select `-std=c++23` (or a newer standard) in their own build; the installed
 pkg-config metadata supplies IntaRNA and dependency flags but deliberately
 does not override the consuming project's language mode.
 
+Configure records the selected mdspan implementation in the installed
+`IntaRNA/intarna_config.h`. By default (`--with-mdspan=auto`), native
+`std::mdspan` is preferred; the bundled Kokkos implementation is selected if
+the standard library does not provide the required API. Use
+`--with-mdspan=std` or `--with-mdspan=kokkos` to require a particular backend.
+The Kokkos headers and licenses are installed below `include/IntaRNA/mdspan`
+and `include/IntaRNA/experimental`, so the public matrix header works without
+adding another include path. See [the storage report](doc/mdspan-storage.md)
+for the pinned upstream revision, validation, and performance results.
+
 ## Mandatory `Easylogging++` initalization !
 
 Since IntaRNA makes heavy use of the `Easylogging++` library, you have to add (and adapt)
@@ -2236,4 +2249,3 @@ flags are used within the IntaRNA configuration:
 [![no](doc/figures/icon-no.39.png)](https://www.freepik.com/free-vector/icons-collection_1638275.htm)
 [![up](doc/figures/icon-up.38.png)](https://www.freepik.com/free-vector/colored-arrows_794372.htm)
 Designed by Freepik
-

configure.ac

@@ -87,6 +87,60 @@ sink << stream.view();
 	 AC_MSG_ERROR([A C++23 standard library providing std::string::contains, std::string::resize_and_overwrite, and std::stringstream::view is required. Upgrade libstdc++/libc++ or select a newer C++23 toolchain.])])
 CXXFLAGS=$intarna_saved_CXXFLAGS
 
+# Prefer the standard implementation, falling back to bundled Kokkos headers.
+# The selected backend is also recorded in the installed public configuration.
+AC_ARG_WITH([mdspan],
+	[AS_HELP_STRING([--with-mdspan=auto|std|kokkos],
+	    [select mdspan implementation (def=auto: prefer std, otherwise Kokkos)])],
+	[intarna_mdspan="$withval"], [intarna_mdspan=auto])
+AS_CASE([$intarna_mdspan], [auto|std|kokkos], [],
+	[AC_MSG_ERROR([--with-mdspan must be auto, std, or kokkos])])
+
+INTARNA_USE_STD_MDSPAN=0
+AS_IF([test x"$intarna_mdspan" != x"kokkos"], [
+AC_MSG_CHECKING([for C++23 std::mdspan over std::vector])
+AC_COMPILE_IFELSE(
+	[AC_LANG_PROGRAM(
+		[[#include <mdspan>
+#include <vector>
+#ifndef __cpp_lib_mdspan
+#error C++23 std::mdspan is unavailable
+#endif]],
+		[[std::vector<int> storage(6);
+std::mdspan<int, std::dextents<std::size_t, 2>> matrix(storage.data(), 2, 3);
+matrix[1, 2] = 42;
+std::mdspan<const int, std::dextents<std::size_t, 1>> packed(storage.data(), 6);
+return packed[5] != 42;]])],
+	[AC_MSG_RESULT([yes])
+	 INTARNA_USE_STD_MDSPAN=1],
+	[AC_MSG_RESULT([no])
+	 AS_IF([test x"$intarna_mdspan" = x"std"],
+		[AC_MSG_ERROR([--with-mdspan=std requires a C++23 standard library providing std::mdspan. Use --with-mdspan=auto or --with-mdspan=kokkos for the bundled implementation.])])])
+])
+
+AS_IF([test "$INTARNA_USE_STD_MDSPAN" = 0], [
+	AC_MSG_CHECKING([for bundled Kokkos mdspan over std::vector])
+	intarna_saved_CPPFLAGS=$CPPFLAGS
+	CPPFLAGS="-I$srcdir/src $CPPFLAGS"
+	AC_COMPILE_IFELSE(
+		[AC_LANG_PROGRAM(
+			[[#include "mdspan/mdspan.hpp"
+#include <vector>
+namespace intarna_mdspan = MDSPAN_IMPL_STANDARD_NAMESPACE;]],
+			[[std::vector<int> storage(6);
+intarna_mdspan::mdspan<int, intarna_mdspan::dextents<std::size_t, 2>> matrix(storage.data(), 2, 3);
+matrix[1, 2] = 42;
+intarna_mdspan::mdspan<const int, intarna_mdspan::dextents<std::size_t, 1>> packed(storage.data(), 6);
+return packed[5] != 42;]])],
+		[AC_MSG_RESULT([yes])],
+		[AC_MSG_RESULT([no])
+		 AC_MSG_ERROR([The bundled Kokkos mdspan headers do not compile with this C++23 toolchain. See config.log and ensure src/mdspan and src/experimental are complete.])])
+	CPPFLAGS=$intarna_saved_CPPFLAGS
+])
+AC_DEFINE_UNQUOTED([INTARNA_USE_STD_MDSPAN], [$INTARNA_USE_STD_MDSPAN],
+	[Use standard mdspan (1) or the bundled Kokkos implementation (0)])
+AC_SUBST([INTARNA_USE_STD_MDSPAN])
+
 # check if python is available
 AM_PATH_PYTHON([$PYTHON_REQUIRED_VERSION],, [:])
 AM_CONDITIONAL([HAVE_PYTHON], [test "$PYTHON" != ":"])

doc/handson/README.md

@@ -5,9 +5,9 @@ In the following, we list some examples how to use IntaRNA in specific applicati
 The examples are presented in detail in our publication
 
 - [How to do RNA-RNA interaction prediction? A use-case driven
-handbook using IntaRNA](http://www.bioinf.uni-freiburg.de/Subpages/publications.html?de#Raden-IntaRNA-handson.abstract)
+handbook using IntaRNA](https://doi.org/10.1007/978-1-0716-3519-3_9)
   - Martin Raden and Milad Miladi
-  - Springer (in press, DOI to come)
+  - In: Lorenz, R. (eds) RNA Folding. Methods in Molecular Biology, vol 2726. Humana, New York, NY. https://doi.org/10.1007/978-1-0716-3519-3_9