Build and change models via math-spec

.github/workflows/test-notebooks.yml

@@ -30,7 +30,7 @@ jobs:
     - name: Install package and dependencies
       run: |
         python -m pip install uv
-        uv pip install --system -e ".[docs]"
+        uv pip install --system -e ".[docs]" --group spec
 
     - name: Execute notebooks
       run: |

.github/workflows/test.yml

@@ -82,7 +82,7 @@ jobs:
     - name: Install package and dependencies
       run: |
         python -m pip install uv
-        uv pip install --system "$(ls dist/*.whl)[dev,solvers,oetc]"
+        uv pip install --system "$(ls dist/*.whl)[dev,solvers,oetc]" --group spec
 
     - name: Test with pytest
       env:
@@ -120,7 +120,7 @@ jobs:
     - name: Install package and dependencies
       run: |
         python -m pip install uv
-        uv pip install --system "$(ls dist/*.whl)[dev]"
+        uv pip install --system "$(ls dist/*.whl)[dev]" --group spec
 
     - name: Run type checker (mypy)
       run: |

.readthedocs.yaml

@@ -8,6 +8,12 @@ build:
   jobs:
     pre_system_dependencies:
     - git fetch --unshallow # Needed to get version tags
+    post_install:
+    # The spec API is documented via autodoc, which imports linopy.spec and so
+    # needs math-spec. It lives in the `spec` dependency group, not the docs
+    # extra; --group needs pip >= 25.1, hence the upgrade.
+    - python -m pip install --upgrade pip
+    - python -m pip install --group spec
 python:
   install:
     - method: pip

benchmarks/models/__init__.py

@@ -21,5 +21,6 @@
     qp,
     sos,
     sparse_network,
+    spec_pypsa,
     storage,
 )

benchmarks/models/spec_pypsa.py

@@ -0,0 +1,55 @@
+"""
+Model built from math-spec's ``pypsa.yaml`` example (requires math-spec).
+
+The subject is :meth:`linopy.Model.from_spec`: lowering a spec of PyPSA's full
+statement, binding synthetic data to it and building every variable and
+constraint it declares. The example lives outside the wheel, so its directory
+comes from ``MATH_SPEC_EXAMPLES`` and the case skips without it. A sweep
+value is the number of labels per dimension; 40 of them is about 20k
+variables.
+"""
+
+from __future__ import annotations
+
+import os
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+from benchmarks.registry import BUILD, FROM_NETCDF, TO_NETCDF, BenchSpec, register
+
+if TYPE_CHECKING:
+    import linopy
+
+SIZES = (5, 40)
+
+EXAMPLES = os.environ.get("MATH_SPEC_EXAMPLES")
+EXAMPLE = Path(EXAMPLES, "pypsa.yaml") if EXAMPLES else None
+
+
+def build_spec_pypsa(n: int) -> linopy.Model:
+    """Lower ``pypsa.yaml`` and build it with ``n`` labels per dimension."""
+    import pytest
+
+    if EXAMPLE is None or not EXAMPLE.exists():
+        pytest.skip("set MATH_SPEC_EXAMPLES to a math-spec examples directory")
+    import math_spec
+
+    import linopy
+    from linopy.spec.testing import synthetic_sources
+
+    path = str(EXAMPLE)
+    sources = synthetic_sources(math_spec.to_program(path), n)
+    with linopy.options as options:
+        options["semantics"] = "v1"
+        return linopy.Model.from_spec(path, sources)
+
+
+SPEC = register(
+    BenchSpec(
+        name="spec_pypsa",
+        build=build_spec_pypsa,
+        sweep=SIZES,
+        phases=frozenset({BUILD, TO_NETCDF, FROM_NETCDF}),
+        requires=("math_spec",),
+    )
+)

conftest.py

@@ -0,0 +1,9 @@
+"""Root pytest configuration for ``--doctest-modules`` collection of ``linopy/``."""
+
+from __future__ import annotations
+
+from importlib.util import find_spec
+
+collect_ignore: list[str] = []
+if find_spec("math_spec") is None:
+    collect_ignore.append("linopy/spec")

doc/api.rst

@@ -115,6 +115,36 @@ IO
    model.Model.to_netcdf
    io.read_netcdf
 
+Building from specs
+-------------------
+
+Build a model from a `math-spec
+<https://github.com/energy-models/math-spec>`__ YAML program attached to
+data. Requires the ``spec`` dependency group.
+
+A spec's grouped sums and windows build dense by default, which is wasteful on
+a skewed topology (a lookup with a few large groups and many small ones, or a
+wide ``sum_back`` window). Set ``linopy.options["sparse_groupby"] = True`` (v1
+semantics) to back grouped sums with :mod:`linopy.csr`, and build the model
+with ``Model(freeze_constraints=True)`` to keep the constraints CSR-backed
+instead of densifying them.
+
+.. autosummary::
+   :toctree: generated/
+
+   model.Model.add_spec
+   model.Model.from_spec
+   model.Model.spec
+   spec.ModelSpec
+   spec.Layer
+   spec.NamedExpressions
+   spec.NamedExpression
+   spec.Declaration
+   spec.Unspecified
+   spec.attach
+   spec.Attached
+   spec.SpecDataError
+
 
 Variable
 ========

doc/building-models-from-specs.nblink

@@ -0,0 +1,3 @@
+{
+    "path": "../examples/building-models-from-specs.ipynb"
+}

doc/contributing.rst

@@ -45,6 +45,9 @@ To run the test suite:
     # Install development dependencies
     uv sync --extra dev --extra solvers
 
+    # Also run the math-spec binder tests (needs Python >= 3.12)
+    uv sync --extra dev --extra solvers --group spec
+
     # Run all tests
     pytest
 

doc/index.rst

@@ -116,6 +116,7 @@ This package is published under MIT license.
    coordinate-alignment
    migrating-to-v1
    manipulating-models
+   building-models-from-specs
 
 .. toctree::
    :hidden:

doc/release_notes.rst

@@ -21,6 +21,17 @@ Upcoming Version
 * Every operation whose result changes under v1 emits a ``LinopySemanticsWarning`` under legacy, naming the fix — so a model can be migrated incrementally before opting in. The full rules are specified in :doc:`the arithmetic convention <design/convention>`.
 
 
+*Build a model from a math-spec program*
+
+* ``Model.from_spec`` / ``model.add_spec`` build a model from a `math-spec <https://github.com/energy-models/math-spec>`__ program attached to data, and ``model.spec`` reads it back as named layers, each with its ``program``, ``parameters``, ``coords`` and ``lookups``. Requires the ``spec`` dependency group (``uv sync --group spec``, Python >= 3.12) and v1 semantics. The API emits an :class:`linopy.EvolvingAPIWarning` once per session while it stabilises. See :doc:`building-models-from-specs`.
+
+* A spec can extend a hand-built model: a ``Variable`` passed in ``sources`` under a declared name binds it instead of building one. Everything a layer builds carries its layer name in ``.spec``; ``model.remove_spec(name)`` unwinds a layer, and removing a spec-owned name any other way is refused. ``model.spec.unspecified`` reports what the model holds beyond its specs.
+
+* ``model.spec.expressions`` evaluates the spec's named expressions against the solved model, and ``model.spec.typeset`` (``to_latex`` / ``to_markdown`` / ``to_typst``) renders the spec, warning where the model has drifted from it.
+
+* Spec models round-trip through ``to_netcdf`` / ``read_netcdf``; a file read without ``math-spec`` installed loads as a plain model with a warning.
+
+
 *Numerical scaling*
 
 * Variables, constraints and the objective accept a ``scaling`` factor that rewrites the problem into better-behaved units for the solver, without changing the answer. Variable scaling is column-like, constraint and objective scaling are row-like, and primal values, duals and the objective are transformed back to the original units after solving. See the :doc:`numerical-scaling` tutorial and the *Numerical scaling* section of the :doc:`user-guide`.

doc/user-guide.rst

@@ -83,6 +83,20 @@ bound, swap a constraint, or copy it for what-if analysis.
   variables.
 
 
+Building a model from a spec
+-----------------------------
+
+Instead of calling ``add_variables`` / ``add_constraints`` directly,
+you can declare a model as a `math-spec
+<https://github.com/energy-models/math-spec>`__ YAML program attached to
+data, and let linopy build it.
+
+- :doc:`building-models-from-specs` — ``Model.from_spec`` and
+  ``model.add_spec``, attaching data to a spec, and reading named
+  expressions back through ``model.spec`` after solving. Requires the
+  ``spec`` dependency group and v1 semantics.
+
+
 Where to go next
 ----------------