New matrix variable functionality (#957)

* Create branch

* start of tests

* Added extra dimension

* Update tests

* First working test

* First working test

* Expr from matrix variables test

* Working version of ExprCons with le

* Can now add linear matrix constraints

* More robust Matrix classes

* add todo

* Clean up addMatrixCons function

* Add MatrixVar methods

* Update TODOs

* Add getter functions to matrixConstraint class@
git push

* Add checks for missing numpy

* Add matrix API to readthedocs

* Add minor grammar stuff. Add GCG to similar software

* Remove universal build. Add type asserts

* Update pipelines to include numpy

* Add numpy as a dependency

* Remove cimports for now

* Support adding MatrixExprCons in addConss

* Changed tests a bit

* Support querying solution values using MatrixExpr

* Add a test

* Remove SCIP bug from tests. Already reported

* Update CHANGELOG

* Add info for 5.4.0 release

* Make numpy a dependency (not just at build time)

* Most comments addressed

* Add test for documentation

* Fix documentation

* Add solution value accessing to docs

* Format test file

* Update docs/tutorials/matrix.rst

Co-authored-by: Mohammed Ghannam <[email protected]>

* Update docs/tutorials/matrix.rst

Co-authored-by: Mohammed Ghannam <[email protected]>

---------

Co-authored-by: Mark Turner <[email protected]>
Co-authored-by: Mohammed Ghannam <[email protected]>
Co-authored-by: Mark Turner <[email protected]>
Co-authored-by: Mohammed Ghannam <[email protected]>

.github/workflows/coverage.yml

@@ -34,7 +34,7 @@ jobs:
       - name: Prepare python environment
         run: |
           python -m pip install --upgrade pip
-          python -m pip install networkx cython pytest-cov
+          python -m pip install networkx cython pytest-cov numpy
 
       - name: Install PySCIPOpt
         run: |

.github/workflows/integration-test.yml

@@ -34,7 +34,7 @@ jobs:
       - name: Prepare python environment
         run: |
           python -m pip install --upgrade pip
-          python -m pip install networkx pytest-cov
+          python -m pip install networkx pytest-cov numpy
 
       - name: Install PySCIPOpt
         run: python -m pip install .
@@ -70,7 +70,7 @@ jobs:
         shell: powershell
         run: |
           python -m pip install --upgrade pip
-          python -m pip install networkx pytest-cov
+          python -m pip install networkx pytest-cov numpy
 
       - name: Install PySCIPOpt
         shell: powershell
@@ -107,7 +107,7 @@ jobs:
       - name: Prepare python environment
         run: |
           python -m pip install --upgrade pip
-          python -m pip install networkx pytest-cov pytest
+          python -m pip install networkx pytest-cov pytest numpy
 
       - name: Install PySCIPOpt
         run: |

CHANGELOG.md

@@ -2,12 +2,22 @@
 
 ## Unreleased
 ### Added
+### Fixed
+### Changed
+### Removed
+
+## 5.4.0 - 2024.02.24
+### Added
 - Added option to get Lhs, Rhs of nonlinear constraints
 - Added cutoffNode and test
 - Added getMajorVersion, getMinorVersion, and getTechVersion
+- Added addMatrixVar and addMatriCons
+- Added MatrixVariable, MatrixConstraint, MatrixExpr, and MatrixExprCons
 ### Fixed
+- Warning at Model initialisation now uses new version calls
 ### Changed
 ### Removed
+- Removed universal wheel type from setup.cfg (support for Python 2)
 
 ## 5.3.0 - 2025.02.07
 ### Added

docs/build.rst

@@ -22,7 +22,7 @@ To download SCIP please either use the pre-built SCIP Optimization Suite availab
     * - SCIP
       - PySCIPOpt
     * - 9.2
-      - 5.3  
+      - 5.3, 5.4+
     * - 9.1
       - 5.1, 5.2.x
     * - 9.0

docs/similarsoftware.rst

@@ -85,6 +85,7 @@ Software using PySCIPOpt
 
 This is software that is built on PySCIPOpt
 
+- `PyGCGOpt <https://github.com/scipopt/PyGCGOpt>`_: An extension of SCIP, using generic decompositions for solving MIPs
 - `GeCO <https://github.com/CharJon/GeCO>`_: Generators for Combinatorial Optimization
 - `scip-routing <https://github.com/mmghannam/scip-routing>`_:  An exact VRPTW solver in Python
 - `PySCIPOpt-ML <https://github.com/Opt-Mucca/PySCIPOpt-ML>`_:  Python interface to automatically formulate Machine Learning models into Mixed-Integer Programs

docs/tutorials/index.rst

@@ -14,6 +14,7 @@ more detailed information see `this page <https://www.scipopt.org/doc/html/index
   vartypes
   constypes
   expressions
+  matrix
   readwrite
   logfile
   branchrule
@@ -22,4 +23,5 @@ more detailed information see `this page <https://www.scipopt.org/doc/html/index
   heuristic
   nodeselector
   lazycons
-  eventhandler
+  eventhandler
+  scipdex

docs/tutorials/matrix.rst

@@ -0,0 +1,148 @@
+##############
+Matrix API
+##############
+
+In this overview of the matrix variable and constraint API in PySCIPOpt
+we'll walk through best practices for modelling them and the various information that
+can be extracted from them.
+
+For the following let us assume that a Model object is available, which is created as follows:
+
+.. code-block:: python
+
+  from pyscipopt import Model, quicksum
+
+  scip = Model()
+
+
+This tutorial should only be read after having an understanding of both the ``Variable``
+object (see :doc:`the constraint tutorial </tutorials/vartypes>`) and the ``Constraint``
+object (see :doc:`the constraint tutorial </tutorials/constypes>`).
+
+.. note::
+
+    The matrix API is built heavily on `numpy <https://numpy.org/>`_. This means that users can
+    use all standard ``numpy`` operations that they are familiar with when handling matrix
+    variables and expressions. For example, using the ``@``, ``matmul``, ``*``,
+    ``+``, ``hstack``, ``vstack``, and ``**`` operations work exactly as they do
+    when handling any standard ``numpy`` array.
+
+.. contents:: Contents
+
+What is a Matrix API?
+======================
+
+The standard approach explained in the variable and constraint tutorials, is to
+build each variable yourself (storing them in some data structure, e.g., a list or dict,
+with some loop), and to construct each constraint in a similar manner. That means building
+up each constraint yourself term by term. This approach is flexible, and still remains the standard,
+but an increasingly common trend is to view the modelling approach from a vector, matrix,
+and tensor perspective. That is, directly operate on larger sets of variables and expressions,
+letting python handle the interaction for each term. For such cases, it is encouraged
+that users now use the new matrix API!
+
+Matrix Variables
+=================
+
+Matrix variables are added via a single function call. It is important beforehand
+to know the ``shape`` of the new set of variables you want to create, where ``shape``
+is some ``tuple`` or ``int``. Below is an example for creating a 2x2 matrix variable
+of type continuous with an ub of 8.
+
+.. code-block:: python
+
+    x = scip.addMatrixVar(shape, vtype='C', name='x', ub=8)
+
+.. note::
+
+    The ``name`` of each variable in the example above becomes ``x_(indices)``
+
+In the case of each ``kwarg``, e.g., ``vtype`` and ``ub``, a ``np.array`` of explicit
+values can be passed. In the example above, this means that each variable within the
+matrix variable can have its own custom information. For example:
+
+.. code-block:: python
+
+    x = scip.addMatrixVar(shape, vtype='C', name='x', ub=np.array([[5, 6], [2, 8]]))
+
+Matrix Constraints
+===================
+
+Matrix constraints follow the same logic as matrix variables. They can be constructed quickly
+and added all at once. The standard variable operators, like ``sin``, ``exp``, ``sqrt``, etc.,
+are applied element-wise. Some examples are provided below (these examples are nonsensical,
+and there to purely understand the API):
+
+.. code-block:: python
+
+    x = scip.addMatrixVar(shape=(2, 2), vtype="B", name="x")
+    y = scip.addMatrixVar(shape=(2, 2), vtype="C", name="y", ub=5)
+    z = scip.addVar(vtype="C", name="z", ub=7)
+
+    scip.addMatrixCons(x + y <= z)
+    scip.addMatrixCons(exp(x) + sin(sqrt(y)) == z + y)
+    scip.addMatrixCons(y <= x @ y)
+    scip.addMatrixCons(x @ y <= x)
+    scip.addCons(x.sum() <= 2) # Matrix variables can also appear in standard constraints, if the result expression is type Expr
+
+.. note::
+
+    When creating constraints, one can mix standard variables and values in the same
+    expressions. ``numpy`` will then handle this, and broadcast the correct operations.
+    In general this can be viewed as creating an imaginary ``np.array`` of the appropriate
+    shape and populating it with the variable / value.
+
+Class Properties
+=================
+
+A ``MatrixVariable`` and ``MatrixConstraint`` object have all the same getter
+functions that are in general available for the standard equivalent. An example
+is provided below for ``vtype``.
+
+.. code-block:: python
+
+    x = scip.addVar()
+    matrix_x = scip.addMatrixVar(shape=(2,2))
+
+    x.vtype()
+    matrix_x.vtype()
+
+The objects are not interchangeable however, when being passed into functions
+derived from the ``Model`` class. That is, there is currently no global support,
+that the following code runs:
+
+.. code-block:: python
+
+    scip.imaginary_function(x) # will always work
+    scip.imaginary_function(matrix_x) # may have to access each variable manually
+
+Accessing Variables and Constraints
+===================================
+
+After creating the matrix variables and matrix constraints,
+one can always access the individual variables or constraints via their index.
+
+.. code-block:: python
+
+    x = scip.addMatrixVar(shape=(2, 2))
+    assert(isinstance(x, MatrixVariable))
+    assert(isinstance(x[0][0], Variable))
+    cons = x <= 2
+    assert(isinstance(cons, MatrixExprCons))
+    assert(isinstance(cons[0][0]), ExprCons)
+
+Accessing Solution Values
+===========================
+
+After optimizing a model, the solution values of a matrix variable can be
+accessed in an identical manner to the standard variant. There are two
+recommended ways to do this.
+
+.. code-block:: python
+
+    matrix_var_vals = scip.getVal(x)
+
+.. code-block:: python
+
+    sol = scip.getBestSol()
+    matrix_var_vals = sol[x] # returns a numpy array of values

docs/tutorials/scipdex.rst

@@ -0,0 +1,7 @@
+###################
+Hands On Tutorials
+###################
+
+For more tutorials with PySCIPOpt, especially interactive ones, please see
+`SCIPDex <https://github.com/mmghannam/scipdex>`_. SCIPDex is a collection of
+interactive exercises to get you started (and more) with PySCIPOpt

pyproject.toml

@@ -8,7 +8,7 @@ description = "Python interface and modeling environment for SCIP"
 authors = [
     {name = "Zuse Institute Berlin", email = "[email protected]"},
 ]
-dependencies = []
+dependencies = ['numpy >=1.16.0']
 requires-python = ">=3.8"
 readme = "README.md"
 license = {text = "MIT"}

setup.cfg

@@ -1,2 +1 @@
-[bdist_wheel]
-universal = 1
+

setup.py

@@ -1,5 +1,5 @@
 from setuptools import find_packages, setup, Extension
-import os, platform, sys, re
+import os, platform, sys
 
 # look for environment variable that specifies path to SCIP
 scipoptdir = os.environ.get("SCIPOPTDIR", "").strip('"')
@@ -19,7 +19,6 @@
     print("Assuming that SCIP is installed globally, because SCIPOPTDIR is undefined.\n")
 
 else:
-
     # check whether SCIP is installed in the given directory
     if os.path.exists(os.path.join(scipoptdir, "include")):
         includedir = os.path.abspath(os.path.join(scipoptdir, "include"))
@@ -109,7 +108,7 @@
 
 setup(
     name="PySCIPOpt",
-    version="5.3.0",
+    version="5.4.0",
     description="Python interface and modeling environment for SCIP",
     long_description=long_description,
     long_description_content_type="text/markdown",

src/pyscipopt/_version.py

@@ -1 +1 @@
-__version__ = '5.3.0'
+__version__ = '5.4.0'