This repository contains quadratic programs (QPs) arising from model predictive control in robotics, in a format suitable for qpbenchmark. Here is the report produced by this benchmarking tool:
The recommended process is to install the benchmark and all solvers in an isolated environment using conda:
conda env create -f environment.yaml
conda activate qpbenchmarkIt is also possible to install the benchmark from PyPI.
Run the test set as follows:
python ./mpc_qpbenchmark.py run
The outcome is a standardized report comparing all available solvers against the different benchmark metrics. You can check out and post your own results in the Results forum.
The problems in this test set have been contributed by:
| Problems | Contributor | Details |
|---|---|---|
QUADCMPC* |
@paLeziart | Proposed in #1, details in this thesis |
LIPMWALK* |
@stephane-caron | Proposed in #3, details in this paper |
WHLIPBAL* |
@stephane-caron | Proposed in #4, details in this paper |
Here are some known areas of improvement for this benchmark:
- Cold start only: we don't evaluate warm-start performance for now.
- CPU thermal throttling: the benchmark currently does not check the status of CPU thermal throttling.
- Adding this feature is a good way to start contributing to the benchmark.
- HPIPM evaluation: currently we evaluate HPIPM against a duality-gap tolerance but the solver is asked to solve a complementarity-slackness condition.
Note that this test set was spun off to benefit from the availability of qpbenchmark and readily-available MPC QPs, but it does not fully reflect the use of QP solvers for MPC in production due, notably, to the cold-start-only limitation.
If you use qpbenchmark in your works, please cite all its contributors as follows:
@software{qpbenchmark2024,
title = {{qpbenchmark: Benchmark for quadratic programming solvers available in Python}},
author = {Caron, Stéphane and Zaki, Akram and Otta, Pavel and Arnström, Daniel and Carpentier, Justin and Yang, Fengyu and Leziart, Pierre-Alexandre},
url = {https://github.com/qpsolvers/qpbenchmark},
license = {Apache-2.0},
version = {2.3.0},
year = {2024}
}If you contribute to this repository, don't forget to add yourself to the BibTeX above and to CITATION.cff.
Related test sets that may be relevant to your use cases:
- Free-for-all: community-built test set, new problems welcome!
- Maros-Meszaros test set: a standard test set with problems designed to be difficult.