Website: solver history analysis

[siddharth-krishna]

One thing to consider, how do we represent the solving speed history e.g. when some problems couldn't get solved.

• I want an easy metric from which I can understand the performance over time ideally for many cases
• I want an easy overview what problems can/ cannot be solved with the solvers (so I can see e.g. that HiGHS can solver more and more problems over time)

Not easy to come up with some useful graphs/ tables/ illustrations.
From: @pz-max

Once #13 is done, we should think about how to analyse and present the performance history of each solver. Some initial ideas:

• Plot shifted geometric mean (SGM) of runtime over time
• Ensure that we have results for the same benchmarks (& sizes) for each solver-year, otherwise SGM is meaningless. If the results have data for different benchmarks for different years, filter to the intersection of benchmarks that are solved by all solvers & years.
• Plot number of solved problems over time
• Allow filtering the above plots by all categories (e.g. benchmark size, modelling framework, LP/MILP, infrastructure/operational, etc)
• Highs is adding a primal-dual integral (see slide 40), which could serve as a good relative measure of performance across different versions of HiGHS