Number of solved LPs increased in the Demo

[liujl11git]

Thanks for the great repo and the clean demos!

I met a confusing problem when I study the set-covering demo in https://github.com/ds4dm/ecole/blob/master/examples/branching-imitation/example.ipynb

When I set DATA_MAX_SAMPLES = 5000, the learned branching rule can beat the SCIP default rule clearly in the sense of time. However, I observed that, with the learned rule, SCIP needs to solve more LPs while the time consumption is lower.

I used “env.model.as_pyscipopt().getNLPs()” to observe the number of solved LPs and used "env.model.as_pyscipopt().getNLPIterations()" to observe the number of LP iterations. I kept other codes exactly the same as the original demo.

I attach my results below. Did I make any mistakes and did you guys observe the same phenomena?

Instance 0 | SCIP nb nodes 3 | SCIP time 2.96 | SCIP nLPs 68 | SCIP nLPIters 1650
| GNN nb nodes 19 | GNN time 2.45 | GNN nLPs 69 | GNN nLPIters 2133
| Gain -533.33% | Gain 17.04% | Gain -1.47% | Gain -29.27%
Instance 1 | SCIP nb nodes 11 | SCIP time 7.06 | SCIP nLPs 129 | SCIP nLPIters 3434
| GNN nb nodes 111 | GNN time 4.48 | GNN nLPs 255 | GNN nLPIters 8420
| Gain -909.09% | Gain 36.59% | Gain -97.67% | Gain -145.20%
Instance 2 | SCIP nb nodes 3 | SCIP time 2.59 | SCIP nLPs 55 | SCIP nLPIters 1741
| GNN nb nodes 27 | GNN time 1.77 | GNN nLPs 75 | GNN nLPIters 2822
| Gain -800.00% | Gain 31.40% | Gain -36.36% | Gain -62.09%
Instance 3 | SCIP nb nodes 13 | SCIP time 9.81 | SCIP nLPs 130 | SCIP nLPIters 4962
| GNN nb nodes 96 | GNN time 7.86 | GNN nLPs 302 | GNN nLPIters 12757
| Gain -638.46% | Gain 19.91% | Gain -132.31% | Gain -157.09%
Instance 4 | SCIP nb nodes 7 | SCIP time 4.65 | SCIP nLPs 94 | SCIP nLPIters 2625
| GNN nb nodes 107 | GNN time 4.37 | GNN nLPs 309 | GNN nLPIters 7010
| Gain -1428.57% | Gain 5.87% | Gain -228.72% | Gain -167.05%
Instance 5 | SCIP nb nodes 7 | SCIP time 4.95 | SCIP nLPs 75 | SCIP nLPIters 2333
| GNN nb nodes 35 | GNN time 3.01 | GNN nLPs 83 | GNN nLPIters 2839
| Gain -400.00% | Gain 39.25% | Gain -10.67% | Gain -21.69%
Instance 6 | SCIP nb nodes 3 | SCIP time 6.03 | SCIP nLPs 88 | SCIP nLPIters 2405
| GNN nb nodes 49 | GNN time 5.09 | GNN nLPs 145 | GNN nLPIters 5040
| Gain -1533.33% | Gain 15.60% | Gain -64.77% | Gain -109.56%
Instance 7 | SCIP nb nodes 1 | SCIP time 2.35 | SCIP nLPs 44 | SCIP nLPIters 1228
| GNN nb nodes 5 | GNN time 2.27 | GNN nLPs 43 | GNN nLPIters 1073
| Gain -400.00% | Gain 3.44% | Gain 2.27% | Gain 12.62%
Instance 8 | SCIP nb nodes 1 | SCIP time 0.76 | SCIP nLPs 4 | SCIP nLPIters 415
| GNN nb nodes 1 | GNN time 0.77 | GNN nLPs 4 | GNN nLPIters 328
| Gain 0.00% | Gain -2.54% | Gain 0.00% | Gain 20.96%
Instance 9 | SCIP nb nodes 1 | SCIP time 3.01 | SCIP nLPs 39 | SCIP nLPIters 852
| GNN nb nodes 1 | GNN time 3.17 | GNN nLPs 39 | GNN nLPIters 902
| Gain 0.00% | Gain -5.36% | Gain 0.00% | Gain -5.87%
Instance 10 | SCIP nb nodes 11 | SCIP time 6.83 | SCIP nLPs 97 | SCIP nLPIters 3603
| GNN nb nodes 71 | GNN time 5.05 | GNN nLPs 164 | GNN nLPIters 5826
| Gain -545.45% | Gain 26.06% | Gain -69.07% | Gain -61.70%
Instance 11 | SCIP nb nodes 17 | SCIP time 6.91 | SCIP nLPs 124 | SCIP nLPIters 4449
| GNN nb nodes 73 | GNN time 5.26 | GNN nLPs 264 | GNN nLPIters 8936
| Gain -329.41% | Gain 23.99% | Gain -112.90% | Gain -100.85%
Instance 12 | SCIP nb nodes 1 | SCIP time 0.70 | SCIP nLPs 8 | SCIP nLPIters 418
| GNN nb nodes 1 | GNN time 1.41 | GNN nLPs 19 | GNN nLPIters 606
| Gain 0.00% | Gain -103.38% | Gain -137.50% | Gain -44.98%
Instance 13 | SCIP nb nodes 3 | SCIP time 3.36 | SCIP nLPs 45 | SCIP nLPIters 1763
| GNN nb nodes 17 | GNN time 3.16 | GNN nLPs 68 | GNN nLPIters 2683
| Gain -466.67% | Gain 5.89% | Gain -51.11% | Gain -52.18%
Instance 14 | SCIP nb nodes 3 | SCIP time 4.71 | SCIP nLPs 66 | SCIP nLPIters 2463
| GNN nb nodes 33 | GNN time 4.12 | GNN nLPs 125 | GNN nLPIters 3796
| Gain -1000.00% | Gain 12.60% | Gain -89.39% | Gain -54.12%
Instance 15 | SCIP nb nodes 3 | SCIP time 4.42 | SCIP nLPs 62 | SCIP nLPIters 2082
| GNN nb nodes 27 | GNN time 3.45 | GNN nLPs 81 | GNN nLPIters 2650
| Gain -800.00% | Gain 21.99% | Gain -30.65% | Gain -27.28%
Instance 16 | SCIP nb nodes 27 | SCIP time 5.67 | SCIP nLPs 121 | SCIP nLPIters 3550
| GNN nb nodes 109 | GNN time 4.42 | GNN nLPs 189 | GNN nLPIters 6128
| Gain -303.70% | Gain 22.11% | Gain -56.20% | Gain -72.62%
Instance 17 | SCIP nb nodes 3 | SCIP time 3.73 | SCIP nLPs 71 | SCIP nLPIters 1575
| GNN nb nodes 29 | GNN time 3.16 | GNN nLPs 93 | GNN nLPIters 2104
| Gain -866.67% | Gain 15.34% | Gain -30.99% | Gain -33.59%
Instance 18 | SCIP nb nodes 1 | SCIP time 4.25 | SCIP nLPs 59 | SCIP nLPIters 1789
| GNN nb nodes 15 | GNN time 4.58 | GNN nLPs 80 | GNN nLPIters 2393
| Gain -1400.00% | Gain -7.60% | Gain -35.59% | Gain -33.76%
Instance 19 | SCIP nb nodes 7 | SCIP time 3.80 | SCIP nLPs 64 | SCIP nLPIters 1901
| GNN nb nodes 49 | GNN time 3.27 | GNN nLPs 133 | GNN nLPIters 4149
| Gain -600.00% | Gain 13.91% | Gain -107.81% | Gain -118.25%