ENH - Add modular Group Prox Newton solver

[Badr-MOUFAD]

This implements a group proximal Newton solver to extend skglm to handle problems with non-quadratic datafit and group separable penalties.

It proceeds as follows:

• implement GroupProxNewton
• unitest the solver for Group Logistic regression
• perform benchmarks

---

link to benchmarks results
link to benchmark repo

[Badr-MOUFAD]

For the benchmarks,
I am not aware of any solver that solves Group Logistic regression problem
(yngvem do that but uses cross entropy as a loss).
Share with me If you have one in mind. I would like to add it to the benchmark.

[mathurinm]

@Badr-MOUFAD the cross entropy for 2 classes should be the logistic loss

[Badr-MOUFAD]

Badr-MOUFAD the cross entropy for 2 classes should be the logistic loss

Pls have a look at this potion the code. Also in the benchmark, we don't converge to the same objective.

[mathurinm]

https://en.wikipedia.org/wiki/Cross_entropy#Cross-entropy_loss_function_and_logistic_regression

no ?

[Badr-MOUFAD]

https://en.wikipedia.org/wiki/Cross_entropy#Cross-entropy_loss_function_and_logistic_regression

no ?

This is what one would expect. Also, after checking, yngvem LogisticGroupLasso with groups of size one does not give similar results to scikit-learn's LogisiticRegression.

I raised an issue yngvem/group-lasso#36.

[QB3]

First round of review

[QB3]

Thanks a lot for the outstanding work @Badr-MOUFAD !
One last question, GroupProxNewton has not been added to all the tests in the test_group.py file, is there a specific reason?

[Badr-MOUFAD]

One last question, GroupProxNewton has not been added to all the tests in the test_group.py file, is there a specific reason?

It's mainly to avoid redundancy and complications of unittest:

1. for alpha_max test: group log reg and group lasso do not have the same alpha_max
2. I didn't want to add tests for quadratic problems as prox newton is meant for non-quadratic ones, and we test it for log reg