Concerns Regarding the Fairness of SRe2L Comparison

[Jiacheng8]

Thank you for your excellent work! I truly appreciate the effort put into this study. However, I have some concerns regarding the fairness of the comparison for SRe2L.

As highlighted in the recent work "Dataset Distillation via Committee Voting" (https://arxiv.org/abs/2501.07575), the hyper-parameters and the recovery process for small datasets like CIFAR-10 and CIFAR-100 were found to be highly suboptimal, leading to a significant performance gap (whether using hard labels or soft labels). Given this, I believe it is crucial to re-evaluate the leaderboard using the updated version of SRe2L++ to ensure a fair comparison.

Additionally, the current dataset benchmark appears to be relatively simple. To better assess the performance of different methods, it would be beneficial to include large-scale datasets such as ImageNet-1k and its subsets. This would provide a more comprehensive evaluation of each approach.

I appreciate your time and consideration, and I look forward to your thoughts on this matter.