some unreasonable results when reproduce

[LeighSu]

I am trying to add new self-defined loss function, but when I want to compare the result with the existed loss function, the reproduced result seems unreasonable.
I used the following two command to reproduce the result, and the teacher model seems normally, it can archive high accurate.

but the student model only 8% acc for Top1 and 22% acc for Top 5.

Is there anyting I misunderstand or how can I improve?

ilsvrc2012_cckd_resnet18_from_resnet50.txt
ilsvrc2012_kd_resnet18_from_resnet152.txt

python3 -m torch.distributed.launch --nproc_per_node=2 --use_env examples/image_classification.py --world_size 2 --config configs/sample/ilsvrc2012/single_stage/kd/resnet18_from_resnet152.yaml --log log/ilsvrc2012/kd/resnet18_from_resnet152.txt

python3 -m torch.distributed.launch --nproc_per_node=2 --use_env examples/image_classification.py --world_size 2 --config configs/sample/ilsvrc2012/single_stage/cckd/resnet18_from_resnet50.yaml --log log/ilsvrc2012/cckd/resnet18_from_resnet50.txt

Addition message:
I checkout the branch t0 tag v0.3.3 .

[yoshitomo-matsubara]

Hi @LeighSu

What results are you trying to reproduce?

I believe you're using sample configs for ImageNet datasets, which are not tuned at all or whose results are not reported in some papers either.

Please read configs/legacy/README.md

Most of the configurations in sample/ should be used for debugging or as starter configs to design your experiments.
If you use the methods as baselines, you SHOULD tune the hyperparameters in the configuration for a fair comparison.