CutMix data augmentation

[navreeetkaur]

CutMix is an image data augmentation technique increasingly used nowadays in training pipelines to improve performance. It is one of the best performing augmentation methods on CIFAR.

A patch in the image is removed and padded by a patch from another image in the dataset. The ground truth labels are also mixed proportionally to the number of pixels of combined images.

I use cutmix along with transforms of random crop, horizontal flip and normalisation as used by the original code, and train four models with cutmix probability of 0.5 and beta 1.0. The test results are as follows -

Model	Acc. w/o Cutmix	Acc. with Cutmix
VGG16	92.64%	94.77%
ResNet18	93.02%	95.84%
GoogleLeNet	93.64%	95.71%
SimpleDLA	94.89%	95.35%

These models were trained using MultiStepLR with milestones of 50 and 100 epochs, and gamma of 0.1

I noticed that MultiStepLR helps achieve the same accuracy earlier in the training procedure as compared to CosineAnealingLR. With max epochs of 200, learning rate at 50-60 epochs is around 0.08 while using CosineAnealingLR. This seems a bit high as accuracy and loss fluctuates and training progresses slowly. MultiStepLR helps use lr of 0.01 after 50 epochs, and 0.001 after 100 epochs, which helps reach higher accuracy faster. Hence, I have also added a parser argument to choose the scheduler from either CosineAnealingLR or MultiStepLR.

[navreeetkaur]

#116 #120 #130