CNN: Normalization before/after L-inf is applied?

[ChristopherBrix]

I did not participate in the competition, but would like to use the CNN challenges to evaluate my own toolkit (Debona, based on Neurify). I've noticed that @FlorianJaeckle stated that the perturbation should be applied after the normalization (divide by 255, subtract mean, divide by variance). However, I can only reproduce the CIFAR10 results if I apply it before the normalization (after dividing by 255).
nnenum (@stanleybak) seems to do the same, as the networks are modified to include the normalization, and the perturbation is applied to the input

So is it safe to assume that the normalization should always be done last, for all 4 CNNs?

[stanleybak]

There were two benchmarks proposed for CNN networks, one from @GgnDpSngh called ETH in nnenum which you linked to (called ggn in the report), and the other was proposed by @FlorianJaeckle that we call OVAL in nnenum. The normalization is done differently for these two. See here for the oval file we were working on with nnenum, although I'm not even sure if that's correct since nnenum didn't have results for the oval benchmarks in time for the report. The OVAL benchmarks are pretty hard, and use a long timeout.

For making sure the normalization is done correctly, the strategy I used was to make sure the images classified as expected. For OVAL, all images should classify correctly if the normalization is performed correctly. Hope that helps clear things up.

[ChristopherBrix]

Great, thanks a lot! This makes sense now.