batch size and iteration

[pursueorigin]

Hi authors,
This is an impressing work. However, I am confused about your classification experiments for small batch size.

'setting batch size to 256 and iteration size to 1 (for example) is equivalent to setting batch size to 1 and iteration size to 256.'

Why are they equivalent? For the first case, it only updates parameters 1 times while the second case updates parameters 256 times? Am I right?

Thanks.

[joe-siyuan-qiao]

Thanks for your interest in WS. Here, the iteration size is basically "iter_size" in caffe, which specifies the number of iterations to take before one parameter update. You can check the implementation of "iter_size" in caffe for more details and why they are equivalent.

[pursueorigin]

Got it. Thanks.