The steps of QAT?

[twmht]

From Nvidia quantization toolkit, the first step is to calibration to get the quantization scale, and the second step is to finetune based on the scale.

And from their experience, they said.

Do not change quantization representation (scale) during training, at least not too frequently. Changing scale every step, it is effectively like changing data format (e8m7, e5m10, e3m4, et.al) every step, which will easily affect convergence.

But from the examples of QAT, you change the scale every train iteration.

is this a better way in your experience?