ibert seems to be quite slow in quant_mode = True

[huu4ontocord]

Hi, for iBert -

I found it is MUCH slower in quant_mode = True. here's a notebook with a slightly modified version of the HF code to allow dynamically switching quant_mode. You can see the timing difference.

https://colab.research.google.com/drive/1DkYFGc18oPvAn5nyGEL1aIFHmD_aNlXW

@patrickvonplaten

kssteven418/I-BERT#21

[patrickvonplaten]

Hey @ontocord,

I don't have much experience with iBERT. @kssteven418 any ideas here? Is such a slowdown expected?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.