Things that are different compared to the article

[ivankunyankin]

There are a number of differences compared to the source:

• original article: https://arxiv.org/pdf/1910.10261.pdf

Differences:

1. The default training script uses LibriTTS dataset instead of LibriSpeech.
2. The model implementation uses time-channel separable 1D convolutional modules without groups and shuffling. For details refer Fig. 2 from the article.
3. Instead of NovoGrad optimizer I use PyTorch's Adam optimizer with weight decay.
4. Instead of Cutout for augmentation I use a custom function based on PyTorch's masking functions. You can find more details about augmentation here.
5. Authors of the article utilise cosine annealing learning rate policy with learning rate warmup. I use PyTorch's implementation of one cycle learning rate policy. This policy is well described in this article.

[raotnameh]

What is the WER are you able to achieve? Also, can you share the epoch vs WER and CER and LOSS? @ivankunyankin