Hyperparameters 📚

[ivankunyankin]

A short description of hyperparameters from config.yml

Training parameters

1. max_length - maximum file length in seconds. Longer files are omitted. Used to avoid OOM (out-of-memory) error. Also reduces the amount of data samples used for training and validation.
2. weight_decay - regularization parameter. Can be increased when using constant learning rate
3. use_onecyclelr - use PyTorch's implementation of cyclic learning rate policy
4. spec_params - a set of parameters for generating mel spectrograms. Described here. When changing these one should consider changing parameters of masking.
5. normalize - apply normalization. By default applies normalization from stats saved in assests/stats.npy to each mel channel separately. Saved stats were calculated from LibriTTS. If training on different data, these should be recalculated. More details here.

Augmentation parameters

1. speed_perturbation - randomly chooses the rate of perturbation between 1 +- specified value.
2. chunk_size - size of each spectrogram segment (time axis) to apply augmentation to. Set equal to -1 to apply original pytorch's masking. Examples shown below
3. freq_masking, time_masking - the maximum width of masking band. More details here

Augmentation examples

Original pytorch functions applied. Less aggresive augmentation

Custom function applied. More aggresive augmentation