Sortformer Diarizer 4spk v1 model PR Part 1: models, modules and dataloaders

[tango4j]

What does this PR do ?

Sortformer Diarizer Model, 4 speaker limit, v1

Sortformer Paper Link

In this PR, we are adding: model files, module files and corresponding dataloader and evalutations.

Collection: ASR/speaker_tasks

Changelog

• model files
  nemo/collections/asr/models/sortformer_diar_models.py

• module files
  nemo/collections/asr/modules/sortformer_modules.py

• evaluation files
  nemo/collections/asr/metrics/der.py
  nemo/collections/asr/metrics/multi_binary_acc.py

• dataloader files
  NeMo/nemo/collections/asr/data/audio_to_diar_label.py
  NeMo/nemo/collections/asr/data/audio_to_diar_label_lhotse.py

• training yaml
  examples/speaker_tasks/diarization/conf/neural_diarizer/sortformer_diarizer_hybrid_loss_4spk-v1.yaml

• post-processing yaml files
  NeMo/examples/speaker_tasks/diarization/conf/post_processing/sortformer_diar_4spk-v1_callhome-part1.yaml
  NeMo/examples/speaker_tasks/diarization/conf/post_processing/sortformer_diar_4spk-v1_dihard-dev.yaml
  NeMo/nemo/collections/asr/data/audio_to_diar_label.py
  NeMo/nemo/collections/asr/data/audio_to_diar_label_lhotse.py

• util files
  NeMo/nemo/collections/asr/parts/utils/speaker_utils.py
  NeMo/nemo/collections/asr/parts/utils/vad_utils.py

• Changed the file names of these yaml files

examples/speaker_tasks/diarization/neural_diarizer/sortformer_diar_train.py
nemo/collections/asr/data/audio_to_diar_label.py

nemo/collections/asr/models/init.py

nemo/collections/asr/modules/sortformer_modules.py
nemo/collections/asr/parts/utils/asr_multispeaker_utils.py
nemo/collections/asr/parts/utils/speaker_utils.py
nemo/collections/asr/parts/utils/vad_utils.py
nemo/collections/common/parts/preprocessing/collections.py

Usage

• You can potentially add a usage example below

python ${NEMO_ROOT}/examples/speaker_tasks/diarization/neural_diarizer/e2e_diarize_speech.py \
     model_path=/path/to/diar_sortformer_4spk-v1.nemo \
     dataset_manifest=/path/to/eval_dataset.json

GitHub Actions CI

CI tests will be added in the second PR.
Third PR will include documents and tutorials.

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you add or update any necessary documentation?
• Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
• Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

• New Feature
• Bugfix
• Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the ASR and speaker_tasks

[github-advanced-security]

CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

[nithinraok]

LGTM, Great work!

[github-actions]

beep boop 🤖: 🙏 The following files have warnings. In case you are familiar with these, please try helping us to improve the code base.

---

Your code was analyzed with PyLint. The following annotations have been identified:

------------------------------------
Your code has been rated at 10.00/10

---

Thank you for improving NeMo's documentation!

[github-actions]

beep boop 🤖: 🚨 The following files must be fixed before merge!

---

Your code was analyzed with PyLint. The following annotations have been identified:

------------------------------------
Your code has been rated at 10.00/10

---

Thank you for improving NeMo's documentation!