(Question): using a custom segmentation model

[IzzyHibbert]

Hi @kadirnar

I wanted Whisperplus to use a specific segmentation model of pyannote. Basically I fine-tuned one.
I understood that to get that, I should modify the ASRDiarizationPipeline: moving from the current pipeline of pyannote to a custom one,

Here is how the pipeline object of pyannote is initialized.

https://github.com/pyannote/pyannote-audio/blob/286ea1a4e34e2dd7d7926f590e402dac1e17494b/pyannote/audio/pipelines/speaker_diarization.py#L115C1-L126C7

So I went to whisper_diarize.py and adapt it use the new pipeline containing a segmentation which is my custom one.
It looks like there are some conflicts that originate :

base.py", line 916, in __init__
    self._preprocess_params, self._forward_params, self._postprocess_params = self._sanitize_parameters(**kwargs)
                                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: AutomaticSpeechRecognitionPipeline._sanitize_parameters() got an unexpected keyword argument 'segmentation'

How I updated whisper_diarize.py is here
ANy chance that you guide me to the solution ?
I also think that my request can be higly beneficial if we can offer a chance to use/switch to a specific custom segmentation model for improving the performances in the diarization.

[TinaTe]

Hey there, did you eventually succeed with this?
I'm not quite at this stage of my project yet, but did you follow the instructions from pyannote? (using your own model / running model from local)?
https://github.com/pyannote/pyannote-audio/blob/develop/tutorials/add_your_own_model.ipynb
https://github.com/pyannote/pyannote-audio/blob/develop/tutorials/applying_a_model.ipynb