Implementing voice chat

[Bios-Marcel]

A big part of what makes Discord so great, is that it offers all kinds of features for all amounts of people. You can share your screen, videochat, do groupchats and whatever not. Cordless however is currently text only, meaning you can't do voice chatting, video chatting and all of the other fancy audio / graphical features.

While some of those, for example screenshare and videochatting are pretty much impossible to implement, voice chat should be possible.

The UI in cordless should be relatively similar to the one in Discord, at least for rendering voice channels in servers. Meaning all users participating should be listed in the channel tree. Whether someone is muted or not could be signaled by emojis in front of the users name in the channel tree.

As for private calls, those should not be similar to the original client, as it wastes a lot fo space for additional UI elements and displaying the profile pictures, which we won't / can't do anyway.

Instead it should just be a single component above the chat view, that is possible permanently visible as long as you are in a voicecall. The design isn't very important for now, as this is somewhat of an implementation detail to me. The most important thing is to know that other people might here you as you speak and being able to see who is muted and such.

In order to implement this, Portaudio and gopus could potentionally be used, but there might be other options, which on the other hand might also require more work to be put into this.

The goal is to have audio working on MacOS, Windows and Linux. The more platforms the better.

There are also branches where I played around with this a bit, one of them worked really well though, so it's probably not of use.

[p0da]

Any updates on this front? Its the only thing holding me back from uninstalling the full client.

[Bios-Marcel]

I am not knowledgable enough to fix the issues with this. It just randomly craps outer after some seconds. There'a a branch you can select a voicechannel in a server and it'll join, but it really doesn't work well. I honestly don't know how to debug this.

[Doonut]

i checked out your v2 branch, and tested the audio. For me it was very crackily, but i changed the sampling rate from 48K to 96K to match my DAC, and at least with one other person in the channel, it fixed all my crackling. I didnt even need to look for and change the bitrate from 16 to 24, but im not sure if thats even specified. It looks like the lib your using is really sensitive to the audio timing, probablly because its using ALSA and not going through pulseaudio. Maybe this is a timer vs interrupt problem? something similar to this

[Bios-Marcel]

I never actually tested Windows for example. Maybe it'd be good to see if it's the same.

[Doonut]

im on arch not win

[Doonut]

to elaborate, i changed your hardcoded 48K sampling rate in your code, not in my driver. Pulseaudio was already set to output to 96K per my config, but I already know ALSA does not have very good support for switching sample rates, and usually down/up samples everything to 48K. I would maybe try switching to a pulseaudio solution on linux

[Bios-Marcel]

I know, but i generally thought it be good to see if it shows similar symptoms on win

[r3k2]

@Doonut so this worked after your change on arch GNU/Linux for you? I'm on Linux as well don't care if it works on windows or not, as long it works on Linux I will compile it and use it, I hate electron apps.

[r3k2]

hi, any news on this? at this rate matrix will do everything we need :)

[w3bb]

Checked out the VC branch, but it just crashes when I join a voice channel. Was anyone able to get it to work? I want to hack on this a bit.

[r3k2]

@w3bb on linux?

[w3bb]

@w3bb on linux?

Yes.

[DeltoidDelta]

Hi there, I'm on GNU/Linux, compiled the branch myself and it seems to be working somewhat. There are issues when multiple people join the server, and voice stream is a bit choppy, but it seems to be somewhat usable as it is.

What's the plan moving forward?

[r3k2]

yeah apparently the main dev is not wanting to work on this only because is not working on windows.. and seems bias towards linux... and this is totally BS because honestly people who will want to use this are mostly on linux, and even if not, better to have it working for 1 OS than none.... 1 > none...

[Bios-Marcel]

Tbh, i just don't feel like working on this either, I am only human too, you know. This isn't my job, it's a thing I do in my freetime.

[w3bb]

yeah apparently the main dev is not wanting to work on this only because is not working on windows.. and seems bias towards linux... and this is totally BS because honestly people who will want to use this are mostly on linux, and even if not, better to have it working for 1 OS than none.... 1 > none...

How is wanting a stable cross platform way of doing voice chat being biased? It's better to have a good, full solution instead of rushing out a half baked one when it isn't ready. Plus you can always use the branch or patch it yourself in the meantime.

Tbh, i just don't feel like working on this either, I am only human too, you know. This isn't my job, it's a thing I do in my freetime.

Don't feel the pressure man, the work you've done already is above and beyond.

[ChillerDragon]

Wow what a ambitious issue o.O

@Bios-Marcel no pressure its a cool idea and I really appreciate your effort!!!

Btw the link should probably be updated in the issue description since it looks like https://github.com/Bios-Marcel/cordless/tree/voice-support moved to https://github.com/Bios-Marcel/cordless/tree/voice-attempt-2