[META] Implement microphone, webcam, and other bidirectional capabilities

[ehfd]

Basically, this is the opposite direction to the audio capabilities available now.

1. Fetch microphone permissions from the web browser.
2. Receive the microphone stream and encode to opus within the browser.
3. Send the audio stream using RTP wrapped with WebRTC and the host/container Python component receives the stream.
4. The host/container Python component first decodes the opus codec, and then creates a microphone device and sends audio to the host/container with pulsesink.

The gst-web components should be overhauled to accomodate new improvements such as webcams, printing, and file sharing in the future as well as to finish this job.

Would be convenient to investigate the following issues along the way for better usability if we are indeed overhauling gst-web for this task (Unfortunately, I don't know JavaScript too well compared to Python): #9 (AZERTY keyboard) #22 (Touch keyboard capabilities), #25 (URL parameters for authentication).

Additional issues (intertwined with gst-web and Python): #39 (Multiple users), #48 (WebTransport), #74 (WebRTC chunked DataChannel)

[ehfd]

This issue is from a request by @GeorgeDeac .

[ehfd]

Implemented support for Stereo on Opus. 1845bda

[ehfd]

Good references:

https://github.com/evshiron/parsec-web-client
https://parsec.app/blog/game-streaming-tech-in-the-browser-with-parsec-5b70d0f359bc
https://github.com/parsec-cloud/parsec-sdk

[satyaakam]

Hi , i see this issue labelled under Funding , can anyone from core developers let me know what amount of funding are you expecting in order to see this issue resolved. Need few details like amount , timelines , and process of resolving it in order to get the ball rolling .

[ehfd]

Hi! @satyaakam

Unfortunately, the core developers are currently all busy, and no cost may justify the time we are to spend on this enhancement at this time.

However, @PMohanJ, another Indian developer and junior member of the Selkies team, is hired by his corporation to develop the webcam capabilities. I suggest waiting for them to open-source this capability.

[satyaakam]

Hi @ehfd thanks for the response , I am sorry if i sounded like someone who would like to throw money or people at a problem to get it resolved :-) , I understand the way FOSS communities work , my request was more from the point of view of collaboration, thanks for pointing to @PMohanJ will be more than happy to collaborate and work towards getting it resolved .

[ehfd]

I greatly appreciate your interest in this project. @satyaakam
I would have taken the job if I had extra time.
This project competes technically with well-funded proprietary remote desktop products and conversely relies on the free time of experienced developers in very specific, but wide-spectrum technology stacks. So donations never hurt.
But currently, we really need people willing to learn the code and make their own contributions.
Bounties might work out.

[debaa98]

is there any bounty going to be paid to contributer?

[ehfd]

@debaa98 It is possible, if you are indeed interested.

[debaa98]

Yaa I am intrested

[ehfd]

But I am not the entity which issues the bounty. Check with @satyaakam.

[debaa98]

yas,i had a contact with @satyaakam