[META] Rust Components: WebRTCSink, WHIP/WHEP integration

[tt2468]

When browsing this project as a potential option to replace apache guacamole in my setup, I noticed a lot of the set up and complexity comes from configuring the various WebRTC components to work effectively together. I've recently worked on some implementations of the WebRTC WHIP standard, and I think it could be an effective option to simplify things. There's options for paid hosted services, like Cloudflare Stream, and self-hosted server instances like Broadcast Box

WHIP standard: https://www.ietf.org/archive/id/draft-ietf-wish-whip-01.html
WHEP standard: https://www.ietf.org/archive/id/draft-murillo-whep-02.html

Gstreamer recently implemented WHIP support via their rust plugins, which may possibly be used to reduce implementation difficulty. Otherwise, the code from an OBS Studio PR I've worked on may be a useful reference: obsproject/obs-studio#7926

[ehfd]

#7 (comment)

Relevant.
Would this replace the need for TURN servers?

[ehfd]

https://www.100ms.live/blog/webrtc-turn-server

Nope. Does not eliminate TURN.

[tt2468]

Does not eliminate TURN, but in most usage cases, WHIP is not used in a peer-to-peer style. For the example of Cloudflare Stream, it routes all traffic through their network before exiting as WHEP.

[ehfd]

From @boxerab

Has anyone looked into using the Rust element webrtcsink ?

https://mathieuduponchelle.github.io/2021-12-14-webrtcsink.html

One nice feature is that is handles congestion control.

[ehfd]

To enable gstreamer-rs, rustc, cargo, cargo-c, tomli, and other packages are required. FYI when we actually use a Rust GStreamer element.

[ehfd]

I almost included gstreamer-rs to the GStreamer build pipeline, but there was an upstream issue with meson preventing builds in Ubuntu 20.04.

Postponing until the next major release.

[ehfd]

The issue: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/398

[ehfd]

Integrated in build pipeline with GStreamer 1.24.0.

[tt2468]

I should probably point out that I think this should be possible without the Rust gstreamer system. WHIP is just signalling, and from a (admittedly rather quick) glance at the code, the signalling for the current webRTC system is done in python, with SDP's handed to/from GST.

[ehfd]

m1k1o/neko#371

@tt2468 What do you think of this blueprint?

[ehfd]

WebRTCSink is a wrapper for WebRTCBin, where settings are available in https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/blob/main/net/webrtc/src/webrtcsink/imp.rs

The issue of WHIP/WHEP is that there is no DataChannel for keys, cursors, and other information. Only read-only access would be possible without other transport protocols.

[ehfd]

The immediate property should rather be a multi-user SFU server, based on GStreamer's own signalling server.

https://gstreamer.freedesktop.org/documentation/rswebrtc/index.html#usage

#39 (comment)