Streaming Video

[conner-replogle]

We have 2 broad ways of doing this.

General Overview

1. On Rover web server.
   We run a ros2 node on the rover that will handle encoding and streaming the video using a streaming Method.

Pros:

• We send less data over the radios
• Simpler to implement since we can use webrtc pretty easily and not have to worry about anything else.

Cons:

• We are already planning on using ros2 integration on backend for control surfaces so it is dividing the code base.
• More load on rovers computer for encoding transcoding
• The website will have 2 backend which means we cant use relative urls and will have to use the ip/path for the camera node. Unless we relay it in the backend

2. Ros2 integration at base station and the UI backend handles the video streaming.
   In this example we talk to the cameras over ros2 at the base station and then handle encoding and streaming at the base.

Pros

• Simpler code base
• Website only has 1 backend and can use relative urls for everything as the video is just another api path.

Cons

• More data over radio

[CrispyAlice2]

I definitely think minimizing data over the radio should be a priority. The less things are going over the link, the more extra room there is for retransmissions in case of lost packets or intermittent connections. One of the other teams, I can't remember which, even had a dynamic bitrate setting to prioritize certain cameras in certain situations and compress the other ones to basically nothing. Not being able to use a relative URL is not a huge concern, any modern web framework's templating engine can solve that easily.

[conner-replogle]

Currently, I've implemented the second method within the rover-ui directory, where the UI sends uncompressed RGB frames (640p) from the rover to the base station laptop. This works but introduces a 1-2 second latency, and it struggles over low internet speeds due to the large data size. To resolve this, I’m switching to the first approach, where the rover will handle video transcoding. It will send a compressed video stream to the base station instead of transmitting the raw RGB data, improving performance under bandwidth constraints.

Regarding the relative URLs, I didn’t explain my concern clearly earlier. It’s not about the difficulty of using relative URLs but more about ensuring that the UI remains unaware of any backend configurations. Ideally, the UI should only deal with relative paths for data access and not need to know the rover’s IP or other details. This way, the UI stays agnostic to the backend, which allows us to use mock data for testing purposes. I’ve already set up mock telemetry and video data to simulate these interactions when developing the UI on machines that aren't connected to the rover.

This can be resolved by using the backend as a proxy for the rover’s video stream. For example, the relative URL /rgb_feed can point to the backend, which will internally fetch data from ROVER_IP:PORT/rgb_feed. This keeps the frontend UI “dumb” to the backend setup, making it easier to work with both live and mock data during development.

[CrispyAlice2]

The proxy through the UI backend is probably a good idea, it also opens up more options if we end up needing a fancier video system.