[Design] How to sync across clients

[snowymo]

Is your feature request related to a problem? Please describe.
How to synchronize all the information across different clients through the server.

Describe the solution you'd like

1. We need to prepare a welcome package that includes the states of everything that needs to be shared when a new user enters metaroom. For example, we need to send the snapshot that contains the avatar representation as well as object transformation.
2. We only need to sync input events to the server and all remote clients can handle the input events locally.
3. [question] How the local client who sends the input server to the server behave? waiting for the server to broadcast the input events => cause latency; apply the input event directly => cause inconsistency.
4. [question] Send snapshots from the server once a while?

Describe alternatives you've considered
Synchronize everything for now.

[snowymo]

@cdefanti I will work on a document that records what types and for each type what attributes we want to sync over the network for now. And looking forward to your design doc.

[snowymo]

@cdefanti roughly a doc describing how the current network is implemented.

https://github.com/futurerealitylab/immersive-presentation/wiki/%5BDesign%5D-Synchronization-between-clients

[cdefanti]

Here's a few thoughts I jotted down today: https://docs.google.com/document/d/1UIjWdOAi3U-oRR-vsdAQhsjw6I77IBnaqHuLfxfyuWQ/edit?usp=sharing
We should discuss them when we can next meet, but I think having a base structure of a Networked Object that can have some automatically supported synchronized variables, such as an ID and transform, would be really beneficial. The avatar could inherit this base class, plus add some of the properties already noted in your wiki document.

[keruwang]

Hi @snowymo. I just created a simple demo called SyncTest in branch syncTesting. In the demo you could lift the cube by pressing the space bar. I hope this could be helpful for you to test the event synchronisation!

[snowymo]

Hi @snowymo. I just created a simple demo called SyncTest in branch syncTesting. In the demo you could lift the cube by pressing the space bar. I hope this could be helpful for you to test the event synchronisation!

It'd be better to inform me the code position for control and for drawing. Oh wait we don't have drawing here. So the sample may be too simple.

Also, we need to apply our implementation to the controller-based event in the future, so it'd be better that this sample is functional on a controller event. For example, When user A presses a trigger, the cube lifts. In the meantime, we have a hack that when user A presses key 'space', the cube also lifts. And the function called by pressing event should be the same so I don't need to add network code in two different places.

Another reason I need to know the code location because we probably will have a separate place to handle all the controller events even for local ones.

[snowymo]

I have a simple implementation for events. Check the new branch https://github.com/futurerealitylab/immersive-presentation/tree/input_event.

Also here is how it looks like. Since Keru only provided an example for moving an object, I did not take use of the controller's information but only the button event.

20210530_193347_Trim_Trim.mp4

[snowymo]

We need to have a welcome package for every newly joined user.