Scene replication

[blueforesticarus]

My (rough draft) proposal for replication of scene children (and other client spawned entities).

Generic case:

1. Children to be replicated are given a ReplicationId component (on both client and server) containing a deterministic ID.
2. Entities with ReplicationId are automatically given Replicated on the server (allowing the same spawning code on client and server).
3. ReplicationId is included when replicating server entities to the client
4. replicated entities with a ReplicationId component are buffered by the client until the client-side entity with the same ReplicationId become available, at which point a client side owned entity map (as opposed to the one currently owned by the server) is updated to point the server entity to the client entity, and the buffered replication messages are applied.
5. duplicate ReplicationId's can be supported for symmetric cases (ie. spawning 10 pigs)
6. on connection of a new client, entities will get replicated as normal, with the ReplicationId, and presumably whatever machinery spawned the parent (ie. MySceneSpawnerMarker ) gets replicated as well and creates the needed children on the new client.

Scene case:

1. Children to be replicated should have AutoReplicationId component attached by client and server
2. a system should be written to replace AutoReplicationId with a ReplicationId based on the hash of the hierarchy path using Name components, using as the root the nearest parent which is already replicated.
   ie. 123/my/scene/ball_12 where 123 is the server entity_id of a replicated entity ( most likely with a MySceneSpawner or such component )
3. the system will also add Replicated to the components.

Potential issues:

• AutoReplicationId might end up with race condition if spawning replicated children of replicated children (which entity is used as the root)
• what should the server do if ReplicationId is added to an already replicated entity.

Prior art:
I haven't been able to find anything specific to scene replication.

Inspiration for using a deterministic hash for mapping comes from how lightyear does prespawning.
https://docs.rs/lightyear/latest/lightyear/client/prediction/prespawn/struct.PreSpawnedPlayerObject.html

[Shatur]

Adding ReplicationId to each entity will significantly increase the replication size. I am also not sure how adding it to each entity solves the problem.

The problem

The issue is that if you attach Replicated to a scene entity, it will be replicated as a new entity. But we want to map it to an already existing entity.

Possible solution

1. Detect if the entity is part of a replicated scene (contains Replicated and Handle<Scene> or Handle<DynamicScene>).
2. If the server adds Replicated, send information to the client to map it to an already spawned scene entity.

But what if the scene hasn't been spawned by the client yet? 🤔

This means we need to spawn it as part of the replication process (just to know scene entities), which might require embedding scene spawning into replication. Currently, to replicate a scene, the user simply replicates a marker and initializes the scene manually in a system.

Another option would be to buffer scene entities until the scene is spawned. Could store scene entities as slices of Bytes.

Maybe you have another ideas?

Mapping could be encoded in the replication message like this

1. The server Entity of the root scene instance.
2. An array of Entity to Entity mappings:
   2.1 The local scene Entity (each entity in the scene has it, essentially the ID before spawning).
   2.2 The server-spawned Entity.

1 and 2.2 will be mapped to client-local entities, and 2.1 is already known by the client.

We have two kinds of replication messages: UpdateMessage and MutateMessage. This information should be part of UpdateMessage. The current sections are defined here:

bevy_replicon/src/core/replication/update_message_flags.rs

Lines 8 to 13 in 6b9bd43

	pub(crate) struct UpdateMessageFlags: u8 {
	const MAPPINGS = 0b00000001;
	const DESPAWNS = 0b00000010;
	const REMOVALS = 0b00000100;
	const CHANGES = 0b00001000;
	}

We might need to add another section for scene mappings.

Prior art

Both Unreal Engine and Godot allow replication to be enabled for every actor or node in a scene. It would be nice to look at their approach 🤔

[blueforesticarus]

I was not suggesting adding ReplicationId to every entity, only descendant entities which you want to be replicated to an entity spawned by the client.

I think this issue is more general than scenes. It applies to any hierarchical structures.

The method of spawning a scene with a marker Component seems reasonable to me to keep the same. The problem is matching up the child entities.

I think buffering scene entities (really, anything with ReplicationId) is the right move. It seems like replicon already does some client side buffering for another purpose (though I didn't look to closely at why).

[Shatur]

Ah, got it!

I would consider hierarchical replication as a separate thing. For scenes we only need to match entities once on scene spawning. If we solve this, attaching ParentSync to such entity will just work for synchronizing hierarchy changes.

It seems like replicon already does some client side buffering

Yes, we buffer mutate messages to apply them only if update message is received.

Before I write detailed information about the implementation, we need to figure out how to properly generate IDs. Looks like the upcoming BSN format won't have Entity inside the scene. They rely on scene path. But I need to clarify this, already asked in the working group in Discord.

[blueforesticarus]

I would consider hierarchical replication as a separate thing.

I think maybe we are still talking past each other. What I mean is: the problem in my mind is replicating things which, as part of their spawning, spawn child entities which need to be replicated. (I'm not, and haven't been, talking about changes to hierarchy)

A scene is one example where this is useful. It also has an extra constraint.

1. because scenes are often assets, we want to avoid having needing to write a system specific to the scene. We'd prefer some generic solution using a marker component that can work for all scenes.

Thinking in terms of the current Marker component method for spawning.

1. MyPlayerMarker hydrates my player entity by adding components (like mesh and material) symmetrically on both client and server. This works.
2. MyPlayerMarker2 is the same but also spawns children. This is a problem atm in replicon.

Really at the low level the problem is the replication of entities that (on the client) must be spawned by client code. Replicon currently assumes that anything it replicates to the client, it also spawns on the client (ignoring prespawning).
Ie. creation and mutation are a package deal.
If this assumption is removed, we need a way for the replicon client to map entities it didn't spawn. A unique id (hashed to a u64 or something), seems like the obvious choice for the low level. Any other method of mapping can be implemented as a system which assigns these unique ids.

Before I write detailed information about the implementation, we need to figure out how to properly generate IDs.

So to this, I would say, I think (optional) ID based replication is more important/fundamental.

Technically the IDs don't even need to be generated by replicon (though I do think we should provide a solution, but if BSN is evolving it might not be a super stable one).

[Shatur]

2. MyPlayerMarker2 is the same but also spawns children. This is a problem atm in replicon.

There is no issue here. You can spawn hierarchies on the server with ParentSync and have systems that initialize entities for both the client and server. It works exactly like "1."

The scene use case is completely different. You want to have base entities on both the client and server and receive updates to these values if they change. There is no need to introduce custom IDs for hierarchy replication. We need them specifically for scenes to determine which scene entity belongs to which spawned entity.

[blueforesticarus]

There is no issue here. You can spawn hierarchies on the server with ParentSync and have systems that initialize entities for both the client and server. It works exactly like "1."

...spawns children that themselves need replication. I should have been more clear.

I maintain that scenes are only a single instance of the more general client side / symmetric spawning problem. Which I suppose is potentially more general than hierarchical structures, just any group of entities spawned by the same code.

[Shatur]

...spawns children that themselves need replication.

Yes, I get this. It's supported. With both spawning on server and pre-spawning on client.

Which I suppose is potentially more general than hierarchical structures, just any group of entities spawned by the same code.

I can't think of any use case other then scenes. For all other use cases you can use regular replication from server or pre-spawning.

[blueforesticarus]

Yes, I get this. It's supported. With both spawning on server and pre-spawning on client.

I don't get how currently you can accomplish this with the pre-spawning system. It seems like the client must spawn the entity first, because it has to give the server the entity ID.

Maybe I don't actually get how pre-spawning works, it does seem bizarre and roundabout to me.

At least this example: https://docs.rs/bevy_replicon/latest/bevy_replicon/server/client_entity_map/struct.ClientEntityMap.html

A) is not symmetric
B) seems to require spawning on the client first.

[Shatur]

It seems like the client must spawn the entity first, because it has to give the server the entity ID.

That's the whole idea of pre-spawning. You spawn entity or entities and notify server about them. Then server spawns, and client will receive replication as like they were spawned on server. It's useful if you want to hide delay on client, typically needed for prediction for fast-paced games.

If you don't need to spawn on client first - you just spawn things on server. If you also want to replicate the hierarchy - you insert ParentSync.

A) is not symmetric
B) seems to require spawning on the client first.

Yes, it's completely different from the scene case.