User Interaction in Blender

[BradyAJohnston]

I've got a working prototype of Warp running and being interacted with inside of Blender:

blender_Y3ji8MOz1b.mp4

blender_S86FD61Oa4.mp4

So far it is working really well, but before I get too ahead of myself I wanted to see if anyone had any tips on best way to interface user interaction.

The current setup I have is to have a rigid body added to the scene, then I am just manually setting the body_q based on the transformations the user controlled object in Blender.

Setup of rigid body and particles:

        b = builder.add_body()

        builder.add_shape_box(
            pos=wp.vec3(0, 0, 0),
            hx=1 * self.scale,
            hy=1 * self.scale,
            hz=1 * self.scale,
            body=b,
        )

        builder.add_particle_grid(
            dim_x=n_x,
            dim_y=n_y,
            dim_z=n_z,
            cell_x=0.1 * 2.0,
            cell_y=0.1 * 2.0,
            cell_z=0.1 * 2.0,
            pos=wp.vec3(-1.0, 0.0, 0.0),
            rot=wp.quat_identity(),
            vel=wp.vec3(0.0, 0.0, 10.0),
            mass=1,
            jitter=self.radius * 0.1,
        )

Handling simulation steps:

    def simulate(self):
        self.state_0.clear_forces()
        self.state_1.clear_forces()
        self.model.particle_grid.build(self.state_0.particle_q, self.radius * 2)
        wp.sim.collide(self.model, self.state_0)
        self.integrator.simulate(self.model, self.state_0, self.state_1, 1 / 24)
        # swap states
        (self.state_0, self.state_1) = (self.state_1, self.state_0)

    def step(self):
        # get the blender object the user is interacting with and extract
        # the transformations from it
        obj = bpy.data.objects["Cube"]
        transform = wp.transform(
            wp.vec3(*obj.location), wp.quat(*obj.rotation_quaternion)
        )
        self.state_0.body_q.assign([transform])
        # Run simulation substeps
        self.simulate()

After each simulation step I am updating the mesh inside of Blender with the particle positions for interactive rendering.

Is using a rigid body for this the best approach? More substeps (XPBDIntegrator) does lead to a slightly more stable simulation, but particles are still bouncing away pretty wildly (because of my infinitely heavy object I cam controlling moving through the scene I suppose).

Just after some potential tips for this approach if possible, would be very much appreciated!

[christophercrouzet]

Hi @BradyAJohnston, first of all, nice work, that's great to see such an integration!

You might have seen it already but we're shipping a similar sample scene within our omni.warp extension for Omniverse, and it is set-up very similarly to the code you've shared. In that sample scene, there is some amount of bounciness when interacting with the particles but not as extreme as in your case, and it's likely because it's running at 60 FPS while yours is running at a lower frame rate of 24 FPS. In fact, if I reduce the framerate of our sample scene, then I can observe a behaviour similar to your videos.

I'm guessing the reason for your lower frame rate is due to the particle data needing to do roundtrips between the CPU and GPU, which can end up being a large overhead in comparison to the compute cost.

Usually, the larger time delta between two simulation steps, the larger interpenetrations the solver needs to resolve, and the more unstable collision handling becomes. Adding more substeps is necessary to provide a finer granularity to the solver and give it a chance to handle these collisions more gracefully. On the bright side, you should only need to send the data back to the CPU after each simulation step, not after each substep, so adding more substeps shouldn't be too costly.

[BradyAJohnston]

I now have the basic functionality for multiple rigid bodies, with some being manually user controlled, inside of an add-on for Blender: https://github.com/BradyAJohnston/warbler

With the current method of setting the positions of the rigid bodies manually, there are still a few occasions where things get really explodey:
https://github.com/user-attachments/assets/9fb5f5a1-d24a-4413-a059-45c145e6a9da

I'm guess what is happening is that with the current setup, I am manually setting the positions of the rigid body from user inputs, running a simulation step, in which the simulation is correcting the rigid body to be above the floor. This correction is what is sending the particles flying, but because I have 'user control' enabled, the rigid body is placed back in the same location every time.

Is there a way I can tell the integrator to not try and adjust the rigid body if it is being 'user controlled'?

[christophercrouzet]

Is there a way I can tell the integrator to not try and adjust the rigid body if it is being 'user controlled'?

Does that help if you set their mass to 0?

[BradyAJohnston]

I wasn't manually setting their mass so they were defaulting to 0.0 I think. I just tried with explicitly setting 0.0, really larger 1e5 and also really small 1e-5 and they all behave essentially the same, with each frame the simulation bringing it back above the ground plane, then the 'user input' manually setting it back below the plane.

If this is a just a fundamental limitation, then that's OK and I can instead maybe work with manually creating a rigid body plane that is slightly above the (0,0) ground plane and it's just a limitation of the add-on.

[eric-heiden]

Hi @BradyAJohnston,

You could try setting wp.sim.Model.particle_max_velocity to some lower value than the default to limit how fast the particles can move. Another idea is to decrease the soft_contact_relaxation parameter given to the XPBDIntegrator constructor. This parameter controls how much impulse is given to the particles to resolve contact.

Generally, when you initialize the simulation with a large penetration, i.e. many particles start inside the box, it will be tricky to achieve stable behavior. So, if possible, you may want to initialize the scene in a way that there are less severe overlaps. Also, when you apply the movement to the box you may want to use some smoothing to interpolate between the user inputs, e.g. a simple low-pass filter where the transform from the previous frames is combined with the latest transform using some weighted average that places larger weight on the most recent time step. Then you could also infer the velocity of the box (setting its body_qd) from difference(s) between the transforms which will make the simulation more realistic since contact friction (which depends on velocity) can now be computed between the box and the particles.

[BradyAJohnston]

Hi @eric-heiden thanks for the tips! The tip about lerping / smoothing out the input movements of the rigid body solves all but one of the issues with the particles. Now with general user interaction everything behaves much more like you would expect, and I can adjust the smoothing to get the desired result.

The only remaining issue I am having is when the rigid body is placed below the world ground. The simulator on each frame moves it back above ground to remove any collisions, then the user input immediately moves it back below the ground, leading the catapult of particles.

Is there a way to tell the simlator (currently using XPBD) to not resolve the collisions of certain rigid bodies?

One solution would be to limit the user input of a rigid body under the world ground, but I can see this also becomes a problem if two user-controlled rigid bodies overlap, the same correction, explosion of particles, then back to the user input position happens there as well.

I might be able to solve some of it with changing the impulse / max velocity like you suggested, haven't tried those yet, but ideally I would be able to just tell a rigid body to not be moved by the integrator.

Blender.Onlpax71oa-1.mp4