Warp Investigations

[alberthli]

Currently, there seems to be a bug in the way warp collisions are processed in the batched setting.

What works: With the warp box-box collider fix on the @pculbertson fork, we can get box-box collision gradients (as well as the visualized collision point geoms) to match for two identical box-box collisions in a batched call.

What fails: When we extend this to the imported LEAP hand colliding with a box, setting the state identically for both elements of a batch produces inconsistent results. In particular, there are two issues at play.

1. Because Warp and mujoco use different axis conventions (Warp uses y-up, mujoco z-up), the models themselves somehow aren't importing correctly from XML.
2. However, assuming that the first element in the batch as a reference for the second, the results still don't match. In particular, the second element in the batch thinks collisions are happening very far away from the box object and its gradients don't match the first element of the batch.

Action items:

• Investigate the minimum example that breaks (e.g., does loading a URDF of a two-link "robot" cause the kinematics/collider issues?).
  • Doesn't break! Fixed the collision kernel.
• Write unit tests for collision pair checking in a batched context
• Clean up torch/warp interop, potentially generalize the creation of torch.autograd.Function
• Write unit tests that check whether gradients point in the "right" direction simple test examples (e.g., cube-cube stacked points gradients up/down).
• Write a unit test for loss decrease after several torch gradient steps on a simple system.
• Fix up a clean URDF of the LEAP hand. Potentially replace the fingertip geoms with boxes as well in the short term.
• Ensure clean LEAP hand model loads cleanly into Warp using their GL viz tools