Is it possible to teleoperate with the remote instead of hand tracking?

[jmcoholich]

Hi,

We are teleoperating a Franka Panda robot with just a gripper. We have found that the gripper actuation (clicking the thumb and pinky together) is unreliable and causes the hand position to shift in the process. This makes grasping objects very difficult.

It it possible to teleoperate with the Meta Quest remote and use a button for actuating the gripper? This would give us better position tracking and easier gripper actuation.

If not, could you recommend any other codebase/stack that does this?

Thanks,
Jeremiah

[dannyXSC]

If you want to use remote, it should work. The key of this task is to find the position and rotation information of the remote, for which you need to refer to the documentation.
However, it is possible to control the gripper actuation with thumb and Index finger without exerting a strong influence on the position of the hands.
If you can provide the tasks you want to accomplish and the situations you cannot tolerate, it will be easier for me to help you.

[jmcoholich]

I see. In order to implement this change, we would have to change the Unity code and the .apk file right?

"However, it is possible to control the gripper actuation with thumb and Index finger without exerting a strong influence on the position of the hands."

• Unfortunately, we found this to be false. Everytime we try to actuate the the gripper with our fingers, the hand position shifts. We tried the thumb and index finger, but the fingers are too close and we get false positives on them touching.