model mediated teleoperation
For one of our projects, we have demonstrated the concept of building intelligent robotic systems to assist surgeons by (i) regulating the interaction force with the anatomy and (ii) computing and rendering an assistive “virtual fixture” force on surgeons’ hand to reflect the location of the anatomy on the remote side.
On the patient’s side (remote side) we use a da Vinci Research Kit (dVRK) Patient Side Manipulator (PSM) and a mockup silicone phantom model mounted on a force sensor. We have an inexpensive stereo camera system to provide stereo views to the user, to replace the Endoscopic Camera Manipulator (ECM) that is used in a real surgery.
For the surgeon (user) console side, we use a da Vinci Research Kit (dVRK) Master Tool Manipulator (MTM), including two master arms (only one is used for this demo) and a stereo viewer system that projects the two video feeds from the remote side. A standard foot pedal system is used for control.
Surgeons (users) in this demo can see “inside” the anatomy through stereo video feeds from the remote side. But to give a sense of touch, a more complicated method is required. Simply sending the readings from the force sensor will be too noisy and will add a lag between the user movements and the force feedback. To combat these problems, a model representing the environment is updated behind the scenes and that model is used to generate haptic rendering forces based on a simple spring force model. Additionally, without any input by the user, the patient side robot manipulator regulates the interaction force with the environment, relying on its own hybrid force motion controller.
This demo uses bits and pieces of the whole cisst-saw-nri framework. However, few important files are mentioned below.
- Code of the main TeleOp script
- Code for hybrid force-motion control
- Code to set the virtual fixture gains
- Code to start MTM
- Code for PSM Prox
cisst-saw-nri needs to be built and compiled. Check here for build/compile instructions.
This is a temporary method, and we need to replace this approach with a cisst implementation later. Currently we have a program developed under xPC that generates a raster trajectory scan pattern, executes(sends) the trajectory interpolation in real time, and calculates compatible (decomposed) positions in force controlled direction and motion controlled direction (hybrid force position admittance conrol). To run this part, please refer to Force-controlled scan using an xPC program.