Advanced Remote Control Using Gesture-Based Teleoperation and Naked-Eye 3D Display

Abstract

The design of the remote control interface offers the surgeon a better, intuitive way to perform robot-assisted minimally invasive surgeries. Among various control interfaces, the gesture-based controller provides increased mobility and convenience for surgeons as if they were operating with their bare hands. However, arm fatigue issues have been reported as the surgeons have to hold their arms in an unsupported manner such that the surgical instrument is steadily located at the target position. We propose a teleoperation interface based on a novel motion mapping to solve this problem. Surgeons wave their arms or palms to move or insert the teleoperated instrument. In addition, naked-eye 3D visualization is integrated with the control interface. Surgeons are provided with adequate depth information but are free from discomfort symptoms caused by head-mounted devices. The proposed framework is demonstrated on the LapaRobot, a robotic surgical system designed for telementoring and training in laparoscopic surgery. The experimental results indicate that the proposed interface reduces users' burden and achieves more precise master path planning.