부산대학교 도서관

Autonomous surgical robotics is a growing area of research, with advances being made in the areas of vision and control. Central to this research is the need for simulations to facilitate data collection and simulate learning environments for Reinforcement Learning (RL) agents. Recent simulators have facilitated RL policy generation, but lack a robust sim2real pipeline and a proven vision-based policy that can use any type of camera including the da Vinci Surgical System (dVSS) Endoscope. To solve this, we build a ROS-based sim2real pipeline that incorporates a Unity3D da Vinci Research Kit (dVRK) simulation, modular kinematics, and shared interfaces. We examine the vision-based task of cube pushing, and train RL policies to execute in real life through Domain Randomization. Our experiments evaluate model success in simulation and two camera systems: OAK-1 and the dVSS Endoscope. Our results indicate that Domain Randomization is effective at bridging the sim2real gap, and even extends to the difficult endoscope scenario. We achieve 100% transfer success rate on both OAK-1 and the dVSS Endoscope, with gains of over 60% compared to a base model with no Domain Randomization. We examine the various randomization parameters, including lighting, camera, and physics variables, and determine that all parameters play a significant role in bridging the sim2real gap. Testing across extreme lighting and camera configurations not seen in simulation, our models continue to perform well, with 85% accuracy on the OAK-1 camera. Our future work will extend to other tasks and more complex policies to take advantage of stereo-camera imaging. Further project information is available at https://medcvr.utm.utoronto.ca/iros2022-sim2real.html