Automating Surgical Peg Transfer: Calibration With Deep Learning Can Exceed Speed, Accuracy, and Consistency of Humans

Abstract

Peg transfer is a well-known surgical training task in the Fundamentals of Laparoscopic Surgery (FLS). While human surgeons teleoperate robots such as the da Vinci to perform this task with high speed and accuracy, it is challenging to automate. This paper presents a novel system and control method using a da Vinci Research Kit (dVRK) surgical robot and a Zivid depth sensor, and a human subjects study comparing performance on three variants of the peg-transfer task: unilateral, bilateral without handovers, and bilateral with handovers. The system combines 3D printing, depth sensing, and deep learning for calibration with a new analytic inverse kinematics model and time-minimized motion controller. In a controlled study of 3384 peg transfer trials performed by the system, an expert surgical resident, and 9 volunteers, results suggest that the system achieves accuracy on par with the experienced surgical resident and is significantly faster and more consistent than the surgical resident and volunteers. The system also exhibits the highest consistency and lowest collision rate. To our knowledge, this is the first autonomous system to achieve “superhuman” performance on a standardized surgical task. All data is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://sites.google.com/view/surgicalpegtransfer</uri> Note to Practitioners—This paper presents a new approach to calibrating cable-driven robots based on a combination of 3D printing, depth sensing, inverse kinematics, convex optimization, and deep learning. The approach is applied to calibrating the da Vinci, commercial surgical-assist robot, to automate a standard “pick and place” task. Experiments suggest that the resulting system matches human surgical expert performance in speed and accuracy and significantly outperforms humans in terms of consistency. All details on the system including CAD models, code, and user study data are available online.