A Low-Cost Computer Vision Based Approach for Tracking Surgical Robotic Tools

Abstract

The number of Robot-Assisted Minimally Invasive Surgery (RMIS) procedures has grown immensely in recent years. The number of surgeries performed with the da Vinci (Intuitive Surgical, Sunnyvale, CA) worldwide in 2005 was under 50,000. This number grew to more than 350,000 by 2011 [1]. RMIS procedures provide improved patient recovery time and reduced trauma due to smaller incisions relative to traditional open procedures. Given the rise in RMIS procedures, several organizations and companies have made efforts to develop training and certification criteria for the da Vinci robot. Mimic technologies (Mimic, Seattle, WA) and Intuitive Surgical both produce virtual reality (VR) training simulators based on the da Vinci system. The Fundamentals of Robotic Surgery (FRS) consortium has been created with the goal of developing a standardized, high-stakes certification exam for robotic surgery [1]. While still in the development stage, this exam will consist of seven tasks carried out on a small physical module with an actual RMIS system. Each task is evaluated with a certain set of criteria including completion time, total tool path length and economy of motion, which is a measurement of deviation from an ‘ideal’ path. All of these metrics can benefit greatly from an accurate, inexpensive and modular tool tracking system that requires no modification to the existing robot. While the da Vinci uses joint kinematics to calculate the tool tip position and movement internally, this data is not openly available to users. Even if this data was open to researchers, the accuracy of kinematic calculations of end effector position suffers from compliance in the joints and links of the robot as well as finite uncertainties in the sensors. In order to find an accurate, available and low-cost alternative to tool tip localization, we have developed a computer vision based design for surgical tool tracking. Vision systems have the added benefit of being low cost with typical high resolution webcams costing around $50. The stereo setup for this design cost around $120. Chmarra et al. reviewed the available non-robotic, laparoscopic tracking devices in 2007 and discussed 4 main technologies for tracking; mechanical, visual, ultrasonic, and electromagnetic [2]. From this work, it became apparent that in order to track robotic tools, only the visual or ultrasound-based methods would be feasible. The goal of our research was to develop a tracking system which could accompany the FRS module or be used separately during real procedures to gauge performance post-procedure using the da Vinci camera feed. The system was designed to use only a camera setup and a computer loaded with the computer vision software. Given the bandwidth of surgical tool motions has been experimentally determined as falling below 8 Hz [3], we adopt 16 Hz, or frames per second (FPS), as a minimal frame rate required to accurately capture surgical tool motion data. 2 Methods