Augmented reality provision in robotically assisted minimally invasive surgery

Abstract

Since the widespread introduction of minimally invasive surgery, extensive evidence has demonstrated its advantages over open surgery. Although there are clear benefits, such surgery creates an unnatural environment where the surgeon has lost orientation, the eye–hand–target axis and visual depth perception. Computer-enhanced surgical technology has been proposed to overcome some of these drawbacks. Master–slave tele-manipulator systems enhance manipulation capabilities and increase performance precision, but the field of view presented to the surgeon is still restricted. The aim of our research is to improve visualization during robotically assisted minimally invasive surgery by providing augmented reality facilities that will allow the surgeon to view on demand information relevant to the operation. This paper presents a technique for the calibration of the endoscopic cameras and the slave manipulators of the da Vinci™ surgical system. Calibrating the slave manipulators will enable us to address the problem of updating the virtual objects overlaid on the video frames by tracking tool–tissue interactions and the resulting dynamic changes within the scene. Initial work regarding the registration of computer-generated graphics with images obtained from the da Vinci™ surgical system is also described.