Touchless gesture user interface for interactive image visualization in urological surgery

Abstract

In the operating room (OR) a touchless interface is an ideal solution since it does not demand any physical contact and still can provide the necessary control features in a cleansed and sterilized environment. Using open-source software libraries and image processing techniques, we implemented a hand tracking and gesture recognition system based on the Kinect device that enables surgeon to successfully touchlessly navigate through the image in the intraoperative setting through a personal computer. We used the InVesalius software, which provides high-quality 3D reconstruction of medical images. Computed tomography data were intraoperatively fruitfully accessed through a simple and cheap solution in 4 tumor enucleations in 3 male patients in whom elective nephron-sparing surgeries were performed for small non-exophitic tumors. Mean tumor length was 2.7 cm (2.1, 2.7, 2.9 and 3.1 cm), and real-time ultrasound was not necessary for intraoperative identification in 3 of 4 endorenal tumors. All pathological reports revealed renal cell carcinoma, Fuhrman grade I, and negative inked surgical margins. No intra- or postoperative complication was reported. For the first time in the literature, a touchless user interface solution applying the Kinect device showed to be very efficient and enabled a low-cost and accurate control of the software InVesalius intraoperative, just using hand gestures. It can be used with any mouse-controlled software, opening an avenue for potential applications in many other areas, such as data visualization, augmented reality, accessibility, and robotics. The further validation and advancement of this technology are underway.