Soft Boundary Approach for Unsupervised Gesture Segmentation in Robotic-Assisted Surgery

Abstract

Robotic-assisted surgery holds significant promise to improve patient treatment by allowing surgeons to perform many types of complex operations with greater precision and flexibility than before. In order to facilitate automation of robotic surgery and more practical training for surgeons, more detailed comprehension of the surgical procedures is needed. In this regard, a key step is to develop techniques that segment and recognize surgical tasks intelligently. Surgeries involve complex continuous activities that may contain superfluous, repeated actions, and temporal variation. Therefore, any segmentation approach that has the capability to account for all these characteristics is of increased interest. Toward this goal, we develop a new segmentation algorithm, namely soft-boundary unsupervised gesture segmentation (Soft-UGS), to segment the temporal sequence of surgical gestures and model gradual transitions between them using fuzzy membership scores. The proposed framework is evaluated using a real robotic surgery dataset. Our extensive set of experiments and evaluation metrics show that the proposed Soft-UGS method is able to match manual annotations with upto 83% sensitivity, 81% precision, and 73% segmentation score. The results show that the proposed soft boundary approach can provide more insight into the surgical activities and can contribute to the automation of robotic surgeries.