Smoke detection in endoscopic surgery videos: a first step towards retrieval of semantic events

Abstract

Event-based annotation of surgical operations has not received much attention, mainly due to diversity of the visual content. As a first attempt at retrieval of surgical events, we address the problem of detecting the smoke produced by electrosurgery tasks. After video decomposition into shots, a grid of particles is placed over the initial frame. The grid is advected with the space-time optical flow and a number of ad hoc kinematic features are extracted. After feature selection, a one-class support vector machine is employed for classification. A vision-based fire surveillance method is used for comparison. Experimental evaluation is performed on individual shots and laparoscopic cholecystectomy videos. In the first set-up, average specificity and sensitivity were 86% and 83%, respectively. In video-based assessment the recognition accuracy was ≥ 80% for two of the three videos tested. The fire surveillance method had a maximum accuracy of 63%. The irregular movement of smoke was captured robustly by the proposed features, which could also be employed for interpretation of other semantic occurrences in surgical videos.