Sensor-based surgical activity recognition in unconstrained environments

Abstract

Introduction: Automatic surgical activity recognition in the operating room (OR) is mandatory to enable assistive surgical systems to manage the information presented to the surgical team. Therefore the purpose of our study was to develop and evaluate an activity recognition model. Material and methods: The system was conceived as a hierarchical recognition model which separated the recognition task into activity aspects. The concept used radio frequency identification (RFID) for instrument recognition and accelerometers to infer the performed surgical action. Activity recognition was done by combining intermediate results of the aspect recognition. A basic scheme of signal feature generation, clustering and sequence learning was replicated in all recognition subsystems. Hidden Markov models (HMM) were used to generate probability distributions over aspects and activities. Simulated functional endoscopic sinus surgeries (FESS) were used to evaluate the system. Results and discussion: The system was able to detect surgical activities with an accuracy of 95%. Instrument recognition performed best with 99% accuracy. Action recognition showed lower accuracies with 81% due to the high variability of surgical motions. All stages of the recognition scheme were evaluated. The model allows distinguishing several surgical activities in an unconstrained surgical environment. Future improvements could push activity recognition even further.