A Deep Learning Framework for Recognising Surgical Phases in Laparoscopic Videos

Abstract

Abstract Image-based surgical phase recognition is a fundamental component for developing context-aware systems in future operating rooms (ORs) and thus enhance patient outcomes. To date, phase recognition in laparoscopic videos has been investigated, and spatio-temporal deep learning-based approaches have been introduced. However, phase recognition in laparoscopic videos is still a challenging task and requires ongoing research. In this work, a spatio-temporal deep learning approach for recognising surgical phases is proposed. The proposed framework consists of a convolutional neural network (CNN) and a cascade of three long short-term memory (LSTM) networks. The first and second LSTM networks were trained to learn temporal information from short video clips and the complete video sequence to perform tool detection. The last LSTM was employed to enforce temporal constraints of surgical phases. The proposed approach was thoroughly evaluated on the Cholec80 dataset, and the experimental results demonstrate the high recognition performance of this method.