Abstract
Abstract Online recognition of surgical phases is essential to develop systems able to effectively conceive the workflow and provide relevant information to surgical staff during surgical procedures. These systems, known as context-aware system (CAS), are designed to assist surgeons, improve scheduling efficiency of operating rooms (ORs) and surgical team and promote a comprehensive perception and awareness of the OR. State-of-the-art studies for recognizing surgical phases have made use of data from different sources such as videos or binary usage signals from surgical tools. In this work, we propose a deep learning pipeline, namely a convolutional neural network (CNN) and a nonlinear autoregressive network with exogenous inputs (NARX), designed to predict surgical phases from laparoscopic videos. A convolutional neural network (CNN) is used to perform the tool classification task by automatically learning visual features from laparoscopic videos. The output of the CNN, which represents binary usage signals of surgical tools, is provided to a NARX neural network that performs a multistep-ahead predictions of surgical phases. Surgical phase prediction performance of the proposed pipeline was evaluated on a dataset of 80 cholecystectomy videos (Cholec80 dataset). Results show that the NARX model provides a good modelling of the temporal dependencies between surgical phases. However, more input signals are needed to improve the recognition accuracy.
Highlights
With the rising number of medical devices and complexity of technology in operating rooms (OR), intelligent systems are strongly required to be adopted in the surgical environment to compensate the complexity of surgical workflow and streams of data coming from medical devices
We present a method that combines a convolutional neural network (CNN) with a nonlinear autoregressive network with exogenous inputs (NARX) to perform the phase recognition task
Once the CNN was trained, the tool presence probabilities obtained by the CNN were passed to the NARX neural network to predict the surgical phases
Summary
With the rising number of medical devices and complexity of technology in operating rooms (OR), intelligent systems are strongly required to be adopted in the surgical environment to compensate the complexity of surgical workflow and streams of data coming from medical devices. While tool binary signals are provided into a machine learning method like Hidden Markov Model (HMM) [5,6] or Dynamic Time Warping (DTW) [6] to recognize a particular phase from these signals. These binary signals are generally obtained via manual annotation of laparoscopic videos or by installing additional sensors. The tool usage signals generated by the CNN were provided as input to the NARX neural network for performing a multistep-ahead prediction of the surgical phases of cholecystectomy procedures. Once the CNN was trained, the tool presence probabilities obtained by the CNN were passed to the NARX neural network to predict the surgical phases
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
