Lightweight Deep Neural Network for Real-Time Instrument Semantic Segmentation in Robot Assisted Minimally Invasive Surgery

Abstract

Vision-based detection and tracking of surgical instrument is attractive because it relies purely on existing setup already in the operating scenario for Robot assisted minimally invasive surgery (RMIS). While significant advanced approaches have been made in recent years, how to effectively carry out the efficient algorithm in the abdominal environment for real-time surgical instrument semantic segmentation is still a fundamental challenging topic. For this purpose, this letter reports the development of a surgical instrument semantic segmentation method for RMIS scenario. We have demonstrated a novel lightweight deep neural network to exploit surgical instrument semantic segmentation that meets real-time, accuracy and robustness. Specifically, our model contains an innovative design of two stages lightweight network by integrating the Ghost modules with MobileNetV3. Given that reducing computation cost with holding high accuracy is crucial for the task, in the first stage, Ghost module is explored to model the bottleneck for extracting the semantic features. We also design the lightweight Segmentation head as the second stage to further reduce computation cost, the model outputs a pixel-by-pixel mask to represent the surgical instrument parts semantic segmentation results. Promising validation has been conducted to evaluate the proposed lightweight network, the results report that the method can exhibit the outstanding surgical instrument semantic segmentation and surpass existing algorithms in the real-time performance with holding the high accuracy. The letter demonstrates the feasibility to perform real-time surgical instrument detection by taking advantage of the components of surgical robot system, contributing to the reference for further surgical intelligence.