Adaptive kernel selection network with attention constraint for surgical instrument classification

Yaqing Hou,Qiang Zhang,Hongwei Ge,Xiaopeng Wei,Wenkai Zhang,Jun Meng,Qian Liu

doi:10.1007/s00521-021-06368-x

Yaqing Hou, Qiang Zhang + Show 5 more

Open Access

https://doi.org/10.1007/s00521-021-06368-x

Copy DOI

Abstract

Computer vision (CV) technologies are assisting the health care industry in many respects, i.e., disease diagnosis. However, as a pivotal procedure before and after surgery, the inventory work of surgical instruments has not been researched with the CV-powered technologies. To reduce the risk and hazard of surgical tools’ loss, we propose a study of systematic surgical instrument classification and introduce a novel attention-based deep neural network called SKA-ResNet which is mainly composed of: (a) A feature extractor with selective kernel attention module to automatically adjust the receptive fields of neurons and enhance the learnt expression and (b) A multi-scale regularizer with KL-divergence as the constraint to exploit the relationships between feature maps. Our method is easily trained end-to-end in only one stage with few additional calculation burdens. Moreover, to facilitate our study, we create a new surgical instrument dataset called SID19 (with 19 kinds of surgical tools consisting of 3800 images) for the first time. Experimental results show the superiority of SKA-ResNet for the classification of surgical tools on SID19 when compared with state-of-the-art models. The classification accuracy of our method reaches up to 97.703%, which is well supportive for the inventory and recognition study of surgical tools. Also, our method can achieve state-of-the-art performance on four challenging fine-grained visual classification datasets.

Highlights

The health care sector has long been an early adopter and benefited greatly from technological advances
To reduce the risk and hazard of surgical tools’ loss, we propose a study of systematic surgical instrument classification and introduce a novel attention-based deep neural network called SKA-ResNet which is mainly composed of: (a) A feature extractor with selective kernel attention module to automatically adjust the receptive fields of neurons and enhance the learnt expression and (b) A multi-scale regularizer with KL-divergence as the constraint to exploit the relationships between feature maps
Despite the remarkable success of Computer vision (CV) on auxiliary medical diagnosis, recent studies have actively ventured into other emerging application domains in the health care sector, for example, the robot-assisted surgery based on a 3D camera [15], rehabilitation training based on vision reconstruction for people with visual impairment [6], health monitoring on patients for disease prediction and prevention [33], etc

Summary

Introduction

The health care sector has long been an early adopter and benefited greatly from technological advances. Despite the remarkable success of CV on auxiliary medical diagnosis, recent studies have actively ventured into other emerging application domains in the health care sector, for example, the robot-assisted surgery based on a 3D camera [15], rehabilitation training based on vision reconstruction for people with visual impairment [6], health monitoring on patients for disease prediction and prevention [33], etc Among these studies, relevant research works on medical instrument images, i.e., the surgical instruments, which are the most important tools in the procedure of surgery, have less been explored. In the detection of COVID-19, Apostolopoulos et al [2] suggest that the state-of-the-art CNN architectures proposed over the recent years for medical image classification with transfer learning are successful in extracting significant biomarkers related to the COVID-19 disease based on X-ray imaging

Methods

Results

Conclusion