Deep multi-feature fusion residual network for oral squamous cell carcinoma classification and its intelligent system using Raman spectroscopy

Abstract

Using fiber optic Raman spectroscopy and deep neural networks, we develop an intelligent system which will be used to assist surgeons accurately and efficiently to identify oral squamous cell carcinomas (OSCC). This system is able to classify 6 types of oral tissues. To achieve this goal, a novel classification framework called deep multi-feature fusion residual network (DMFF-ResNet) is proposed. This model is based on 16,200 Raman spectral data, obtained from the normal oral tissues and the OSCC of 90 patients through the surgical resection. Firstly, the 1-dimensional RestNet50 is taken as its backbone network. Then, the output spectral features of last three blocks are extracted from backbone network for feature fusion, which is expected to learn more spatial representations and have more discriminative power. Lastly, the derived spectral features are sent into a fully-connected neural network for performing the multiclassification task. Experimental results show that the proposed model achieves a competitive classification performance compared with state-of-the-art classifiers, and its accuracy, precision, and sensitivity reach 93.28%, 93.53%, and 93.13%, respectively. Further, the proposed framework is deployed on an edge computing device to form a prototype intelligent system for OSCC detection. To validate this system, we perform an offline test experiment in another 20 patients which demonstrates the developed intelligent system can successfully discriminate OSCC and normal oral tissues, with accuracy, precision, and recall of 92.78%, 92.33%, and 92.57%, respectively. The code was available at https://github.com/ISCLab-Bistu/retinanet-OSCC.