Fine-Grained Lung Cancer Classification from PET and CT Images Based on Multidimensional Attention Mechanism

Ruoxi Qin,Zhenzhen Wang,Junling Xu,Jian Chen,Jinjin Hai,Lingyun Jiang,Bin Yan,Kai Qiao,Dapeng Shi

doi:10.1155/2020/6153657

Abstract

Lung cancer ranks among the most common types of cancer. Noninvasive computer-aided diagnosis can enable large-scale rapid screening of potential patients with lung cancer. Deep learning methods have already been applied for the automatic diagnosis of lung cancer in the past. Due to restrictions caused by single modality images of dataset as well as the lack of approaches that allow for a reliable extraction of fine-grained features from different imaging modalities, research regarding the automated diagnosis of lung cancer based on noninvasive clinical images requires further study. In this paper, we present a deep learning architecture that combines the fine-grained feature from PET and CT images that allow for the noninvasive diagnosis of lung cancer. The multidimensional (regarding the channel as well as spatial dimensions) attention mechanism is used to effectively reduce feature noise when extracting fine-grained features from each imaging modality. We conduct a comparative analysis of the two aspects of feature fusion and attention mechanism through quantitative evaluation metrics and the visualization of deep learning process. In our experiments, we obtained an area under the ROC curve of 0.92 (balanced accuracy = 0.72) and a more focused network attention which shows the effective extraction of the fine-grained feature from each imaging modality.

Highlights

In the 21st century, cancer is still considered a serious disease as the mortality rates are high
For non-small-cell lung cancer, a subcategorization into lung squamous cell carcinoma (LUSC) and lung adenocarcinoma (LUAD) is further used. ese types of cancers account for approximately 85% of lung cancer cases [3]
Our experiments mainly demonstrate the methods presented in this paper considering three aspects: (1) the validity of multimodality data. (2) e validity of multidimensional attention mechanism on each modality. (3) e validity of the feature fusion strategy

Summary

Introduction

In the 21st century, cancer is still considered a serious disease as the mortality rates are high. Ere are few related studies on how to use the attention mechanism more effectively on images with different imaging modalities, so the deep learning model based on the multimodality dataset still has problems in fine-grained problems. 2. Methods e network construction is mainly divided into the following two parts: (1) e multidimensional attention mechanism proposed in the single-path network architecture is the method of fine-grained feature extraction for each modality. In contrast to the widely used connection operation, GMU allows to use hidden structures and gate controls to learn the intermediate representation of the multimodality features, enabling the prediction layer to assign weights to features that have intrinsic associations better. Under the premise of a final classification, this training method forces the network to extract better high-level features from each modality for the generation of spatial attention to avoid trapping in a local minimum because of the use of feature from each level

Experiments and Results

Visualization Experiment and Discussion

Conclusions