Multimodality Attention-Guided 3-D Detection of Nonsmall Cell Lung Cancer in 18F-FDG PET/CT Images

Abstract

In this article, a 3-D detection framework for detecting nonsmall cell lung cancer (NSCLC) in ¹⁸F-fluorodeoxyglucose (¹⁸F-FDG) positron emission tomography/computed tomography (PET/CT) images and guided by a multimodality attention fusion is proposed. A total of 250 ¹⁸F-FDG PET/CT scans between January 1, 2015 and December 31, 2019 of patients who had histopathologically proven NSCLC were acquired. A customized dual-path 3-D CenterNet is used for NSCLC detection. Moreover, we propose a multimodality attention module that adaptively refines the multimodality feature map fusion. Since the 3-D convolutional neural network (CNN) requires many graphic memories and sliding windows, a <inline-formula> <tex-math notation="LaTeX">$384\times 384\times 32$ </tex-math></inline-formula> patch size is designed and used for training and testing. Fivefold cross-validation is applied for this study. The sensitivity and false positive per scan (FPPS) obtained by our proposed method are 0.96 and 1.04, respectively. Our method significantly outperforms 3-D CenterNet in terms of sensitivity <inline-formula> <tex-math notation="LaTeX">$(P = 0.031)$ </tex-math></inline-formula>. This module demonstrates the potential to be implemented in other multimodality applications. Our result performs competitively against other lung cancer detections. Furthermore, case studies show that the proposed method can detect difficult-to-diagnose NSCLCs. Our result shows the proposed method can help radiologists and medical physicists diagnose NSCLC.