COVID-19 diagnosis based on swin transformer model with demographic information fusion and enhanced multi-head attention mechanism

Abstract

Coronavirus disease 2019 (COVID-19) is an acute disease, which can rapidly become severe. Hence, it is of great significance to realize the automatic diagnosis of COVID-19. However, existing models are often inapplicable for fusing patients’ demographic information due to its low dimensionality. To address this, we propose a COVID-19 patient diagnosis method with feature fusion and a model based on Swin Transformer. Specifically, two auxiliary tasks are added for fusing computed tomography (CT) images and patients’ demographic information, which utilizes the patients’ demographic information as the label for the auxiliary tasks. Besides, our approach involves designing a Swin Transformer model with Enhanced Multi-head Self-Attention (EMSA) to capture different features from CT data. Meanwhile, the EMSA module is able to extract and fuse attention information in different representation subspaces, further enhancing the performance of the model. Furthermore, we evaluate our model in COVIDx CT-3 dataset with different tasks to classify Normal Controls (NC), COVID-19 cases and community-acquired pneumonia (CAP) cases and compare the performance of our method with other models, which show the effectiveness of our model. In addition, we have conducted various visualization efforts to demonstrate the interpretability of our model, including principal component analysis, attention heatmaps, etc. Various results indicate that our model is capable of making reasonable diagnosis.