Whole-brain parcellation for macaque brain magnetic resonance images based on attention mechanism and multi-modality feature fusion

Abstract

We propose a novel deep learning algorithm based on attention mechanism and multimodality feature fusion (DDAM) to achieve whole-brain parcellation of macaque brain magnetic resonance (MR) images. We collected multimodality brain MR images of 68 macaques (aged 13 to 36 months) with corresponding ground truth labels. To address the complexity and complementary nature of the multimodality data, we employed a multi- encoder structure to adapt to different modalities and performed feature extraction. In the decoder, an attention mechanism was introduced to construct the Attention-based Multimodality Feature Fusion module (AMFF). Leveraging the rich and complementary information between modalities, AMFF effectively integrated multimodality features of varying scales and complexities to enhance the performance of parcellation. We conducted ablation experiments and statistically analyzed the results. The incorporation of the multi- encoder structure and attention mechanism significantly improved the performance of the network for integrating the multimodality features, and achieved an average DSC of 0.904 and an ASD as low as 0.131 for macaque whole-brain parcellation (P < 0.05). The ablation experiments validated the effectiveness of each component of the DDAM method. The proposed DDAM method, with its enhanced ability to extract and integrate multimodality features, can significantly improve the accuracy of whole-brain MR image segmentation.