Multimodality Alzheimer's Disease Analysis in Deep Riemannian Manifold

Abstract

Structural Magnetic Resonance Image (sMRI) and functional MRI (fMRI) are two of the most important modalities to unveil brain disorders for Alzheimer's disease (AD) analysis. Comprehensively utilizing both modalities is the way to ensure an accurate AD diagnosis. Currently, the most common computational approach to aid the AD diagnosis is to formulate the sMRI and fMRI into graphs and then extract discriminative features through Graph Neural Networks (GNNs). However, most GNNs rely heavily on the aggregation operation on each node, which exploits the local topological information from the neighborhood nodes but does not fully respect the characteristics of the global graph topology. Also, only a few works addressed the structural and functional coupling problem on the graphs. In this paper, a novel Riemannian manifold-based model, called Cross-Modal Riemannian Network (CMRN), is proposed to solve the above issues, which respects the global topologies and invariant characteristics of the sMRI and fMRI graphs by fully operating on the Riemannian Manifold. Furthermore, a novel cross-modal attention mechanism is proposed to enable the interactions between two modalities on the Riemannian manifold, which helps the model comprehensively utilize both modalities to identify the most discriminative information for AD diagnosis. Extensive experimental results on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset demonstrate the effectiveness of the proposed method.