Multimodal Magnetic Resonance Research Articles

Bridging MRI Cross-Modality Synthesis and Multi-Contrast Super-Resolution by Fine-Grained Difference Learning.

In multi-modal magnetic resonance imaging (MRI), the tasks of imputing or reconstructing the target modality share a common obstacle: the accurate modeling of fine-grained inter-modal differences, which has been sparingly addressed in current literature. These differences stem from two sources: 1) spatial misalignment remaining after coarse registration and 2) structural distinction arising from modality-specific signal manifestations. This paper integrates the previously separate research trajectories of cross-modality synthesis (CMS) and multi-contrast super-resolution (MCSR) to address this pervasive challenge within a unified framework. Connected through generalized down-sampling ratios, this unification not only emphasizes their common goal in reducing structural differences, but also identifies the key task distinguishing MCSR from CMS: modeling the structural distinctions using the limited information from the misaligned target input. Specifically, we propose a composite network architecture with several key components: a label correction module to align the coordinates of multi-modal training pairs, a CMS module serving as the base model, an SR branch to handle target inputs, and a difference projection discriminator for structural distinction-centered adversarial training. When training the SR branch as the generator, the adversarial learning is enhanced with distinction-aware incremental modulation to ensure better-controlled generation. Moreover, the SR branch integrates deformable convolutions to address cross-modal spatial misalignment at the feature level. Experiments conducted on three public datasets demonstrate that our approach effectively balances structural accuracy and realism, exhibiting overall superiority in comprehensive evaluations for both tasks over current state-of-the-art approaches. The code is available at https://github.com/papshare/FGDL.

M 2 FTrans: Modality-Masked Fusion Transformer for Incomplete Multi-Modality Brain Tumor Segmentation.

Brain tumor segmentation is a fundamental task and existing approaches usually rely on multi-modality magnetic resonance imaging (MRI) images for accurate segmentation. However, the common problem of missing/incomplete modalities in clinical practice would severely degrade their segmentation performance, and existing fusion strategies for incomplete multi-modality brain tumor segmentation are far from ideal. In this work, we propose a novel framework named M 2 FTrans to explore and fuse cross-modality features through modality-masked fusion transformers under various incomplete multi-modality settings. Considering vanilla self-attention is sensitive to missing tokens/inputs, both learnable fusion tokens and masked self-attention are introduced to stably build long-range dependency across modalities while being more flexible to learn from incomplete modalities. In addition, to avoid being biased toward certain dominant modalities, modality-specific features are further re-weighted through spatial weight attention and channel- wise fusion transformers for feature redundancy reduction and modality re-balancing. In this way, the fusion strategy in M 2 FTrans is more robust to missing modalities. Experimental results on the widely-used BraTS2018, BraTS2020, and BraTS2021 datasets demonstrate the effectiveness of M 2 FTrans, outperforming the state-of-the-art approaches with large margins under various incomplete modalities for brain tumor segmentation. Code is available at https://github.com/Jun-Jie-Shi/M2FTrans.

Quantitative Analysis of Multimodal MRI Markers and Clinical Risk Factors for Cerebral Small Vessel Disease Based on Deep Learning.

Cerebral small vessel disease lacks specific clinical manifestations, and extraction of valuable features from multimodal images is expected to improve its diagnostic accuracy. In this study, we used deep learning techniques to segment cerebral small vessel disease imaging markers in multimodal magnetic resonance images and analyze them with clinical risk factors. We recruited 211 lacunar stroke patients and 83 control patients. The patients' cerebral small vessel disease markers were automatically segmented using a V-shaped bottleneck network, and the number and volume were calculated after manual correction. The segmentation results of the V-shaped bottleneck network for white matter hyperintensity and recent small subcortical infarction were in high agreement with the ground truth (DSC>0.90). In small lesion segmentation, cerebral microbleed (average recall=0.778; average precision=0.758) and perivascular spaces (average recall=0.953; average precision=0.923) were superior to lacunar infarct (average recall=0.339; average precision=0.432) in recall and precision. Binary logistic regression analysis showed that age, systolic blood pressure, and total cerebral small vessel disease load score were independent risk factors for lacunar stroke (P<0.05). Ordered logistic regression analysis showed age was positively correlated with cerebral small vessel disease load score and total cholesterol was negatively correlated with cerebral small vessel disease score (P<0.05). Lacunar stroke patients exhibited higher cerebral small vessel disease imaging markers, and age, systolic blood pressure, and total cerebral small vessel disease score were independent risk factors for lacunar stroke patients. V-shaped bottleneck network segmentation network based on multimodal deep learning can segment and quantify various cerebral small vessel disease lesions to some extent.

Spatial and Modal Optimal Transport for Fast Cross-Modal MRI Reconstruction.

Multi-modal magnetic resonance imaging (MRI) plays a crucial role in comprehensive disease diagnosis in clinical medicine. However, acquiring certain modalities, such as T2-weighted images (T2WIs), is time-consuming and prone to be with motion artifacts. It negatively impacts subsequent multi-modal image analysis. To address this issue, we propose an end-to-end deep learning framework that utilizes T1-weighted images (T1WIs) as auxiliary modalities to expedite T2WIs' acquisitions. While image pre-processing is capable of mitigating misalignment, improper parameter selection leads to adverse pre-processing effects, requiring iterative experimentation and adjustment. To overcome this shortage, we employ Optimal Transport (OT) to synthesize T2WIs by aligning T1WIs and performing cross-modal synthesis, effectively mitigating spatial misalignment effects. Furthermore, we adopt an alternating iteration framework between the reconstruction task and the cross-modal synthesis task to optimize the final results. Then, we prove that the reconstructed T2WIs and the synthetic T2WIs become closer on the T2 image manifold with iterations increasing, and further illustrate that the improved reconstruction result enhances the synthesis process, whereas the enhanced synthesis result improves the reconstruction process. Finally, experimental results from FastMRI and internal datasets confirm the effectiveness of our method, demonstrating significant improvements in image reconstruction quality even at low sampling rates.

A multimodal MRI study of XNKQ acupuncture for limb dysfunction after ischemic stroke: a randomized controlled study protocol.

Limb motor dysfunction is one of the challenges in rehabilitation after cerebral ischemic stroke (CIS) and greatly affects the quality of life of patients. This study aims to investigate the central mechanisms of the curative effect with multimodal magnetic resonance imaging (MRI), which will provide additional evidence to support the application of Xingnao Kaiqiao (XNKQ) acupuncture. This trial is a randomized controlled trial. Patients who meet the criteria will be recruited and randomly divided into 2 groups. One group will receive acupuncture treatment and another group will not receive acupuncture treatment. Both groups will receive conventional treatment. In addition, 20 healthy individuals will be recruited who will not receive any treatment. The total course of treatment is 14 days. The primary outcome is multimodal MRI analysis. For safety assessment, adverse events will be observed and recorded. The study involving human subjects was reviewed and approved by the Ethics Committee of IRB of The First Teaching Hospital of Tianjin University of TCM (TYLL2023[K]031). This study complied with the Declaration of Helsinki. Written informed consent about this study was provided by the participants. The results of this study will be published in a peer-reviewed journal. Chinese Clinical Trial Registration Center (ChiCTR2300078315) https://www.chictr.org.cn/.

Multimodal Transformer of Incomplete MRI Data for Brain Tumor Segmentation.

Accurate segmentation of brain tumors plays an important role for clinical diagnosis and treatment. Multimodal magnetic resonance imaging (MRI) can provide rich and complementary information for accurate brain tumor segmentation. However, some modalities may be absent in clinical practice. It is still challenging to integrate the incomplete multimodal MRI data for accurate segmentation of brain tumors. In this paper, we propose a brain tumor segmentation method based on multimodal transformer network with incomplete multimodal MRI data. The network is based on U-Net architecture consisting of modality specific encoders, multimodal transformer and multimodal shared-weight decoder. First, a convolutional encoder is built to extract the specific features of each modality. Then, a multimodal transformer is proposed to model the correlations of multimodal features and learn the features of missing modalities. Finally, a multimodal shared-weight decoder is proposed to progressively aggregate the multimodal and multi-level features with spatial and channel self-attention modules for brain tumor segmentation. A missing-full complementary learning strategy is used to explore the latent correlation between the missing and full modalities for feature compensation. For evaluation, our method is tested on the multimodal MRI data from BraTS 2018, BraTS 2019 and BraTS 2020 datasets. The extensive results demonstrate that our method outperforms the state-of-the-art methods for brain tumor segmentation on most subsets of missing modalities.

Deep learning‐based survival prediction of brain tumor patients using attention‐guided 3D convolutional neural network with radiomics approach from multimodality magnetic resonance imaging

AbstractAutomatic survival prediction of gliomas from brain magnetic resonance imaging (MRI) volumes is an essential step for a patient's prognosis analysis. Radiomics research delivers beneficial feature information from MRI imaging which is substantially required by clinicians and oncologists for predicting disease prognosis for precise surgical treatment and planning. In recent years, the success of deep learning has been vast in the field of medical imaging, and it shows state‐of‐the‐art performance in applications like segmentation, classification, regression, and detection. Therefore, in this paper, we proposed a collective method using deep learning and radiomics techniques for the survival prediction of brain tumor patients. We first propose a hierarchical channel attention (HAM) module and a multi‐scale‐aware feature enhancement (MSAFE) to efficiently fuse adjacent hierarchical features in the proposed segmentation model. After segmentation, deep/latent features (LCNN) are extracted from the bottom layer of the proposed segmentation model. Later, we extracted selected radiomics features (histogram, location, and shape) using input images and segmented masks from the proposed segmentation model. Further, the 3D deep learning regressor has been trained for 3D regressor‐based deep feature extraction. We proposed the method of overall survival prediction for the brain tumor patients by combining all the meaningful features including clinical features (age) that also favorably contribute to the survival days prediction for the glioma's patients. To predict the survival days for each patient, the selected features are trained to analyze the performance of various regression techniques like random forest (RF), decision tree (DT), and XGBoost. Our proposed combined feature‐based method achieved the highest performance for survival days prediction over the state‐of‐the‐art methods. We also perform extensive experiments to show the effectiveness of each feature extraction method. The experimental results infer that deep learning‐based features along with radiomic features and clinical features are truly vital paradigms to estimate survival days.

Whole-brain parcellation for macaque brain magnetic resonance images based on attention mechanism and multi-modality feature fusion

We propose a novel deep learning algorithm based on attention mechanism and multimodality feature fusion (DDAM) to achieve whole-brain parcellation of macaque brain magnetic resonance (MR) images. We collected multimodality brain MR images of 68 macaques (aged 13 to 36 months) with corresponding ground truth labels. To address the complexity and complementary nature of the multimodality data, we employed a multi- encoder structure to adapt to different modalities and performed feature extraction. In the decoder, an attention mechanism was introduced to construct the Attention-based Multimodality Feature Fusion module (AMFF). Leveraging the rich and complementary information between modalities, AMFF effectively integrated multimodality features of varying scales and complexities to enhance the performance of parcellation. We conducted ablation experiments and statistically analyzed the results. The incorporation of the multi- encoder structure and attention mechanism significantly improved the performance of the network for integrating the multimodality features, and achieved an average DSC of 0.904 and an ASD as low as 0.131 for macaque whole-brain parcellation (P < 0.05). The ablation experiments validated the effectiveness of each component of the DDAM method. The proposed DDAM method, with its enhanced ability to extract and integrate multimodality features, can significantly improve the accuracy of whole-brain MR image segmentation.

Leveraging multimodal MRI-based radiomics analysis with diverse machine learning models to evaluate lymphovascular invasion in clinically node-negative breast cancer

ObjectiveThis study aimed to investigate and validate the effectiveness of diverse radiomics models for preoperatively differentiating lymphovascular invasion (LVI) in clinically node-negative breast cancer (BC). MethodsThis study included 198 patients diagnosed with clinically node-negative bc and pathologically confirmed LVI status from January 2018–July 2023. The training dataset consisted of 138 patients, while the validation dataset included 60. Radiomics features were extracted from multimodal magnetic resonance imaging obtained from T1WI, T2WI, DCE, DWI, and ADC sequences. Dimensionality reduction and feature selection techniques were applied to the extracted features. Subsequently, machine learning approaches, including logistic regression, support vector machine, classification and regression trees, k-nearest neighbors, and gradient boosting machine models (GBM), were constructed using the radiomics features. The best-performing radiomic model was selected based on its performance using the confusion matrix. Univariate and multivariable logistic regression analyses were conducted to identify variables for developing a clinical-radiological (Clin-Rad) model. Finally, a combined model incorporating both radiomics and clinical-radiological model features was created. ResultsA total of 6195 radiomic features were extracted from multimodal magnetic resonance imaging. After applying dimensionality reduction and feature selection, seven valuable radiomics features were identified. Among the radiomics models, the GBM model demonstrated superior predictive efficiency and robustness, achieving area under the curve values (AUC) of 0.881 (0.823,0.940) and 0.820 (0.693,0.947) in the training and validation datasets, respectively. The Clin-Rad model was developed based on the peritumoral edema and DWI rim sign. In the training dataset, it achieved an AUC of 0.767 (0.681, 0.854), while in the validation dataset, it achieved an AUC of 0.734 (0.555–0.913). The combined model, which incorporated radiomics and the Clin-Rad model, showed the highest discriminatory capability. In the training dataset, it had an AUC value of 0.936 (0.892, 0.981), and in the validation dataset, it had an AUC value of 0.876 (0.757, 0.995). Additionally, decision curve analysis of the combined model revealed its optimal clinical efficacy. ConclusionThe combined model, integrating radiomics and clinical-radiological features, exhibited excellent performance in distinguishing LVI status. This non-invasive and efficient approach holds promise for aiding clinical decision-making in the context of clinically node-negative BC.

Connectome-wide structure-function coupling models implicate polysynaptic alterations in autism

Autism spectrum disorder (ASD) is one of the most common neurodevelopmental diagnoses. Although incompletely understood, structural and functional network alterations are increasingly recognized to be at the core of the condition. We utilized multimodal imaging and connectivity modeling to study structure-function coupling in ASD and probed mono- and polysynaptic mechanisms on structurally-governed network function. We examined multimodal magnetic resonance imaging data in 80 ASD and 61 neurotypical controls from the Autism Brain Imaging Data Exchange (ABIDE) II initiative. We predicted intrinsic functional connectivity from structural connectivity data in each participant using a Riemannian optimization procedure that varies the times that simulated signals can unfold along tractography-derived personalized connectomes. In both ASD and neurotypical controls, we observed improved structure-function prediction at longer diffusion time scales, indicating better modeling of brain function when polysynaptic mechanisms are accounted for. Prediction accuracy differences (∆prediction accuracy) were marked in transmodal association systems, such as the default mode network, in both neurotypical controls and ASD. Differences were, however, lower in ASD in a polysynaptic regime at higher simulated diffusion times. We compared regional differences in ∆prediction accuracy between both groups to assess the impact of polysynaptic communication on structure-function coupling. This analysis revealed that between-group differences in ∆prediction accuracy followed a sensory-to-transmodal cortical hierarchy, with an increased gap between controls and ASD in transmodal compared to sensory/motor systems. Multivariate associative techniques revealed that structure-function differences reflected inter-individual differences in autistic symptoms and verbal as well as non-verbal intelligence. Our network modeling approach sheds light on atypical structure-function coupling in autism, and suggests that polysynaptic network mechanisms are implicated in the condition and that these can help explain its wide range of associated symptoms.

Investigated Parkinson's disease with multimodal magnetic resonance imaging

Investigated Parkinson's disease with multimodal magnetic resonance imaging

An Intra- and Inter-Modality Fusion Model with Invariant- and Specific-Constraints Using MR Images for Prediction of Glioma Isocitrate Dehydrogenase Mutation Status

In the 2021 World Health Organization classification of gliomas, it is proposed that Isocitrate Dehydrogenase (IDH) plays a key role. The prognosis of glioma is largely affected by IDH mutation status. Therefore, IDH mutation status needs to be predicted in advance before surgery. In the past decade, with the development of machine learning, more and more machine learning methods, especially deep learning methods, have been applied to the development of computer-aided diagnosis systems. At present, in this field, many deep learning and radiomics based methods have been proposed for IDH prediction using multimodal Magnetic Resonance Imaging (MRI). In this study, we proposed an intra- and inter-modality fusion model with invariant- and specific- constraints to improve the performance of IDH status prediction. First, MRI-based radiomics features were fused with deep learning features in each modality (intra-modality fusion) and then the features extracted from each modality of brain MRI were fused by using an inter-modality fusion model with invariant and specific constraints. We experimented our proposed method on the dataset provided by the Affiliated Hospital of Zhengzhou University in Zhengzhou, China and demonstrated the effectiveness of the proposed method. In our study, we propose two inter-modality fusion models, and our experimental results show that our best proposed method outperformed state-of-the-art methods with an accuracy of 0.79, precision of 0.80, recall of 0.75, and F1 score of 0.78. Thus, we predicted the IDH mutation status for glioma treatment with a 2% increase in accuracy and 4% increase in precision to predict the IDH mutation status for glioma treatment.

A Multi-Modality Magnetic Resonance Imaging Model for Predicting Traumatic Brain Injury Outcomes

Traumatic Brain Injury (TBI) is a heterogenous injury and a leading cause of long-term deficits and mortality in the United States. In order to improve TBI outcomes, an effective prognostication tool is necessary. Standard imaging modalities, computerized tomography (CT) and magnetic resonance imaging (MRI), have a limited ability to predict TBI outcomes. Currently, advanced MRI techniques are being studied for their efficacy. The aim of this study is to determine whether a multimodality MRI approach is superior to a single modality MRI approach in determining clinical outcomes of TBI. A secondary data analysis was conducted on TBI data obtained from 31 rat brains; 3-day MRI data in the Ipsilateral Perilesion Cortex and 28-day Behavioral Test data (Novel Object Recognition, Barnes Maze, and Open Field Test Total Distance and Total Act Time) were analyzed. A Best Subset Analysis was conducted for each of the behavioral tests. Three out of four behavioral tests show improved adjusted R2 values for models containing more than one imaging modality. A Multiple Linear Regression Analysis was then conducted on the MRIs from the highest predictive model determined by Best Subset Analysis. This analysis shows that a multimodality MRI approach can explain 25.2% of the variability in behavioral outcomes in the Novel Object Recognition Test with a P value of 0.012. Thus, the study demonstrates that a multi-modality MRI approach has a potential for effectively diagnosing and predicting TBI outcomes.

Characterizing Typhoon-related Posttraumatic Stress Disorder Based on Multimodal Fusion of Structural, Diffusion, and Functional Magnetic Resonance Imaging

Diagnosing posttraumatic stress disorder (PTSD) using only single-modality images is controversial. We aimed to use multimodal magnetic resonance imaging (MRI) combining structural, diffusion, and functional MRI to possibly provide a more comprehensive viewpoint on the decisive characteristics of PTSD patients. Typhoon-exposed individuals with (n = 26) and without PTSD (n = 32) and healthy volunteers (n = 30) were enrolled. Five MRI features from three modalities, including two resting-state functional MRI (rs-fMRI) features (amplitude of low-frequency fluctuation, ALFF; and regional homogeneity, ReHo), one structural MRI feature (gray matter density, GM), and two diffusion tensor imaging (DTI) features (fractional anisotropy, FA; and mean diffusivity, MD) were investigated simultaneously with a multimodal canonical correlation analysis + joint independent component analysis model to identify abnormalities in the PTSD brain. We identified statistical differences between PTSD patients and healthy controls in terms of 1 rs-fMRI (ALFF, ReHo) alterations in the superior frontal gyrus, precuneus, inferior parietal lobule (IPL), anterior cingulate cortex (ACC), and posterior cingulate cortex (PCC), 2 DTI (FA, MD) changes in the pons, genu, and splenium of the corpus callosum, and 3 Structural MRI abnormalities in the precuneus, IPL, ACC, and PCC. A novel ReHo component was found to distinguish PTSD and trauma-exposed controls, including the precuneus, IPL, middle frontal gyrus, middle occipital gyrus, and cerebellum. This study reveals that PTSD individuals exhibit intertwined functional and structural anomalies within the default mode network. Some alterations within this network may serve as a potential marker to distinguish between PTSD patients and trauma-exposed controls.

Investigation of an efficient multi-modal convolutional neural network for multiple sclerosis lesion detection

In this study, an automated 2D machine learning approach for fast and precise segmentation of MS lesions from multi-modal magnetic resonance images (mmMRI) is presented. The method is based on an U-Net like convolutional neural network (CNN) for automated 2D slice-based-segmentation of brain MRI volumes. The individual modalities are encoded in separate downsampling branches without weight sharing, to leverage the specific features. Skip connections input feature maps to multi-scale feature fusion (MSFF) blocks at every decoder stage of the network. Those are followed by multi-scale feature upsampling (MSFU) blocks which use the information about lesion shape and location. The CNN is evaluated on two publicly available datasets: The ISBI 2015 longitudinal MS lesion segmentation challenge dataset containing 19 subjects and the MICCAI 2016 MSSEG challenge dataset containing 15 subjects from various scanners. The proposed multi-input 2D architecture is among the top performing approaches in the ISBI challenge, to which open-access papers are available, is able to outperform state-of-the-art 3D approaches without additional post-processing, can be adapted to other scanners quickly, is robust against scanner variability and can be deployed for inference even on a standard laptop without a dedicated GPU.

Periodontitis is associated with altered brain structure and function in normal cognition middle-aged and elderly individuals.

Numerous studies have proposed that periodontitis is a potential risk factor for Alzheimer's disease. However, the association between periodontitis and brain normal cognition in aged and elderly individuals (NCs) is unclear. Such a link could provide clues to Alzheimer's disease development and strategies for early prevention. To explore the associations between periodontal condition and metrics of both brain structure and function among NCs with the help of multimodal magnetic resonance imaging (MRI). High-resolution T1-weighted structural data, resting-state functional-MRI data, and measures of periodontal condition were collected from 40 NCs. Cortical volume, thickness, and area as well as regional homogeneity were calculated with the aid of DPABISurf software. Correlation analyses were then conducted between each imaging metric and periodontal index. Consistent negative correlations were observed between severity of periodontitis (mild, moderate, severe) and cortical volume, area, and thickness, not only in brain regions that took charge of primary function but also in brain regions associated with advanced cognition behavior. Among participants with mild attachment loss (AL) and a shallow periodontal pocket depth (PPD), periodontal index was positively correlated with most measures of brain structure and function, while among participants with severe AL and deep PPD, periodontal index was negatively correlated with measures of brain structure and function (all p < .005 for each hemisphere). Our results demonstrate that periodontitis is associated with widespread changes in brain structure and function among middle-aged and elderly adults without signs of cognitive decline, which might be a potential risk factor for brain damage.

Microstructural and functional alterations of the ventral pallidum are associated with levodopa-induced dyskinesia in Parkinson's disease.

The ventral pallidum (VP) regulates involuntary movements, but it is unclear whether the VP regulates the abnormal involuntary movements in Parkinson's disease (PD) patients who have levodopa-induced dyskinesia (LID). To further understand the role of the VP in PD patients with LID (PD-LID), we explored the structural and functional characteristics of the VP in such patients using multimodal magnetic resonance imaging (MRI). Thirty-one PD-LID patients, 39 PD patients without LID (PD-nLID), and 28 healthy controls (HCs) underwent T1-weighted MRI, quantitative susceptibility mapping, multi-shell diffusion MRI, and resting-state functional MRI (rs-fMRI). Different measures characterizing the VP were obtained using a region-of-interest-based approach. The left VP in the PD-LID group showed significantly higher intracellular volume fraction (ICVF) and isotropic volume fraction (IsoVF) compared with the PD-nLID and HC groups. Rs-MRI revealed that, compared with the PD-nLID group, the PD-LID group in the medication 'off' state had higher functional connectivity (FC) between the left VP and the left anterior caudate, left middle frontal gyrus and left precentral gyrus, as well as between the right VP and the right posterior ventral putamen and right mediodorsal thalamus. In addition, the ICVF values of the left VP, the FC between the left VP and the left anterior caudate and left middle frontal gyrus were positively correlated with Unified Dyskinesia Rating Scale scores. Our multimodal imaging findings show that the microstructural changes of the VP (i.e., the higher ICVF and IsoVF) and the functional change in the ventral striatum-VP-mediodorsal thalamus-cortex network may be associated with pathophysiological mechanisms of PD-LID.

Multimodal magnetic resonance imaging on brain structure and function changes in vascular cognitive impairment without dementia.

Vascular cognitive impairment not dementia (VCIND) is one of the three subtypes of vascular cognitive impairment (VCI), with cognitive dysfunction and symptoms ranging between normal cognitive function and vascular dementia. The specific mechanisms underlying VCIND are still not fully understood, and there is a lack of specific diagnostic markers in clinical practice. With the rapid development of magnetic resonance imaging (MRI) technology, structural MRI (sMRI) and functional MRI (fMRI) have become effective methods for exploring the neurobiological mechanisms of VCIND and have made continuous progress. This article provides a comprehensive overview of the research progress in VCIND using multimodal MRI, including sMRI, diffusion tensor imaging, resting-state fMRI, and magnetic resonance spectroscopy. By integrating findings from these multiple modalities, this study presents a novel perspective on the neuropathological mechanisms underlying VCIND. It not only highlights the importance of multimodal MRI in unraveling the complex nature of VCIND but also lays the foundation for future research examining the relationship between brain structure, function, and cognitive impairment in VCIND. These new perspectives and strategies ultimately hold the potential to contribute to the development of more effective diagnostic tools and therapeutic interventions for VCIND.

An attention-based multi-modal MRI fusion model for major depressive disorder diagnosis

Objective. Major depressive disorder (MDD) is one of the biggest threats to human mental health. MDD is characterized by aberrant changes in both structure and function of the brain. Although recent studies have developed some deep learning models based on multi-modal magnetic resonance imaging (MRI) for MDD diagnosis, the latent associations between deep features derived from different modalities were largely unexplored by previous studies, which we hypothesized may have potential benefits in improving the diagnostic accuracy of MDD. Approach. In this study, we proposed a novel deep learning model that fused both structural MRI (sMRI) and resting-state MRI (rs-fMRI) data to enhance the diagnosis of MDD by capturing the interactions between deep features extracted from different modalities. Specifically, we first employed a brain function encoder (BFE) and a brain structure encoder (BSE) to extract the deep features from fMRI and sMRI, respectively. Then, we designed a function and structure co-attention fusion (FSCF) module that captured inter-modal interactions and adaptively fused multi-modal deep features for MDD diagnosis. Main results. This model was evaluated on a large cohort and achieved a high classification accuracy of 75.2% for MDD diagnosis. Moreover, the attention distribution of the FSCF module assigned higher attention weights to structural features than functional features for diagnosing MDD. Significance. The high classification accuracy highlights the effectiveness and potential clinical of the proposed model.

PU-NET DEEP LEARNING ARCHITECTURE FOR GLIOMAS BRAIN TUMOUR SEGMENTATION IN MAGNETIC RESONANCE IMAGES

Automatic medical image segmentation is one of the main tasks for many organs and pathology structure delineation. It is also a crucial technique in the posterior clinical examination of brain tumours, like applying radiotherapy or tumour restrictions. Various image segmentation techniques have been proposed and applied to different image types of images. Recently, it has been shown that the deep learning approach accurately segments images, and its implementation is usually straightforward. In this paper, we proposed a novel approach, called PU-net, for automatic brain tumour segmentation in multi-modal magnetic resonance images (MRI) based on deep learning. We introduced an input processing block to a customised fully convolutional network derived from the U-Net network to handle the multi-modal inputs. We performed experiments over the Brats brain tumour dataset collected in 2018 and achieved dice scores of 0.905,0.827, and 0.803 for the whole tumour (WT), tumour core (TC), and enhancing tumour (ET) classes, respectively. This study also provides promising results compared to the traditional machine learning methods, such as support vector machines (SVM), random forest (RF) and other deep learning methods used in this context.