Multimodal Magnetic Resonance Research Articles

Radiomics Based on Multimodal magnetic resonance imaging for the Differential Diagnosis of Benign and Malignant Vertebral Compression Fractures.

Recent studies have indicated that radiomics may have excellent performance and clinical application prospects in the differential diagnosis of benign and malignant vertebral compression fractures (VCFs). However, multimodal magnetic resonance imaging (MRI)-based radiomics model is rarely used in the differential diagnosis of benign and malignant VCFs, and is limited to lumbar. Herein, this study intends to develop and validate MRI radiomics models for differential diagnoses of benign and malignant VCFs in patients. This cross-sectional study involved 151 adult patients diagnosed with VCF in The First Affiliated Hospital of Soochow University in 2016-2021. The study was conducted in three steps: (i) the original MRI images were segmented, and the region of interest (ROI) was marked out; (ii) among the extracted features, those features with Pearson's correlation coefficient lower than 0.9 and the top 15 with the highest variance and Lasso regression coefficient less than and more than 0 were selected; (iii) MRI images and combined data were studied by logistic regression, decision tree, random forest and extreme gradient boosting (XGBoost) models in training set and the test set (ratio of 8:2), respectively; and the models were further verified and evaluated for the differential diagnosis performance. The evaluated indexes included area under receiver (AUC) of operating characteristic curve, accuracy, sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV), and 95% confidence intervals (CIs). The AUCs were used to assess the predictive performance of different machine learning modes for benign and malignant VCFs. A total of 1144 radiomics features, and 14 clinical features were extracted. Finally, 12 radiomics features were included in the radiomics model, and 12 radiomics features with 14 clinical features were included in the combined model. In the radiomics model, the differential diagnosis performance in the logistic regression model with the AUC of 0.905 ± 0.026, accuracy of 0.817 ± 0.057, sensitivity of 0.831 ± 0.065, and negative predictive value of 0.813 ± 0.042, was superior to the other three. In the combined model, XGBoost model had the superior differential diagnosis performance with specificity (0.979 ± 0.026) and positive predictive value (0.971 ± 0.035). The multimodal MRI-based radiomics model performed well in the differential diagnosis of benign and malignant VCFs, which may provide a tool for clinicians to differentially diagnose VCFs.

Unified Multi-Modal Image Synthesis for Missing Modality Imputation.

Multi-modal medical images provide complementary soft-tissue characteristics that aid in the screening and diagnosis of diseases. However, limited scanning time, image corruption and various imaging protocols often result in incomplete multi-modal images, thus limiting the usage of multi-modal data for clinical purposes. To address this issue, in this paper, we propose a novel unified multi-modal image synthesis method for missing modality imputation. Our method overall takes a generative adversarial architecture, which aims to synthesize missing modalities from any combination of available ones with a single model. To this end, we specifically design a Commonality- and Discrepancy-Sensitive Encoder for the generator to exploit both modality-invariant and specific information contained in input modalities. The incorporation of both types of information facilitates the generation of images with consistent anatomy and realistic details of the desired distribution. Besides, we propose a Dynamic Feature Unification Module to integrate information from a varying number of available modalities, which enables the network to be robust to random missing modalities. The module performs both hard integration and soft integration, ensuring the effectiveness of feature combination while avoiding information loss. Verified on two public multi-modal magnetic resonance datasets, the proposed method is effective in handling various synthesis tasks and shows superior performance compared to previous methods.

Linking functional and structural brain organisation with behaviour in healthy adults.

Multimodal data integration approaches, such as Linked Independent Component Analysis (LICA), increase sensitivity to brain-behaviour relationships and allow us to probe the relationship between modalities. Here we focus on inter-regional functional and structural organisation to determine if organisational patterns persist across modalities and if investigating multi-modality organisations provides increased sensitivity to brain-behaviour associations. We utilised multimodal magnetic resonance imaging (MRI; T1w, resting-state functional [fMRI] and diffusion weighted [DWI]) and behavioural data from the Human Connectome Project (HCP, n=676; 51% female). Unimodal features were extracted to produce individual grey matter density maps, probabilistic tractography connectivity matrices and connectopic maps from the T1w, DWI and fMRI data, respectively. DWI and fMRI analyses were restricted to subcortical regions for computational reasons. LICA was then used to integrate features, generating 100 novel independent components. Associations between these components and demographic/behavioural (n=308) variables were examined. 15 components were significantly associated with various demographic/behavioural measures. 2 components were strongly related to various measures of intoxication, driven by DWI and resemble components previously identified. Another component was driven by striatal functional data and related to working memory. A small number of components showed shared variance between structure and function but none of these displayed any significant behavioural associations. Our working memory findings provide support for the use of fMRI connectopic mapping in future research of working memory. Given the lack of behaviourally relevant shared variance between functional and structural organisation, as indexed here, we question the utility of integrating connectopic maps and tractography data.

SIMFusion: A semantic information-guided modality-specific fusion network for MR Images

Multi-modal medical image fusion aims to integrate distinct imaging modalities to yield more comprehensive and precise medical images, which can benefit the subsequent image analysis tasks. However, prevailing state-of-the-art image fusion methods, despite substantial advancements, do not explicitly address the efficient handling of complementary information between modalities. Moreover, most current multi-modal medical image fusion methods encounter challenges when integrating with practical tasks, which lack the guidance of semantic information, thus hindering the generation of high-quality images for accurate identification of lesion areas. To deal with these challenges, this paper introduces a novel semantic information-guided modality-specific fusion network for multi-modal magnetic resonance (MR) images, named SIMFusion. Specifically, we propose the decomposition branch to capture common and specific features from MR images of different modalities for fusion to reduce information redundancy through correlated mutual information loss. Subsequently, we obtain their semantic characteristics via the semantic branch utilizing a pre-train segmentation network and, finally, achieve an adaptive balance between the two sets of features through a specialized fusion strategy. Extensive experiments demonstrate the superiority of SIMFusion over existing competing techniques on both BraTS2019 and ISLES2022 datasets, with 8.1% improvement in MI and 38.7 % in VIFF in T2-T1ce image pairs, highlighting the potential of the proposed method as a promising solution for MR image fusion in practical applications.

The differential effects of integrase strand transfer inhibitors and efavirenz on neuropsychiatric conditions and brain imaging in HIV-positive men who have sex with men

Integrase strand transfer inhibitors (INSTIs) have emerged as the first-line choice for treating human immunodeficiency virus (HIV) infection due to their superior efficacy and safety. However, the impact of INSTIs on the development of neuropsychiatric conditions in people living with HIV (PLWH) is not fully understood due to limited data. In this study, we conducted a cross-sectional examination of PLWH receiving antiretroviral therapy, with a specific focus on HIV-positive men who have sex with men (MSM) on INSTI-based regimens (n = 61) and efavirenz (EFV)-based regimens (n = 28). Participants underwent comprehensive neuropsychiatric evaluations and multimodal magnetic resonance imaging (MRI) scans, including T1-weighted images and resting-state functional MRI. Compared to the EFV group, the INSTI group exhibited primarily reduced gray matter volume (GMV) in the right superior parietal gyrus, higher regional homogeneity (ReHo) in the left postcentral gyrus, lower ReHo in the right orbital part of the inferior frontal gyrus, and increased voxel-wise functional connectivity for the seed region in the left inferior temporal gyrus with clusters in the right cuneus. Furthermore, the analysis revealed a main effect of antiretroviral drugs on GMV changes, but no main effect of neuropsychiatric disorders or their interaction. The repeated analysis of participants who did not switch regimens confirmed the GMV changes in the INSTI group, validating the initial findings. Our study demonstrated gray matter atrophy and functional brain changes in PLWH on INSTI-based regimens compared to those on EFV-based regimens. These neuroimaging results provide valuable insights into the characteristics of brain network modifications in PLWH receiving INSTI-based regimens.

Multimodal imaging evaluation of early neurological deterioration following acute ischemic stroke.

Early neurologic deterioration occurs in up to one-third of patients with acute ischemic stroke (IS), often leading to poor functional outcomes. At present, few studies have applied amide proton transfer (APT) imaging to the evaluation of early neurological deterioration (END). This study analyzed the value of computed tomography perfusion (CTP) combined with multimodal magnetic resonance imaging (MRI) in patients with acute IS with END. This retrospective study included patients with acute IS who were admitted to the neurology inpatient department in a tertiary hospital from October 2021 to June 2023. Patients with acute IS underwent CTP within 24 hours of stroke onset and MRI [arterial spin labeling (ASL), susceptibility-weighted imaging (SWI), and APT] within 7 days. END was defined as an elevation of ≥2 points on the National Institute of Health Stroke Scale (NIHSS) within 7 days of stroke onset. Univariable and multivariable analyses were used to compare clinical and imaging biomarkers in patients with acute IS with and without END. The performance of potential biomarkers in distinguishing between the two groups was evaluated using receiver operating characteristic (ROC) curve analysis. Among the 70 patients with acute IS, 20 (29%) had END. After conducting univariable analysis, variables were selected for entry into a binary logistic regression analysis based on our univariable analysis results, previous research findings, clinical experience, and methodological standards. The results indicated that relative cerebral blood volume (CBV) on CTP, relative cerebral blood flow (CBF) on ASL, and relative signal intensity on amide proton transfer-weighted (APTw) imaging were independent risk factors for END. The areas under the ROC curves for these risk factors were 0.710 [95% confidence interval (CI): 0.559-0.861, P=0.006], 0.839 (95% CI: 0.744-0.933, P<0.001), and 0.804 (95% CI: 0.676-0.932, P<0.001), respectively. The combined area under the curve (AUC), sensitivity, and specificity of the four indices (0.941, 100%, and 78%, respectively) were higher than those of the four indices alone. CTP combined with multi-modal MRI better evaluated hemodynamics, tissue metabolism, and other relevant patient information, providing an objective basis for the clinical assessment of patients with acute IS with END and facilitating the development of accurate and personalized treatment plans.

Resting-state neural activity and cerebral blood flow alterations in type 2 diabetes mellitus: Insights from hippocampal subfields.

In this study, multimodal magnetic resonance imaging (MRI) imaging was used to deeply analyze the changes of hippocampal subfields perfusion and function in patients with type 2 diabetes mellitus (T2DM), aiming to provide image basis for the diagnosis of hippocampal-related nerve injury in patients with T2DM. We recruited 35 patients with T2DM and 40 healthy control subjects (HCs). They underwent resting-state functional MRI (rs-fMRI), arterial spin labeling (ASL) scans, and a series of cognitive tests. Then, we compared the differences of two groups in the cerebral blood flow (CBF) value, amplitude of low-frequency fluctuation (ALFF) value, and regional homogeneity (ReHo) value of the bilateral hippocampus subfields. The CBF values of cornu ammonis area 1 (CA1), dentate gyrus (DG), and subiculum in the right hippocampus of T2DM group were significantly lower than those of HCs. The ALFF values of left hippocampal CA3, subiculum, and bilateral hippocampus amygdala transition area (HATA) were higher than those of HCs in T2DM group. The ReHo values of CA3, DG, subiculum, and HATA in the left hippocampus of T2DM group were higher than those of HCs. In the T2DM group, HbAc1 and FINS were negatively correlated with imaging characteristics in some hippocampal subregions. This study indicates that T2DM patients had decreased perfusion in the CA1, DG, and subiculum of the right hippocampus, and the right hippocampus subiculum was associated with chronic hyperglycemia. Additionally, we observed an increase in spontaneous neural activity within the left hippocampal CA3, subiculum, and bilateral HATA regions, as well as an enhanced local neural coordination in the left hippocampal CA3, DG, HATA, and subiculum among patients with type 2 diabetes, which may reflect an adaptive compensation for cognitive decline. However, this compensation may decline with the exacerbation of metabolic disorders.

CMAF-Net: a cross-modal attention fusion-based deep neural network for incomplete multi-modal brain tumor segmentation.

The information between multimodal magnetic resonance imaging (MRI) is complementary. Combining multiple modalities for brain tumor image segmentation can improve segmentation accuracy, which has great significance for disease diagnosis and treatment. However, different degrees of missing modality data often occur in clinical practice, which may lead to serious performance degradation or even failure of brain tumor segmentation methods relying on full-modality sequences to complete the segmentation task. To solve the above problems, this study aimed to design a new deep learning network for incomplete multimodal brain tumor segmentation. We propose a novel cross-modal attention fusion-based deep neural network (CMAF-Net) for incomplete multimodal brain tumor segmentation, which is based on a three-dimensional (3D) U-Net architecture with encoding and decoding structure, a 3D Swin block, and a cross-modal attention fusion (CMAF) block. A convolutional encoder is initially used to extract the specific features from different modalities, and an effective 3D Swin block is constructed to model the long-range dependencies to obtain richer information for brain tumor segmentation. Then, a cross-attention based CMAF module is proposed that can deal with different missing modality situations by fusing features between different modalities to learn the shared representations of the tumor regions. Finally, the fused latent representation is decoded to obtain the final segmentation result. Additionally, channel attention module (CAM) and spatial attention module (SAM) are incorporated into the network to further improve the robustness of the model; the CAM to help focus on important feature channels, and the SAM to learn the importance of different spatial regions. Evaluation experiments on the widely-used BraTS 2018 and BraTS 2020 datasets demonstrated the effectiveness of the proposed CMAF-Net which achieved average Dice scores of 87.9%, 81.8%, and 64.3%, as well as Hausdorff distances of 4.21, 5.35, and 4.02 for whole tumor, tumor core, and enhancing tumor on the BraTS 2020 dataset, respectively, outperforming several state-of-the-art segmentation methods in missing modalities situations. The experimental results show that the proposed CMAF-Net can achieve accurate brain tumor segmentation in the case of missing modalities with promising application potential.

Exploring potential neuroimaging biomarkers for the response to non-steroidal anti-inflammatory drugs in episodic migraine

BackgroundNon-steroidal anti-inflammatory drugs (NSAIDs) are considered first-line medications for acute migraine attacks. However, the response exhibits considerable variability among individuals. Thus, this study aimed to explore a machine learning model based on the percentage of amplitude oscillations (PerAF) and gray matter volume (GMV) to predict the response to NSAIDs in migraine treatment.MethodsPropensity score matching was adopted to match patients having migraine with response and nonresponse to NSAIDs, ensuring consistency in clinical characteristics and migraine-related features. Multimodal magnetic resonance imaging was employed to extract PerAF and GMV, followed by feature selection using the least absolute shrinkage and selection operator regression and recursive feature elimination algorithms. Multiple predictive models were constructed and the final model with the smallest predictive residuals was chosen. The model performance was evaluated using the area under the receiver operating characteristic (ROCAUC) curve, area under the precision-recall curve (PRAUC), balance accuracy (BACC), sensitivity, F1 score, positive predictive value (PPV), and negative predictive value (NPV). External validation was performed using a public database. Then, correlation analysis was performed between the neuroimaging predictors and clinical features in migraine.ResultsOne hundred eighteen patients with migraine (59 responders and 59 non-responders) were enrolled. Six features (PerAF of left insula and left transverse temporal gyrus; and GMV of right superior frontal gyrus, left postcentral gyrus, right postcentral gyrus, and left precuneus) were observed. The random forest model with the lowest predictive residuals was selected and model metrics (ROCAUC, PRAUC, BACC, sensitivity, F1 score, PPV, and NPV) in the training and testing groups were 0.982, 0.983, 0.927, 0.976, 0.930, 0.889, and 0.973; and 0.711, 0.648, 0.639, 0.667,0.649, 0.632, and 0.647, respectively. The model metrics of external validation were 0.631, 0.651, 0.611, 0.808, 0.656, 0.553, and 0.706. Additionally, a significant positive correlation was found between the GMV of the left precuneus and attack time in non-responders.ConclusionsOur findings suggest the potential of multimodal neuroimaging features in predicting the efficacy of NSAIDs in migraine treatment and provide novel insights into the neural mechanisms underlying migraine and its optimized treatment strategy.

Spatial and temporal pattern of structure-function coupling of human brain connectome with development.

Brain structural circuitry shapes a richly patterned functional synchronization, supporting for complex cognitive and behavioural abilities. However, how coupling of structural connectome (SC) and functional connectome (FC) develops and its relationships with cognitive functions and transcriptomic architecture remain unclear. We used multimodal magnetic resonance imaging data from 439 participants aged 5.7-21.9 years to predict functional connectivity by incorporating intracortical and extracortical structural connectivity, characterizing SC-FC coupling. Our findings revealed that SC-FC coupling was strongest in the visual and somatomotor networks, consistent with evolutionary expansion, myelin content, and functional principal gradient. As development progressed, SC-FC coupling exhibited heterogeneous alterations dominated by an increase in cortical regions, broadly distributed across the somatomotor, frontoparietal, dorsal attention, and default mode networks. Moreover, we discovered that SC-FC coupling significantly predicted individual variability in general intelligence, mainly influencing frontoparietal and default mode networks. Finally, our results demonstrated that the heterogeneous development of SC-FC coupling is positively associated with genes in oligodendrocyte-related pathways and negatively associated with astrocyte-related genes. This study offers insight into the maturational principles of SC-FC coupling in typical development.

A novel fully automatic segmentation and counting system for metastatic lymph nodes on multimodal magnetic resonance imaging: Evaluation and prognostic implications in nasopharyngeal carcinoma

BackgroundThe number of metastatic lymph nodes (MLNs) is crucial for the survival of nasopharyngeal carcinoma (NPC), but manual counting is laborious. This study aims to explore the feasibility and prognostic value of automatic MLNs segmentation and counting. MethodsWe retrospectively enrolled 980 newly diagnosed patients in the primary cohort and 224 patients from two external cohorts. We utilized the nnUnet model for automatic MLNs segmentation on multimodal magnetic resonance imaging. MLNs counting methods, including manual delineation-assisted counting (MDAC) and fully automatic lymph node counting system (AMLNC), were compared with manual evaluation (Gold standard). ResultsIn the internal validation group, the MLNs segmentation results showed acceptable agreement with manual delineation, with a mean Dice coefficient of 0.771. The consistency among three counting methods was as follows 0.778 (Gold vs. AMLNC), 0.638 (Gold vs. MDAC), and 0.739 (AMLNC vs. MDAC). MLNs numbers were categorized into three-category variable (1–4, 5–9, > 9) and two-category variable (<4, ≥ 4) based on the gold standard and AMLNC. These categorical variables demonstrated acceptable discriminating abilities for 5-year overall survival (OS), progression-free, and distant metastasis-free survival. Compared with base prediction model, the model incorporating two-category AMLNC-counting numbers showed improved C-indexes for 5-year OS prediction (0.658 vs. 0.675, P = 0.045). All results have been successfully validated in the external cohort. ConclusionsThe AMLNC system offers a time- and labor-saving approach for fully automatic MLNs segmentation and counting in NPC. MLNs counting using AMLNC demonstrated non-inferior performance in survival discrimination compared to manual detection.

Preoperative Grading of Brain Gliomas Using 3D-ResNet18 Based on Multimodal MRI and Attention Mechanism

In response to the prevalent challenge of imprecise preoperative glioma grading prediction, a multimodal Magnetic Resonance Imaging (MRI) image with attention mechanism prediction model based on Residual Network (ResNet) is proposed for accurate prediction. To achieve this, publicly accessible datasets from BraTS 2019 and BraTS 2020 were employed. MRI images across three modalities underwent preprocessing, cropping, and subsequent stacking to generate comprehensive multimodal representations. Meanwhile, An Efficient Channel Attention (ECA) mechanism based on 3D data (3D-ECA) is proposed and introduced to reduce the problem of slow network convergence and overfitting. Notably, experiments demonstrate that the 3D-ECA attention mechanism added to the model improves the network training speed and the classification accuracy of the model, and the classification accuracy reached 88.1%, with an impressive area under the subject working characteristic curve of 0.918. Therefore, the 3D-ResNet18 deep learning model, incorporating multimodal MRI and 3D-ECA attention mechanisms, demonstrates high accuracy and robustness in distinguishing between high-grade and low-grade gliomas. Thus, showcasing significant clinical potential for glioma classification and diagnosis.

Amygdala structural and functional reorganization as an indicator of affective dysfunction in patients with tinnitus.

The aim of this study was to systematically investigate structural and functional alterations in amygdala subregions using multimodal magnetic resonance imaging (MRI) in patients with tinnitus with or without affective dysfunction. Sixty patients with persistent tinnitus and 40 healthy controls (HCs) were recruited. Based on a questionnaire assessment, 26 and 34 patients were categorized into the tinnitus patients with affective dysfunction (TPAD) and tinnitus patients without affective dysfunction (TPWAD) groups, respectively. MRI-based measurements of gray matter volume, fractional anisotropy (FA), fractional amplitude of low-frequency fluctuations (fALFF), regional homogeneity (ReHo), degree centrality (DC), and functional connectivity (FC) were conducted within 14 amygdala subregions for intergroup comparisons. Associations between the MRI properties and clinical characteristics were estimated via partial correlation analyses. Compared with that of the HCs, the TPAD and TPWAD groups exhibited significant structural and functional changes, including white matter integrity (WMI), fALFF, ReHo, DC, and FC alterations, with more pronounced WMI changes in the TPAD group, predominantly within the left auxiliary basal or basomedial nucleus (AB/BM), right central nucleus, right lateral nuclei (dorsal portion), and left lateral nuclei (ventral portion containing basolateral portions). Moreover, the TPAD group exhibited decreased FC between the left AB/BM and left middle occipital gyrus and right superior frontal gyrus (SFG), left basal nucleus and right SFG, and right lateral nuclei (intermediate portion) and right SFG. In combination, these amygdalar alterations exhibited a sensitivity of 65.4% and specificity of 96.9% in predicting affective dysfunction in patients with tinnitus. Although similar structural and functional amygdala remodeling were observed in the TPAD and TPWAD groups, the changes were more pronounced in the TPAD group. These changes mainly involved alterations in functionality and white matter microstructure in various amygdala subregions; in combination, these changes could serve as an imaging-based predictor of emotional disorders in patients with tinnitus.

#2484 Diffusion magnetic resonance imaging, a noninvasive tool to assess fibrosis level in both native and allograft kidneys

Abstract Background and Aims Renal cortical interstitial fibrosis is a well-established prognostic indicator for renal health, conventionally estimated by biopsy. Our prior research has enhanced DW-MRI sequencing, facilitating precise distinction between the renal cortex and medulla. This has enabled the measurement of the cortico-medullary apparent diffusion coefficient difference (ΔADC), which inversely correlates with the extent of histologically assessed interstitial fibrosis. The present study is designed to validate these findings, assess their reproducibility in a separate institution, and ascertain their significance in delineated subgroups, particularly between allograft recipients and native kidney patients. Method From 2014 to 2023, we orchestrated a prospective cohort study encompassing 309 patients diagnosed with chronic kidney disease (n = 106) or those with a renal allograft (n = 203). Each patient had undergone a biopsy, subsequent to which a multimodal magnetic resonance imaging (MRI) was systematically conducted in the following weeks. Initially, participants were recruited from Geneva hospitals, with subsequent inclusion of patients from Lausanne in Switzerland. The study's objective was to evaluate the correlation between ΔADC and the degree of renal fibrosis at biopsy. We performed subgroup analyses (per center, native/allograft kidneys) and applied linear regression models to investigate these associations rigorously. Results We included 270 participants in Geneva and 39 patients in Lausanne. A significant negative correlation was observed between ΔADC and the degree of fibrosis across the entire cohort (R = −0.41, p &amp;lt; 0.001). Subgroup analyses yielded consistent findings in both Geneva (R = −0.43, p &amp;lt; 0.001) and Lausanne University Hospitals (R = −0.35, p &amp;lt; 0.001). Furthermore, this association remained significant within subgroups of both native and allograft kidneys (respectively: R = −0.48, p &amp;lt; 0.001 and R = −0.38, p &amp;lt; 0.001). Conclusion Our findings reinforce the link between ΔADC and renal fibrosis, indicating that this association is consistent regardless of the center or kidney type (native or allograft). This supports the potential utility of diffusion MRI as a non-invasive tool for the assessment of renal fibrosis.

Misalignment-Resistant Deep Unfolding Network for multi-modal MRI super-resolution and reconstruction

Multi-modal Magnetic Resonance Imaging (MRI) super-resolution (SR) and reconstruction aims to obtain a high-quality target image from corresponding sparsely sampled signals under the guidance of a reference image. However, existing techniques typically assume that the input multi-modal MR images are well aligned, which is challenging to achieve in clinical practice. This naive assumption has made their algorithms vulnerable to misalignment scenarios. Moreover, they often neglect many non-local common characteristics within and between modalities. In this work, we proposed a MisAlignment-Resistant Deep Unfolding Network (MAR-DUN) embedded in the tailored gradient descent module (GDM) and proximal mapping module (PMM) for multi-modal MRI SR and reconstruction. In the GDM, we employ an adaptive step-size sub-network (ASS-Net) to enhance the texture representation capacity of the proposed MAR-DUN. Furthermore, in the PMM, we propose a cross-modality non-local module (CNLM) featuring the inverse deformation layer (IDL). The IDL aligns features between the target and reference images by adaptively learning their spatial transformations, thus enhancing the robustness of the proposed network and allowing the CNLM to further explore the cross-modality non-local characteristics. On the other hand, the proposed CNLM aims to establish both the intra-modality and inter-modality non-local dependencies for fully exploiting the correlations between the target and reference images. Extensive experimental results show that our proposed method consistently achieves state-of-the-art reconstruction performance in alignment and misalignment scenarios, demonstrating its significant promise for real-world applications.

Multi-teacher cross-modal distillation with cooperative deep supervision fusion learning for unimodal segmentation

Accurate brain tumor segmentation is a labor-intensive and time-consuming task that requires automation to enhance its efficacy. Recent advanced techniques have shown promising results in segmenting brain tumors; however, their dependency on extensive multimodal magnetic resonance imaging (MRI) data limits their practicality in clinical environments where such data may not be readily available. To address this, we propose a novel multi-teacher cross-modal knowledge distillation framework, which utilizes the privileged multimodal data during training while relying solely on unimodal data for inference. Our framework is tailored to the unimodal segmentation of the T1ce MRI sequence, which is prevalently available in clinical practice and structurally akin to the T1 modality, providing ample information for the segmentation task. Our framework introduces two learning strategies for knowledge distillation (KD): (1) performance-aware response-based KD and (2) cooperative deep supervision fusion learning (CDSFL). The first strategy involves dynamically assigning confidence weights to each teacher model based on its performance, ensuring that the KD is performance-driven, and the CDSFL module augments the learning capabilities of the multi-teacher models by fostering mutual learning. Moreover, the fused information is distilled into the student model to improve its deep supervision. Extensive experiments on BraTS datasets demonstrate that our framework achieves promising unimodal segmentation results on the T1ce and T1 modalities and outperforms previous state-of-the-art methods. Code is available at https://github.com/ami-lab-knut/mtcm_kd.

Comparison of different group-level templates in gradient-based multimodal connectivity analysis

Abstract The study of large-scale brain connectivity is increasingly adopting unsupervised approaches that derive low-dimensional spatial representations from high-dimensional connectomes, referred to as gradient analysis. When translating this approach to study interindividual variations in connectivity, one technical issue pertains to the selection of an appropriate group-level template to which individual gradients are aligned. Here, we compared different group-level template construction strategies using functional and structural connectome data from neurotypical controls and individuals with autism spectrum disorder (ASD) to identify between-group differences. We studied multimodal magnetic resonance imaging data obtained from the Autism Brain Imaging Data Exchange (ABIDE) Initiative II and the Human Connectome Project (HCP). We designed six template construction strategies that varied in whether: (1) they included typical controls in addition to ASD; or (2) they mapped from one dataset onto another. We found that aligning a combined subject template of the ASD and control subjects from the ABIDE Initiative onto the HCP template exhibited the most pronounced effect size. This strategy showed robust identification of ASD-related brain regions for both functional and structural gradients across different study settings. Replicating the findings on focal epilepsy demonstrated the generalizability of our approach. Our findings will contribute to improving gradient-based connectivity research.

Multi-modal Magnetic Resonance Imaging to Assess Cerebrovascular Function in Atrial Fibrillation Patients

Atrial fibrillation (AF) is associated with an increased risk of cognitive decline, dementia, and stroke. Low cerebral blood flow (CBF) and low cerebrovascular carbon dioxide reactivity (CVRCO2) are associated with poor brain health outcomes. In the current study we employed magnetic resonance imaging (MRI) to quantify resting CBF and CVRCO2 in patients with AF and age-matched normal sinus rhythm (NSR) controls. We hypothesised that both resting CBF and CVRCO2 would be lower in patients with AF. Patients with AF (n=11, 65±14 years, 2 women) and NSR (n=12, 63±9 years, 7 women) underwent arterial spin labelling (ASL) at rest, and blood oxygen level-dependent (BOLD) MRI during a single-step increase in partial pressure of carbon dioxide (measured by end-tidal carbon dioxide (PETCO2)) induced by changing the fraction of inspired CO2 to 5% (21% O2, balance N2). Region of interest time courses were extracted with grey matter, middle cerebral artery, and posterior cerebral artery territory masks. CVRCO2 was defined as the percentage change (%Δ) in BOLD signal vs %Δ in PETCO2. Baseline CBF was not different between NSR and AF in grey matter (62±17 vs. 58±10 ml/min/100g, respectively; p=0.226), middle cerebral artery (59±16 vs. 56±14 ml/min/100g; p=0.361) and posterior cerebral artery (61±17 vs 53±15 ml/min/100g; p=0.138) territories. BOLD-derived CVRC02 was not different between NSR and AF groups in grey matter (0.11±0.04 vs. 0.12±0.03 %ΔBOLD/%ΔmmHg, respectively; p=0.372), middle cerebral artery (0.09±0.04 vs. 0.10±0.03 %ΔBOLD/%ΔmmHg; p=0.405) and posterior cerebral artery (0.13±0.04 vs. 0.14±0.05 %ΔBOLD/%ΔmmHg; p=0.367) territories. Collectively, these pilot data suggest that MRI-derived cerebrovascular CO2 responsiveness is not different in patients with AF and age-matched NSR controls that have similar resting CBF. Further investigations are required to extend these preliminary findings to a larger cohort to fully elucidate the effects of AF on cerebrovascular function. Support provided by the Royal Society of New Zealand Te Apārangi Marsden Fund (19-UOA-170). This is the full abstract presented at the American Physiology Summit 2024 meeting and is only available in HTML format. There are no additional versions or additional content available for this abstract. Physiology was not involved in the peer review process.

Glioma segmentation based on dense contrastive learning and multimodal features recalibration

Accurate segmentation of different regions of gliomas from multimodal magnetic resonance (MR) images is crucial for glioma grading and precise diagnosis, but many existing segmentation methods are difficult to effectively utilize multimodal MR image information to recognize accurately the lesion regions with small size, low contrast and irregular shape. To address this issue, this work proposes a novel 3D glioma segmentation model DCL-MANet. DCL-MANet has an architecture of multiple encoders and one single decoder. Each encoder is used to extract MR image features of a given modality. To overcome the entangle problems of multimodal semantic features, a dense contrastive learning (DCL) strategy is presented to extract the modality-specific and common features. Following that, feature recalibration block (RFB) based on modality-wise attention is used to recalibrate the semantic features of each modality, enabling the model to focus on the features that are beneficial for glioma segmentation. These recalibrated features are input into the decoder to obtain the segmentation results. To verify the superiority of the proposed method, we compare it with several state-of-the-art (SOTA) methods in terms of Dice, average symmetric surface distance (ASSD), HD95 and volumetric similarity (Vs). The comparison results show that the average Dice, ASSD, HD95 and Vs of DCL-MANet on all tumor regions are improved at least by 0.66%, 3.47%, 8.94% and 1.07% respectively. For small enhance tumor (ET) region, the corresponding improvement can be up to 0.37%, 7.83%, 11.32%, and 1.35%, respectively. In addition, the ablation results demonstrate the effectiveness of the proposed DCL and RFB, and combining them can significantly increase Dice (1.59%) and Vs (1.54%) while decreasing ASSD (40.51%) and HD95 (45.16%) on ET region. The proposed DCL-MANet could disentangle multimodal features and enhance the semantics of modality-dependent features, providing a potential means to accurately segment small lesion regions in gliomas.

Enhancing Robust Liver Cancer Diagnosis: A Contrastive Multi-Modality Learner with Lightweight Fusion and Effective Data Augmentation

This article explores the application of self-supervised contrastive learning in the medical domain, focusing on classification of multi-modality Magnetic Resonance (MR) images. To address the challenges of limited and hard-to-annotate medical data, we introduce multi-modality data augmentation (MDA) and cross-modality group convolution (CGC). In the pre-training phase, we leverage Simple Siamese networks to maximize the similarity between two augmented MR images from a patient, without a handcrafted pretext task. Our approach also combines 3D and 2D group convolution with a channel shuffle operation to efficiently incorporate different modalities of image features. Evaluation on liver MR images from a well-known hospital in Taiwan demonstrates a significant improvement over previous methods. This work contributes to advancing multi-modality contrastive learning, particularly in the context of medical imaging, offering enhanced tools for analyzing complex image data.