Multimodal Imaging Data Research Articles

The negative relationship between brain-age gap and psychological resilience defines the age-related neurocognitive status in older people.

Biological brain age is a brain-predicted age using machine learning to indicate brain health and its associated conditions. The presence of an older predicted brain age relative to the actual chronological age is indicative of accelerated aging processes. Consequently, the disparity between the brain's chronological age and its predicted age (brain-age gap) and the factors influencing this disparity provide critical insights into cerebral health dynamics during aging. In this study, we employed a Lasso regression model and analyzed multimodal imaging data from 124 participants aged 53 to 76 to formulate and predict brain age. Additionally, we conducted partial correlation analyses to explore the complex relationship between the brain-age gap and network metrics, cognitive assessments, and emotional evaluations, while controlling for chronological age, gender, and education. Our findings highlight psychological resilience as a significant mitigating factor against premature brain aging. It is established that psychological resilience significantly influences the modulation of the brain-age gap. Moreover, psychological resilience and the brain-age gap exhibit a high accuracy (above 0.72) in segregating Montreal Cognitive Assessment score-based cohorts. This observation underscores significant insight into the potential of utilizing the brain-age gap as a diagnostic tool for the early detection of accelerated aging. It advocates for the timely application of interventions, including the development of programs aimed at bolstering psychological resilience.

MsiFlow: automated workflows for reproducible and scalable multimodal mass spectrometry imaging and microscopy data analysis

Multimodal imaging by matrix-assisted laser desorption ionisation mass spectrometry imaging (MALDI MSI) and microscopy holds potential for understanding pathological mechanisms by mapping molecular signatures from the tissue microenvironment to specific cell populations. However, existing software solutions for MALDI MSI data analysis are incomplete, require programming skills and contain laborious manual steps, hindering broadly applicable, reproducible, and high-throughput analysis to generate impactful biological discoveries. Here, we present msiFlow, an accessible open-source, platform-independent and vendor-neutral software for end-to-end, high-throughput, transparent and reproducible analysis of multimodal imaging data. msiFlow integrates all necessary steps from raw data import to analytical visualisation along with state-of-the-art and self-developed algorithms into automated workflows. Using msiFlow, we unravel the molecular heterogeneity of leukocytes in infected tissues by spatial regulation of ether-linked phospholipids containing arachidonic acid. We anticipate that msiFlow will facilitate the broad applicability of MSI in multimodal imaging to uncover context-dependent cellular regulations in disease states.

Multi-Modal Twitter Data Analysis for Identifying Offensive Posts Using a Deep Cross Attention based Transformer Framework

In today’s society dissemination of information among the individuals occur very rapidly due to the widespread usage of social media platforms like Twitter (now-a-days acclaimed as X). However, information may pose challenges to maintaining a healthy online environment because often it contains harmful content. This paper presents a novel approach to identify different categories of offensive posts such as hate speech, profanity, targeted insult, and derogatory commentary by analyzing multi-modal image and text data, collected from Twitter. We propose a comprehensive deep learning framework, “Value Mixed Cross Attention Transformer” (VMCA-Trans) that leverage a combination of computer vision and natural language processing methodologies to effectively classify the posts into four classes with binary labels. We have created an in-house dataset (OffenTweet) comprising of Twitter posts having textual content, accompanying with images to build the proposed model. The dataset is carefully annotated by several experts with offensive labels such as hate speech, profanity, targeted insult, and derogatory commentary. VMCA-Trans utilizes fine-tuned state-of-the-art transformer based backbones such as ViT, BERT, RoBERTA etc. The combined representation of image and text embeddings obtained by these fine-tuned transformer encoders, is fed into a classifier to categorize the posts into offensive and non-offensive classes. To assess its effectiveness, we extensively evaluate the VMCA-Trans model using various performance metrics. The results indicate that the proposed multi-modal approach, achieves superior performance compared to traditional unimodal methods.

Light sheet fluorescence microscopy for monitoring drug delivery: Unlocking the developmental phases of embryos.

Light sheet fluorescence microscopy (LSFM) has emerged as a transformative imaging technique in the study of drug delivery and embryonic development, offering high-resolution, real-time visualization with minimal phototoxicity. This review examines the application of LSFM in tracking drug pharmacokinetics, tissue-specific targeting, and drug efficacy during critical phases of embryonic development. Recent advancements in fluorescent labeling and machine learning integration have enabled more precise monitoring of drug release, distribution, and interaction with developing tissues. The ability of LSFM to capture long-term dynamics at single-cell resolution has revolutionized drug discovery, especially in nanomedicine and targeted therapies. By integrating LSFM with multimodal imaging and AI-driven data analysis, researchers are now better equipped to explore complex biological processes and optimize drug delivery in a highly controlled, minimally invasive manner. Finally, the review highlights the pivotal role of LSFM in advancing drug delivery research, addressing existing challenges, and unlocking new frontiers in clinical applications.

A multi-modal neuroimaging data release for Meige Syndrome and Facial Paralysis Research

The sharing of multimodal magnetic resonance imaging (MRI) data is of utmost importance in the field, as it enables a deeper understanding of facial nerve-related pathologies. However, there is a significant lack of multi-modal neuroimaging databases specifically focused on these conditions, which hampers our comprehensive knowledge of the neural foundations of facial paralysis. To address this critical gap and propel advancements in this area, we have released the Multimodal Neuroimaging Dataset of Meige Syndrome, Facial Paralysis, and Healthy Controls (MND-MFHC). This dataset includes detailed clinical assessments of 53 individuals with facial paralysis (FP), 31 patients with Meige syndrome (MS), and 102 healthy controls (HC). To promote open access, the BIDS-formatted data and associated quality control reports can be accessed through the Science Data Bank (SciDB). By sharing this comprehensive dataset, our aim is to facilitate further research and exploration into the intricate neural mechanisms underlying facial nerve-related pathologies.

An Empirical Analysis of Transformer-Based and Convolutional Neural Network Approaches for Early Detection and Diagnosis of Cancer Using Multimodal Imaging and Genomic Data

An Empirical Analysis of Transformer-Based and Convolutional Neural Network Approaches for Early Detection and Diagnosis of Cancer Using Multimodal Imaging and Genomic Data

Enhancing Ophthalmological Diagnoses: An Adaptive Ensemble Learning Approach Using Fundus and OCT Imaging

This study presents an innovative Ensemble Disease Learning Algorithm (EDL) for the detection and classification of retinal diseases using fundus images. We enhance our method by incorporating deep learning techniques and multi-modal imaging data, including optical coherence tomography (OCT) images alongside fundus photographs, to provide a more comprehensive understanding of retinal pathology. The advanced EDL integrates Convolutional Neural Networks (CNNs) and attention mechanisms with Capsule Networks (CapsNet) and Support Vector Machine (SVM) classifiers for more nuanced feature extraction and classification. We introduce a novel ensemble adaptive weighting approach that dynamically adjusts classifier weights based on performance across disease types and severity levels, significantly improving the algorithm's handling of complex and rare cases. To enhance model interpretability, we implement an explainable AI component that provides visual heatmaps of the most significant regions for each diagnosis to clinicians. We evaluate the enhanced EDL on a large, diverse dataset encompassing multiple retinal diseases, including diabetic retinopathy, age-related macular degeneration, and glaucoma, across various ethnicities and age groups. Our results demonstrate superior accuracy, sensitivity, and specificity compared to our previous model and other state-of-the-art approaches. A prospective clinical validation study assesses the algorithm's real-world performance. This research advances automated retinal disease diagnosis by making it more robust, accurate, and clinically relevant, potentially improving patient outcomes and global eye care through early disease detection and treatment planning.

Artificial Intelligence-Based Methodologies for Early Diagnostic Precision and Personalized Therapeutic Strategies in Neuro-Ophthalmic and Neurodegenerative Pathologies.

Advancements in neuroimaging, particularly diffusion magnetic resonance imaging (MRI) techniques and molecular imaging with positron emission tomography (PET), have significantly enhanced the early detection of biomarkers in neurodegenerative and neuro-ophthalmic disorders. These include Alzheimer's disease, Parkinson's disease, multiple sclerosis, neuromyelitis optica, and myelin oligodendrocyte glycoprotein antibody disease. This review highlights the transformative role of advanced diffusion MRI techniques-Neurite Orientation Dispersion and Density Imaging and Diffusion Kurtosis Imaging-in identifying subtle microstructural changes in the brain and visual pathways that precede clinical symptoms. When integrated with artificial intelligence (AI) algorithms, these techniques achieve unprecedented diagnostic precision, facilitating early detection of neurodegeneration and inflammation. Additionally, next-generation PET tracers targeting misfolded proteins, such as tau and alpha-synuclein, along with inflammatory markers, enhance the visualization and quantification of pathological processes in vivo. Deep learning models, including convolutional neural networks and multimodal transformers, further improve diagnostic accuracy by integrating multimodal imaging data and predicting disease progression. Despite challenges such as technical variability, data privacy concerns, and regulatory barriers, the potential of AI-enhanced neuroimaging to revolutionize early diagnosis and personalized treatment in neurodegenerative and neuro-ophthalmic disorders is immense. This review underscores the importance of ongoing efforts to validate, standardize, and implement these technologies to maximize their clinical impact.

Plasma proteomics identify biomarkers and undulating changes of brain aging.

Proteomics enables the characterization of brain aging biomarkers and discernment of changes during brain aging. We leveraged multimodal brain imaging data from 10,949 healthy adults to estimate brain age gap (BAG), an indicator of brain aging. Proteome-wide association analysis across 4,696 participants of 2,922 proteins identified 13 significantly associated with BAG, implicating stress, regeneration and inflammation. Brevican (BCAN) (β = -0.838, P = 2.63 × 10-10) and growth differentiation factor 15 (β = 0.825, P = 3.48 × 10-11) showed the most significant, and multiple, associations with dementia, stroke and movement functions. Dysregulation of BCAN affected multiple cortical and subcortical structures. Mendelian randomization supported the causal association between BCAN and BAG. We revealed undulating changes in the plasma proteome across brain aging, and profiled brain age-related change peaks at 57, 70 and 78 years, implicating distinct biological pathways during brain aging. Our findings revealed the plasma proteomic landscape of brain aging and pinpointed biomarkers for brain disorders.

Dementia prediction with multimodal clinical and imaging data

Dementia affects millions of people worldwide, and poses significant challenges due to its irreversible nature and a lack of effective treatment options. Dementia has a considerable influence on people and society and puts a heavy burden on the healthcare systems. This underscores an urgent need for proactive measures to address this public health concern through early detection and intervention. This paper investigates the use of machine learning for an early detection of dementia and its progression utilizing a public dataset. Various traditional machine learning algorithms, were used on the demographic data, with the Gaussian Naïve Bayes achieving the highest accuracy of 91.30%. Four deep learning models, ResNet50, DenseNet121, VGG16, and Inceptionv3 were used on image data, with the DenseNet121 model achieving the highest accuracy of 90%. We also used SHapley Additive exPlanations (SHAP) framework for dementia progression which revealed that Normalised Whole Brain Volume (nWBV) exhibited higher variability in their impact across models. This study demonstrates the potential of machine learning approaches for early dementia detection and prognosis, which can have significant effect in patient care strategies.

Neural mass modeling for the masses: Democratizing access to whole-brain biophysical modeling with FastDMF.

Different whole-brain computational models have been recently developed to investigate hypotheses related to brain mechanisms. Among these, the Dynamic Mean Field (DMF) model is particularly attractive, combining a biophysically realistic model that is scaled up via a mean-field approach and multimodal imaging data. However, an important barrier to the widespread usage of the DMF model is that current implementations are computationally expensive, supporting only simulations on brain parcellations that consider less than 100 brain regions. Here, we introduce an efficient and accessible implementation of the DMF model: the FastDMF. By leveraging analytical and numerical advances-including a novel estimation of the feedback inhibition control parameter and a Bayesian optimization algorithm-the FastDMF circumvents various computational bottlenecks of previous implementations, improving interpretability, performance, and memory use. Furthermore, these advances allow the FastDMF to increase the number of simulated regions by one order of magnitude, as confirmed by the good fit to fMRI data parcellated at 90 and 1,000 regions. These advances open the way to the widespread use of biophysically grounded whole-brain models for investigating the interplay between anatomy, function, and brain dynamics and to identify mechanistic explanations of recent results obtained from fine-grained neuroimaging recordings.

Identification of mild cognitive impairment using multimodal 3D imaging data and graph convolutional networks

Objective.Mild cognitive impairment (MCI) is a precursor stage of dementia characterized by mild cognitive decline in one or more cognitive domains, without meeting the criteria for dementia. MCI is considered a prodromal form of Alzheimer's disease (AD). Early identification of MCI is crucial for both intervention and prevention of AD. To accurately identify MCI, a novel multimodal 3D imaging data integration graph convolutional network (GCN) model is designed in this paper.Approach.The proposed model utilizes 3D-VGGNet to extract three-dimensional features from multimodal imaging data (such as structural magnetic resonance imaging and fluorodeoxyglucose positron emission tomography), which are then fused into feature vectors as the node features of a population graph. Non-imaging features of participants are combined with the multimodal imaging data to construct a population sparse graph. Additionally, in order to optimize the connectivity of the graph, we employed the pairwise attribute estimation （PAE） method to compute the edge weights based on non-imaging data, thereby enhancing the effectiveness of the graph structure. Subsequently, a population-based GCN integrates the structural and functional features of different modal images into the features of each participant for MCI classification.Main results.Experiments on the AD Neuroimaging Initiative demonstrated accuracies of 98.57%, 96.03%, and 96.83% for the normal controls (NC)-early MCI (EMCI), NC-late MCI (LMCI), and EMCI-LMCI classification tasks, respectively. The AUC, specificity, sensitivity, and F1-score are also superior to state-of-the-art models, demonstrating the effectiveness of the proposed model. Furthermore, the proposed model is applied to the ABIDE dataset for autism diagnosis, achieving an accuracy of 91.43% and outperforming the state-of-the-art models, indicating excellent generalization capabilities of the proposed model.Significance.This study demonstratesthe proposed model's ability to integrate multimodal imaging data and its excellent ability to recognize MCI. This will help achieve early warning for AD and intelligent diagnosis of other brain neurodegenerative diseases.

Characterizing the role of the microbiota-gut-brain axis in cerebral small vessel disease: An integrative multi‑omics study

BackgroundPrior efforts have revealed changes in gut microbiome, circulating metabolome, and multimodal neuroimaging features in cerebral small vessel disease (CSVD). However, there is a paucity of research integrating the multi-omic information to characterize the role of the microbiota-gut-brain axis in CSVD. MethodsWe collected gut microbiome, fecal and blood metabolome, multimodal magnetic resonance imaging data from 37 CSVD patients with white matter hyperintensities and 46 healthy controls. Between-group comparison was performed to identify the differential gut microbial taxa, followed by performance of multi-stage microbiome-metabolome-neuroimaging-neuropsychology correlation analyses in CSVD patients. ResultsOur data showed both depleted and enriched gut microbes in CSVD patients. Among the differential microbes, Haemophilus and Akkermansia were associated with a range of metabolites enriched for Aminoacyl-tRNA biosynthesis pathway. Furthermore, the affected metabolites were associated with neuroimaging measures involving gray matter morphology, spontaneous intrinsic brain activity, white matter integrity, and global structural network topology, which were in turn related to cognition and emotion in CSVD patients. ConclusionOur findings provide an integrative framework to understand the pathophysiological mechanisms underlying the interplay between gut microbiota dysbiosis and CSVD, highlighting the potential of targeting the microbiota-gut-brain axis as a therapeutic strategy in CSVD patients.

Predictive Diagnostic Model for Early Osteoporosis Detection Using Deep Learning and Multimodal Imaging Data: A Systematic Review and Meta-Analysis

Osteoporosis is a common condition that weakens bones, making them more prone to fractures. Early detection is crucial for preventing fractures and improving patients’ quality of life. However, traditional methods, such as Dual-Energy X-ray Absorptiometry (DXA), often struggle to accurately predict fracture risk and may overlook minor changes in bone structure. This study focuses on developing a predictive model for early osteoporosis detection using deep learning algorithms combined with various imaging techniques, including MRI, CT, and High-Resolution Peripheral Quantitative Computed Tomography (HR-pQCT). A systematic review and meta-analysis of studies published between 2014 and 2024 were conducted, examining the use of deep learning models applied to multimodal imaging data. The meta-analysis highlighted differences in the accuracy and effectiveness of various models, and their performance was measured in terms of accuracy, sensitivity, and specificity, following PRISMA guidelines. The results showed that deep learning models outperformed traditional methods in early osteoporosis detection. The use of multiple imaging techniques provided a more detailed assessment of bone health, allowing the models to identify complex patterns that are difficult for human interpretation. These models demonstrated high accuracy and significant potential for improving clinical decision-making. By integrating deep learning with multimodal imaging, this approach offers a promising solution for enhancing the early detection of osteoporosis. The models tested in this study proved to be highly effective, yielding more accurate fracture risk predictions and enabling earlier interventions. This could lead to better patient outcomes and reduced healthcare costs.

Comparison of different group-level templates in gradient-based multimodal connectivity analysis.

The study of large-scale brain connectivity is increasingly adopting unsupervised approaches that derive low-dimensional spatial representations from high-dimensional connectomes, referred to as gradient analysis. When translating this approach to study interindividual variations in connectivity, one technical issue pertains to the selection of an appropriate group-level template to which individual gradients are aligned. Here, we compared different group-level template construction strategies using functional and structural connectome data from neurotypical controls and individuals with autism spectrum disorder (ASD) to identify between-group differences. We studied multimodal magnetic resonance imaging data obtained from the Autism Brain Imaging Data Exchange (ABIDE) Initiative II and the Human Connectome Project (HCP). We designed six template construction strategies that varied in whether (1) they included typical controls in addition to ASD; or (2) they mapped from one dataset onto another. We found that aligning a combined subject template of the ASD and control subjects from the ABIDE Initiative onto the HCP template exhibited the most pronounced effect size. This strategy showed robust identification of ASD-related brain regions for both functional and structural gradients across different study settings. Replicating the findings on focal epilepsy demonstrated the generalizability of our approach. Our findings will contribute to improving gradient-based connectivity research.

Radiation image reconstruction and uncertainty quantification using a Gaussian process prior

We propose a complete framework for Bayesian image reconstruction and uncertainty quantification based on a Gaussian process prior (GPP) to overcome limitations of maximum likelihood expectation maximization (ML-EM) image reconstruction algorithm. The prior distribution is constructed with a zero-mean Gaussian process (GP) with a choice of a covariance function, and a link function is used to map the Gaussian process to an image. Unlike many other maximum a posteriori approaches, our method offers highly interpretable hyperparamters that are selected automatically with the empirical Bayes method. Furthermore, the GP covariance function can be modified to incorporate a priori structural priors, enabling multi-modality imaging or contextual data fusion. Lastly, we illustrate that our approach lends itself to Bayesian uncertainty quantification techniques, such as the preconditioned Crank–Nicolson method and the Laplace approximation. The proposed framework is general and can be employed in most radiation image reconstruction problems, and we demonstrate it with simulated free-moving single detector radiation source imaging scenarios. We compare the reconstruction results from GPP and ML-EM, and show that the proposed method can significantly improve the image quality over ML-EM, all the while providing greater understanding of the source distribution via the uncertainty quantification capability. Furthermore, significant improvement of the image quality by incorporating a structural prior is illustrated.

Prognostic value of multimodality imaging in the contemporary management of cardiac sarcoidosis

BackgroundEchocardiography, cardiac magnetic resonance and cardiac 18fluorodeoxyglucose positron emission tomography (FDG-PET) imaging play key roles in the diagnosis and management of cardiac sarcoidosis (CS), but the relative value of each...

Frontoparietal network topology as a neural marker of musical perceptual abilities

Why are some individuals more musical than others? Neither cognitive testing nor classical localizationist neuroscience alone can provide a complete answer. Here, we test how the interplay of brain network organization and cognitive function delivers graded perceptual abilities in a distinctively human capacity. We analyze multimodal magnetic resonance imaging, cognitive, and behavioral data from 200+ participants, focusing on a canonical working memory network encompassing prefrontal and posterior parietal regions. Using graph theory, we examine structural and functional frontoparietal network organization in relation to assessments of musical aptitude and experience. Results reveal a positive correlation between perceptual abilities and the integration efficiency of key frontoparietal regions. The linkage between functional networks and musical abilities is mediated by working memory processes, whereas structural networks influence these abilities through sensory integration. Our work lays the foundation for future investigations into the neurobiological roots of individual differences in musicality.

Baseline multimodal imaging to predict longitudinal clinical decline in atypical Alzheimer's disease

There are recognized neuroimaging regions of interest in typical Alzheimer's disease which have been used to track disease progression and aid prognostication. However, there is a need for validated baseline imaging markers to predict clinical decline in atypical Alzheimer's Disease. We aimed to address this need by producing models from baseline imaging features using penalized regression and evaluating their predictive performance on various clinical measures.Baseline multimodal imaging data, in combination with clinical testing data at two time points from 46 atypical Alzheimer's Disease patients with a diagnosis of logopenic progressive aphasia (N = 24) or posterior cortical atrophy (N = 22), were used to generate our models. An additional 15 patients (logopenic progressive aphasia = 7, posterior cortical atrophy = 8), whose data were not used in our original analysis, were used to test our models. Patients underwent MRI, FDG-PET and Tau-PET imaging and a full neurologic battery at two time points. The Schaefer functional atlas was used to extract network-based and regional gray matter volume or PET SUVR values from baseline imaging. Penalized regression (Elastic Net) was used to create models to predict scores on testing at Time 2 while controlling for baseline performance, education, age, and sex. In addition, we created models using clinical or Meta Region of Interested (ROI) data to serve as comparisons.We found the degree of baseline involvement on neuroimaging was predictive of future performance on cognitive testing while controlling for the above measures on all three imaging modalities. In many cases, model predictability improved with the addition of network-based neuroimaging data to clinical data. We also found our network-based models performed superiorly to the comparison models comprised of only clinical or a Meta ROI score.Creating predictive models from imaging studies at a baseline time point that are agnostic to clinical diagnosis as we have described could prove invaluable in both the clinical and research setting, particularly in the development and implementation of future disease modifying therapies.

Imaging of Sacroiliac Pain: The Current State-of-the-Art.

Pain in the sacroiliac (SI) region is a common clinical manifestation, often caused by diseases involving the SI joints. This is typically due to inflammation or degenerative changes, while infections or cancer are less frequent causes. The SI joint is challenging to image accurately because of its distinct anatomical characteristics. For an accurate diagnosis, conventional radiography often needs to be supplemented with more precise methods such as magnetic resonance imaging (MRI) or computed tomography (CT). Sacroiliitis, a common presenting feature of axial spondyloarthritis (axial SpA), manifests as bone marrow edema, erosions, sclerosis, and joint space narrowing. Septic sacroiliitis and repetitive stress injuries in sports can also cause changes resembling inflammatory sacroiliitis. Other conditions, such as osteitis condensans ilii (OCI), can mimic the radiologic characteristics of sacroiliitis. Inflammatory lesions are diagnosed by concurrent erosions, hyperostosis, and ankylosis. Ligament ossifications or mechanical stress can also result in arthritic disorders. Determining the exact diagnosis can be aided by the distribution of the lesions. Inflammatory lesions can affect any part of the articulation, including the inferior and posterior portions. Mechanical lesions, such as those seen in OCI, often occur in the anterior middle region of the joint. In cases of idiopathic skeletal hyperostosis, ligament ossification is found at the joint borders. This pictorial essay describes common SI joint problems, illustrated with multimodal imaging data. We, also, discuss strategies for selecting the best imaging modalities, along with imaging pitfalls, key points, and approaches for treating patients with suspected inflammatory back pain.