Multimodal Imaging Research Articles

Predictive Diagnostic Model for Early Osteoporosis Detection Using Deep Learning and Multimodal Imaging Data: A Systematic Review and Meta-Analysis

Osteoporosis is a common condition that weakens bones, making them more prone to fractures. Early detection is crucial for preventing fractures and improving patients’ quality of life. However, traditional methods, such as Dual-Energy X-ray Absorptiometry (DXA), often struggle to accurately predict fracture risk and may overlook minor changes in bone structure. This study focuses on developing a predictive model for early osteoporosis detection using deep learning algorithms combined with various imaging techniques, including MRI, CT, and High-Resolution Peripheral Quantitative Computed Tomography (HR-pQCT). A systematic review and meta-analysis of studies published between 2014 and 2024 were conducted, examining the use of deep learning models applied to multimodal imaging data. The meta-analysis highlighted differences in the accuracy and effectiveness of various models, and their performance was measured in terms of accuracy, sensitivity, and specificity, following PRISMA guidelines. The results showed that deep learning models outperformed traditional methods in early osteoporosis detection. The use of multiple imaging techniques provided a more detailed assessment of bone health, allowing the models to identify complex patterns that are difficult for human interpretation. These models demonstrated high accuracy and significant potential for improving clinical decision-making. By integrating deep learning with multimodal imaging, this approach offers a promising solution for enhancing the early detection of osteoporosis. The models tested in this study proved to be highly effective, yielding more accurate fracture risk predictions and enabling earlier interventions. This could lead to better patient outcomes and reduced healthcare costs.

Multimodal Optical Imaging of Ex Vivo Fallopian Tubes to Distinguish Early and Occult Tubo-Ovarian Cancers.

Background: There are currently no effective screening measures to detect early or occult tubo-ovarian cancers, resulting in late-stage detection and high mortality. This work explores whether an optical imaging catheter can detect early-stage tubo-ovarian cancers or precursor lesions where they originate in the fallopian tubes. Methods: This device collects co-registered optical coherence tomography (OCT) and autofluorescence imaging (AFI). OCT provides three-dimensional assessment of underlying tissue structures; autofluorescence imaging provides functional contrast of endogenous fluorophores. Ex vivo fallopian tubes (n = 28; n = 7 cancer patients) are imaged; we present methods for the calculation of and analyze eleven imaging biomarkers related to fluorescence, optical attenuation, and OCT texture for their potential to detect tubo-ovarian cancers and other lesions of interest. Results: We visualize folded plicae, vessel-like structures, tissue layering, hemosiderin deposits, and regions of fibrotic change. High-grade serous ovarian carcinoma appears as reduced autofluorescence paired with homogenous OCT and reduced mean optical attenuation. Specimens containing cancerous lesions demonstrate a significant increase in median autofluorescence intensity and decrease in Shannon entropy compared to specimens with no lesion. Non-cancerous specimens demonstrate an increase in optical attenuation in the fimbriae when compared to the isthmus or the ampulla. Conclusions: We conclude that this approach shows promise and merits further investigation of its diagnostic potential.

Dissecting neural correlates of theory of mind and executive functions in behavioral variant frontotemporal dementia

Behavioral variant frontotemporal dementia (bvFTD) is characterized by profound and early deficits in social cognition (SC) and executive functions (EF). To date it remains unclear whether deficits of the respective cognitive domains are based on the degeneration of distinct brain regions. In 103 patients with a diagnosis of bvFTD (possible/probable/definite: N = 40/58/5) from the frontotemporal lobar degeneration (FTLD) consortium Germany cohort (age 62.5±9.4 years, gender 38 female/65 male) we applied multimodal structural imaging, i.e. voxel-based morphometry, cortical thickness (CTH) and networks of structural covariance via source based morphometry. We cross-sectionally investigated associations with performance in a modified Reading the Mind in the Eyes Test (RMET; reflective of theory of mind - ToM) and five different tests reflective of EF (i.e. Hamasch-Five-Point Test, semantic and phonemic Fluency, Trail Making Test, Stroop interference). Finally, we investigated the conjunction of RMET correlates with functional networks commonly associated with SC respectively ToM and EF as extracted meta-analytically within the Neurosynth database. RMET performance was mainly associated with gray matter volume (GMV) and CTH within temporal and insular cortical regions and less within the prefrontal cortex (PFC), whereas EF performance was mainly associated with prefrontal regions (GMV and CTH). Overlap of RMET and EF associations was primarily located within the insula, adjacent subcortical structures (i.e. putamen) and the dorsolateral PFC (dlPFC). These patterns were more pronounced after adjustment for the respective other cognitive domain. Corroborative results were obtained in analyses of structural covariance networks. Overlap of RMET with meta-analytically extracted functional networks commonly associated with SC, ToM and EF was again primarily located within the temporal and insular region and the dlPFC. In addition, on a meta-analytical level, strong associations were found for temporal cortical RMET correlates with SC and ToM in particular. These data indicate a temporo-frontal dissociation of bvFTD related disturbances of ToM and EF, with atrophy of the anterior temporal lobe being critically involved in ToM deficits. The consistent overlap within the insular cortex may be attributable to the multimodal and integrative role of this region in socioemotional and cognitive processing.

8-Anilino-1-naphthalenesulfonate-Conjugated Carbon-Coated Ferrite Nanodots for Fluoromagnetic Imaging, Smart Drug Delivery, and Biomolecular Sensing

Background: Superparamagnetic properties and excitation independence have been incorporated into carbon-decorated manganese ferrite nanodots (MnFe@C) to introduce an economical and safer multimodal agent for use in both T1-T2 MRI and fluorescence-based imaging to replace the conventional highly toxic heavy metal contrast agents. Methods: The surface conjugation of 8-anilino-1-naphthalenesulfonate (ANS) to MnFe@C nanodots (ANS-MnFe@C) enhances both longitudinal and transverse MRI relaxation, improves fluorescence for optical imaging, and increases protein detection sensitivity, showing higher multimodal efficacy in terms of molar relaxivity, radiant efficiencies, and fluorescence sensitivity compared to MnFe@C. Results: The band gap energy was determined using Tauc’s equation to be 3.32 eV, while a 72% quantum yield demonstrated that ANS-MnFe@C was highly fluorescent, with the linear range and association constant calculated using the Stern–Volmer relation. The synthesized ANS-MnFe@C demonstrated excellent selectivity and sensitivity for bovine serum albumin (BSA), with a nanomolar detection limit of 367.09 nM and a broad linear range from 0.015 to 0.225 mM. Conclusions: In conclusion, ANS-MnFe@C holds ease of fabrication, good biocompatibility, as assessed in A375 cells, and an effective pH-sensitive doxorubicin release profile to establish anticancer activity in lung cancer cell line (A549), highlighting its potential as an affordable therapeutic agent for multimodal imaging, drug delivery, and protein sensing.

A Multi-Hierarchical Complementary Feature Interaction Network for Accelerated Multi-Modal MR Imaging

Magnetic resonance (MR) imaging is widely used in the clinical field due to its non-invasiveness, but the long scanning time is still a bottleneck for its popularization. Using the complementary information between multi-modal imaging to accelerate imaging provides a novel and effective MR fast imaging solution. However, previous technologies mostly use simple fusion methods and fail to fully utilize their potential sharable knowledge. In this study, we introduced a novel multi-hierarchical complementary feature interaction network (MHCFIN) to realize joint reconstruction of multi-modal MR images with undersampled data and thus accelerate multi-modal imaging. Firstly, multiple attention mechanisms are integrated with a dual-branch encoder–decoder network to represent shared features and complementary features of different modalities. In the decoding stage, the multi-modal feature interaction module (MMFIM) acts as a bridge between the two branches, realizing complementary knowledge transfer between different modalities through cross-level fusion. The single-modal feature fusion module (SMFFM) carries out multi-scale feature representation and optimization of the single modality, preserving better anatomical details. Extensive experiments are conducted under different sampling patterns and acceleration factors. The results show that this proposed method achieves obvious improvement compared with existing state-of-the-art reconstruction methods in both visual quality and quantity.

Transarterial embolization for anterior cranial fossa dural arteriovenous fistula based on multi-modal three-dimensional imaging.

Dural arteriovenous fistula (DAVF) in the anterior cranial fossa (ACF) is known to show a high risk of intracranial hemorrhage. Recently, multi-modal fusion imaging with computed tomography angiography, computed tomography venography, and three-dimensional (3D) rotation angiography have been used preoperatively to ensure anatomical safety. We report on endovascular treatment as a first-line approach for ACFDAVF based on the understanding of vascular anatomy obtained from multi-modal fusion imaging. All patients with ACF-DAVF treated endovascularly as a first-line approach were included in this study. Analyses took into account complications (particularly visual function), immediate angiographic outcomes, and follow-up findings in consecutive patients with ACF-DAVF treated with interventional treatment based on multi-modal fusion imaging. Five patients with ACF-DAVF underwent six sessions of transarterial embolization (TAE) in our institution. The five male patients (mean age, 74.5 years; range, 60-84 years) were treated with liquid embolic agents (Onyx, four procedures; n-butyl 2-cyanoacrylate, two procedures). No difference was seen between preoperative image evaluation and image evaluation during the endovascular procedure, and in all cases, a microcatheter was navigated into a target artery assumed from preoperative multi-modal imaging, allowing treatment completion in a single procedure. In all cases, the shunt disappeared completely and visual function after procedure was maintained. At the last follow-up, all patients showed a modified Rankin scale score of 0 or 1 with no recurrences. Multi-modal fusion imaging facilitates a 3D understanding of the vascular anatomy, allowing TAE as the first-line treatment for ACF-DAVF.

Methodological aspects of correlative, multimodal, multiparametric breast cancer imaging: from data acquisition to image processing for AI-based radioproteomics in a preclinical setting

Preclinical high-field magnetic resonance imaging (MRI) systems offer a diverse array of MRI techniques, providing rich multiparametric MRI (mpMRI) platforms for studying numerous biological parameters. mpMRI platforms prove particularly indispensable when investigating tumors that exhibit profound intratumoral heterogeneity, such as breast cancer. A thoughtful comprehension of the origins of intratumoral heterogeneity is imperative for the judicious assessment of new targeted therapies and treatment interventions. Furthermore, when data from mpMRI are complemented with data from other in vivo imaging modalities, such as positron emission tomography (PET), and correlated with data from ex vivo modalities, such as matrix-assisted laser desorption imaging mass spectrometry (MALDI IMS), the in vivo parameters can be further elucidated at a molecular level and microscopic scale. Nevertheless, extracting meaningful scientific insights from such complex datasets necessitates the utilization of machine learning (ML) approaches to discern region-specific radiomic features. The development of correlative, multimodal imaging (CMI) workflows, such as one incorporating MRI, PET and MALDI IMS, is inherently challenging, given the many technological and methodological challenges related to multimodal data acquisition as well as the physiological limitations of the laboratory mice of the investigation. Standardization efforts in image acquisition and processing are required to increase the reproducibility and translatability of CMI data. To address the challenges of developing standardized CMI workflows and stimulate dialog regarding this area of need, we present a practical workflow to investigate tumor heterogeneity in breast cancer xenografts across various spatial scales. Our workflow entails simultaneous functional MRI and PET acquisitions in living mice, followed by correlation with post-imaging MALDI IMS and histologic data. Additionally, we propose data preprocessing steps for potential ML applications. We illustrate the feasibility of this workflow through two examples, showcasing its effectiveness in comparing in vivo and ex vivo images to evaluate tumor metabolism and hypoxia in mice with breast cancer xenografts.

Structural-informed functional MRI analysis of patients with empathy impairment following stroke.

The underlying functional alterations of brain structural changes among patients with empathy impairment following stroke remain unclear. We sought to investigate functional connectivity changes informed by brain structural abnormalities in multimodal magnetic resonance imaging (MRI) among patients with empathy impairment following stroke. We enrolled people who had experienced their first ischemic stroke, along with healthy controls. We assessed empathy 3 months after stroke using the Chinese version of the Empathy Quotient (EQ). During the acute phase, all patients underwent basic magnetic resonance imaging (MRI), followed by multimodal MRI during follow-up. Our MRI analyses encompassed acute infarction segmentation, volumetric brain measurements, regional quantification of diffusion parameters, and both region-of-interest-based and seed-based functional connectivity assessments. We grouped patients based on the severity of their empathy impairment for comparative analysis. We included 84 patients who had stroke and 22 healthy controls. Patients had lower EQ scores than controls. Patients with low empathy had larger left cortical infarcts (odds ratio [OR] 4.082, 95% confidence interval [CI] 1.183-14.088), more pronounced atrophy in the right cingulate cortex (OR 1.248, 95% CI 1.038-1.502), and lower scores on the Montreal Cognitive Assessment (OR 0.873, 95% CI 0.74-0.947). In addition, the cingulate cortex served as the seed in the seed-based analysis, which showed heightened functional connectivity between the anterior cingulate gyrus and the right superior parietal lobule, specifically in the low-empathy group. We did not evaluate the relationship between specific network involvement and empathy impairment among patients following stroke. Among patients with subacute ischemic stroke, reduced empathy was strongly associated with a more severe cognitive profile and atrophy of the right cingulate cortex. Our subsequent structural-informed functional MRI analysis suggests that the enhanced connectivity between the anterior cingulate gyrus and the superior parietal lobule may function as a compensatory mechanism for this atrophy.

Multimodality imaging of right ventricular apical atypical cavernous hemangioma.

Multimodality imaging of right ventricular apical atypical cavernous hemangioma.

Super Resolved Single-Cell Spatial Metabolomics from Multimodal Mass Spectrometry Imaging guided by Imaging Mass Cytometry.

Mass spectrometry imaging (MSI) is a powerful technique for spatially resolved analysis of metabolites and other biomolecules within biological tissues. However, the inherent low spatial resolution of MSI often limits its ability to provide detailed cellular-level information. To address this limitation, we propose a guided super-resolution (GSR) approach that leverages high-resolution Imaging Mass Cytometry (IMC) images to enhance the spatial resolution of low-resolution MSI data. By using these detailed IMC images as guides, we improve the resolution of MSI images, creating high-resolution metabolite maps. This enhancement facilitates more precise analysis of cellular structures and tissue architectures, providing deeper insights into super-resolved spatial metabolomics at the single-cell level.

An ensemble deep learning model for medical image fusion with Siamese neural networks and VGG-19.

Multimodal medical image fusion methods, which combine complementary information from many multi-modality medical images, are among the most important and practical approaches in numerous clinical applications. Various conventional image fusion techniques have been developed for multimodality image fusion. Complex procedures for weight map computing, fixed fusion strategy and lack of contextual understanding remain difficult in conventional and machine learning approaches, usually resulting in artefacts that degrade the image quality. This work proposes an efficient hybrid learning model for medical image fusion using pre-trained and non-pre-trained networks i.e. VGG-19 and SNN with stacking ensemble method. The model leveraging the unique capabilities of each architecture, can effectively preserve the detailed information with high visual quality, for numerous combinations of image modalities in image fusion challenges, notably improved contrast, increased resolution, and lower artefacts. Additionally, this ensemble model can be more robust in the fusion of various combinations of source images that are publicly available from Havard-Medical-Image-Fusion Datasets, GitHub. and Kaggle. Our proposed model performance is superior in terms of visual quality and performance metrics to that of the existing fusion methods in literature like PCA+DTCWT, NSCT, DWT, DTCWT+NSCT, GADCT, CNN and VGG-19.

Clinical and Genetic characteristics of patients with Peripheral Retinal Fleck in Koreans.

To describe the clinical and genetic features of Korean patients with peripheral retinal flecks unrelated to aging. A retrospective analysis was conducted on the clinical characteristics of patients with symmetric peripheral retinal flecks. Age-related deposits such as reticular pseudodrusen were excluded, as well as secondary deposits related to intraocular inflammation, tumor, and drug toxicity. Multimodal imaging, electrophysiological examinations, and genetic testing were analyzed. A total of 10 patients (2 males and 8 females) with bilateral peripheral flecks were enrolled in this study. A mean age at diagnosis was 30.5±19.6 years (4-59 years). Within the 10 patients, 6 were genetically confirmed with monogenic retinal disorders. Biallelic pathogenic variants in RDH5 were found in 5 patients, and 1 patient was diagnosed with retinopathy related to Alport syndrome due to a pathogenic variant in COL4A5. Although not genetically confirmed, one case associated with nanophthalmos and another case showing chorioretinal mottling in a carrier of ocular albinism have been identified. In one patient, genetic testing also revealed unknown causes. The mean logMAR initial visual acuity was 0.12±0.18 and 0.07±0.18 in right and left eyes, respectively. Night blindness was reported by 4 patients (40%), with 3 showing decreased or delayed rod response in electroretinogram, particularly those with RDH5 mutations. Differences in the deposit layers and the patterns of flecks were observed on multimodal imaging. In the study population, we observed various causes and clinical differences in the retinal fleck patterns among Koreans, including RDH5-related fundus albipunctatus and Alport syndrome. Despite reports of night blindness symptoms in some cases, all patients demonstrated satisfactory corrected visual acuity.

Erosion of the temporal bone by vestibular schwannoma: morphometrics and predictive modeling.

To perform a comprehensive morphometric analysis of vestibular schwannomas (VS) using multimodal imaging, focusing on the relationship between tumor characteristics and internal acoustic canal (IAC) changes. We analyzed a cohort of patients undergoing radiosurgery for VS, utilizing high-definition MRI and bone CT for detailed anatomical assessment. Image co-registration and fusion techniques were employed to examine VS and IAC dimensions. Advanced statistical methods, including logistic regression, were applied to identify patterns of IAC dilation and establish predictive indicators for these morphological changes. The study included 659 patients (51.1% female, mean age 56.37years) with evenly distributed tumor lateralization. Koos grades were I (22.9%), II (29.9%), III (38.2%), IVa (8.9%), and IVb (0.3%). Most tumors (90.9%) extended both inside and outside the IAC. Ipsilateral IAC (IIAC) dimensions were significantly larger than contralateral, with IIAC volume 49% greater (p < .0001). Higher Koos grades correlated with increased intra-canalicular lesion volume (icLV), which was strongly associated with IIAC size. Logistic regression identified icLV as the strongest predictor of IIAC dilation (AUC = 0.7674, threshold = 137.52 mm3). The icLV appears central to the pathophysiological development of VS and its impact on IAC anatomy. While limited by a selective patient base and static imaging data, these findings enhance the understanding of VS-IAC interactions, offering insights for improved clinical management and further research.

Prediction of epileptogenicity in patients with tuberous sclerosis complex using multimodal cerebral MRI

ObjectiveEpilepsy is the most common complication and cause of morbidity and mortality in tuberous sclerosis complex (TSC). Surgery is associated with an increased probability of achieving seizure-freedom. However, the preoperative noninvasive localisation of epileptogenic tubers remains challenging. This study aimed to identify multimodal magnetic resonance imaging (MRI) biomarkers of epilepsy in patients with TSC and develop a prediction model of epileptogenicity in these patients. MethodsPatients with TSC, with or without epilepsy, were recruited. All patients underwent MRI scanning, including T1WI, T2WI, T2W-FLAIR, DTI, and multi-parametric MR with a flexible design (MULTIPLEX). We compared the multimodal cerebral MRI characteristics of the cortical tubers, subependymal nodules, and perilesional tissue between patients with TSC with or without epilepsy and developed a prediction model for epileptogenicity. ResultsAmong the patients with TSC, 32 with and 16 without epilepsy underwent MRI. Higher proton-density mapping (PD) of cortical tubers and decreased fractional anisotropy (FA), increased mean diffusivity (MD), and increased radial diffusivity (RD) of subependymal nodules were associated with epileptogenicity in both the centre and perilesional tissue, independent of TSC gene variation. Based on the above findings, we developed a prediction model for epileptogenicity with an area under the curve of 0.973, specificity of 0.909, and sensitivity of 0.963 (P < 0.001). ConclusionIn patients with TSC, high PD of the cortical tubers, decreased FA, and elevated MD/RD of the subependymal nodules were significantly associated with epileptogenicity. A prediction model based on multimodal cerebral MRI characteristics has the potential to evaluate the likelihood of epilepsy in patients with TSC.

Application of nanoliposome as contrast agents for magnetic resonance imaging technique

AbstractLiposomes, nano‐sized vesicles primarily comprising phospholipids and cholesterol, have emerged as pivotal tools in medical imaging, notably in magnetic resonance imaging (MRI), due to their biocompatibility and ability to encapsulate diverse molecules. Tailorable properties like size, surface charge, and encapsulation capacity make liposomes ideal for targeted delivery of imaging agents and drugs to specific tissues, improving pharmacokinetics. As MRI contrast agent (CA) carriers, liposomes encapsulate gadolinium, mitigating toxicity and boosting relaxivity and circulation times. Functionalization with targeting ligands and stimuli‐responsive designs enhances their controlled release and targeted delivery capabilities, crucial for cancer imaging and therapy. Benefits include reduced toxicity, prolonged circulation, targeted delivery, enhanced bioavailability, and potential for multimodal imaging. Challenges remain, such as stability, clearance, and manufacturing intricacies, requiring further research. Nonetheless, liposomal MRI CAs hold promise for enhancing diagnostic precision and therapeutic effectiveness in oncology and neurology, offering a robust pathway for future biomedical advancements. Addressing existing limitations could unlock their full potential in improving patient care and outcomes.

Clinical Characteristics and Multi-Model Imaging Analysis of Moyamoya Disease: An Observational Study.

Previous studies have lacked a comprehensive analysis of imaging modalities for diagnosing Moyamoya disease (MMD). This study aims to bridge this gap by utilizing multi-modal imaging to provide a more detailed understanding of the clinical and imaging characteristics of MMD. A retrospective analysis was conducted on seventy-eight adult MMD patients enrolled from March 2018 to March 2021. The study focused on clinical features, imaging findings, and treatment outcomes, with a particular emphasis on the comparative efficacy of different imaging modalities. In this series, clinical manifestations varied depending on the type of MMD, with intracerebral hemorrhage (ICH) being the most common (69.2%), followed by cerebral infarction (25.6%). Imaging techniques provided critical diagnostic insights: magnetic resonance imaging (MRI) demonstrated superior sensitivity over computed tomography (CT) in detecting hemorrhages, whereas computed tomography angiography (CTA) and digital subtraction angiography (DSA) identified intricate vascular lesions, including moyamoya vessels and aneurysms. Notably, cerebral perfusion imaging (CTP) highlighted significant differences in cerebral blood flow and volume between infarction and hemorrhage cases. This comprehensive imaging approach guided varied therapeutic strategies, including bypass surgery in 57 patients and interventional embolization for aneurysms in 14 patients. The authors' findings underscore the critical role of early diagnosis using DSA, whereas highlighting CTA and MRA as valuable noninvasive tools for screening and follow-up. The integration of multi-modal imaging provides a detailed vascular assessment crucial for individualized patient management, facilitating timely interventions and significantly improving clinical outcomes.

Multi-Modal Object Detection Method Based on Dual-Branch Asymmetric Attention Backbone and Feature Fusion Pyramid Network

With the simultaneous acquisition of the infrared and optical remote sensing images of the same target becoming increasingly easy, using multi-modal data for high-performance object detection has become a research focus. In remote sensing multi-modal data, infrared images lack color information, it is hard to detect difficult targets with low contrast, and optical images are easily affected by illuminance. One of the most effective ways to solve this problem is to integrate multi-modal images for high-performance object detection. The challenge of fusion object detection lies in how to fully integrate multi-modal image features with significant modal differences and avoid introducing interference information while taking advantage of complementary advantages. To solve these problems, a new multi-modal fusion object detection method is proposed. In this paper, the method is improved in terms of two aspects: firstly, a new dual-branch asymmetric attention backbone network (DAAB) is designed, which uses a semantic information supplement module (SISM) and a detail information supplement module (DISM) to supplement and enhance infrared and RGB image information, respectively. Secondly, we propose a feature fusion pyramid network (FFPN), which uses a Transformer-like strategy to carry out multi-modal feature fusion and suppress features that are not conducive to fusion during the fusion process. This method is a state-of-the-art process for both FLIR-aligned and DroneVehicle datasets. Experiments show that this method has strong competitiveness and generalization performance.

Multimodality imaging of pulmonary artery aneurysms due to giant cell arteritis: a case series.

Pulmonary artery aneurysms (PAAs) are rare, but important vascular phenomena with nonspecific signs and symptoms. While some patients remain asymptomatic, others manifest with serious symptoms such as chest pain, dyspnea, or hemoptysis. Often diagnosed incidentally, PAAs have various underlying etiologies, including congenital, acquired, and idiopathic. Giant cell arteritis (GCA) is a very rare cause of PAAs, with limited reported cases in the literature. This case series will review the clinical presentation, multimodality imaging, and management of four patients with isolated PAA due to GCA.

Brain Tumor Detection through Image Fusion Using Cross Guided Filter and Convolutional Neural Network

This Data fusion has become a significant issue in diagnostic imaging, particularly in medical applications like radiation and guided image surgery. Medical image fusion aims to enhance the precision of tumor diagnosis, by preserving the salient information and characteristics of the original images in the fused image. It has been shown that guided filters are capable of maintaining edges well. In this paper, we propose a novel cross-guided filter-based fusion approach for multimodal medical images utilizing convolutional neural networks. The cross-guided filter is used in the proposed algorithm to extract the detailed features from the source images. Convolutional neural networks are used to generate the feature weights of source images derived from the detail layers. The weighted average rule is used to merge the source images based on these weights. We used thirty distinct types of medical images from diverse sources to compare the effectiveness of the proposed strategy to that of existing methods, both numerically and visually. The experimental findings demonstrated that, in terms of both objective evaluation and qualitative image quality, the suggested system performs better than other standard methods already in use. The quantitative results show that compared to existing methods under consideration for comparison, the proposed algorithm improves mutual information by 25%, image entropy by 9.5%, spatial frequency by 21%, standard deviation by 18.1%, structural similarity index by 30%, and edge strength of the fused image by 39%.

MMCL: Meta-mutual contrastive learning for multi-modal medical image fusion

The number of datasets and computational efficiency are always hindrances in the multi-modal medical image fusion (MMIF) research. To address these challenges, we propose a contrastive learning framework inspired meta-mutual, which divides the medical image fusion task into subtasks and pre-trains an optimal meta-representation suitable for all subtasks. We then fine-tune our proposed network using this optimal meta-representation as initialization, achieving the best model with only a few short datasets. Additionally, extracting source image features in pairs can lead to redundant information due to the invariant and unique features of multi-modal images. Therefore, we introduce novelty mutual contrastive coupled pairs to extract both invariant and unique features from source images. Experimental results demonstrate that our method outperforms other state-of-the-art fusion methods.