Multimodal Imaging Research Articles

Initial investigation on ultrasound-guided percutaneous biopsy of lesions in the first hepatic hilum with fusion of ultrasound and multimodal imaging cognitive guidance.

This study aims to evaluate the efficacy and safety of ultrasound-guided percutaneous biopsy of the first hepatic hilum lesion, and examine its clinical value of diagnosis and treatment. We conducted a retrospective study on patients diagnosed with the first hepatic hilum lesions at Fujian Provincial Hospital between February 2015 and October 2022. We selected patients who had lesions in the first hepatic hilum(including a 2cm surrounding area of the left/right hepatic ducts and upper-middle segment of the common bile duct) and the liver periphery(in the peripheral area of the liver, outside of the above-mentioned first hepatic porta region). These patients underwent percutaneous ultrasound-guided core needle biopsy (PUS-CNB) with cognitive fusion guidance using CT, MRI, or PET-CT. We compared the safety and efficacy of PUS-CNB in the first hepatic hilum and the liver periphery to explore the value of PUS-CNB in optimizing the clinical treatment of the first hepatic hilum lesions. The studied includes 38 cases of the first hepatic hilum cases (18 females; 20 males), 23 presented with mass-forming tumors while the remaining 15 exhibited diffuse infiltrative tumors, with an average diameter of 4.65± 2.51 cm. The percutaneous biopsy procedure, conducted under ultrasound guidance, had an average operation time of 14.55 ± 2.73 minutes, and resulted in a postoperative bleeding volume of approximately 10.79 ± 2.79 ml. The diagnostic success rate was noted to be as high as 92.11% among the participants who underwent percutaneous biopsy of the first hepatic hilum. Procedural complications, such as bleeding, bile leakage, intestinal perforation, infection or needle tract seeding, did not occur during or after the biopsy procedure. Affected by biopsy results, 5 altered their clinical treatment plans accordingly, 24patients received non-surgical treatment, 9 underwent surgical treatment, 5 underwent radiofrequency ablation for the lesions. The study comprised a total of 112 cases for percutaneous biopsy of the liver periphery. The safety and effectiveness of the two biopsy techniques were comparable, with diagnostic success rates of 92.11% VS. 94.34%, respectively (p = 0.61). Cognitive fusion of ultrasound and multi-modal imaging for the first hepatic hilum lesion puncture biopsy is a safe and effective diagnostic procedure, with better diagnostic rate, may improve clinical value of diagnosis and treatment of various diseases.

Central Serous Chorioretinopathy in Nanophthalmos

Purpose: To report a case of bilateral central serous chorioretinopathy (CSC) occurring in a patient with nanophthalmos with long term follow up Methods: Case Report Results: A 37-year old male presented with diminution of vision in right eye (RE) since 3 days. On examination his best corrected visual acuity (BVCA) was 5/60 in RE and 6/18 in LE His refractive correction was +8.00D sphere. Anterior segment examination revealed small eyes with narrow palpebral fissures, shallow anterior chamber and clear lens. His axial length was 15.94mm in the RE and 15.89mm in the LE respectively. Fundus examination revealed a crowded disc with tortuous vessels in both eyes and the presence of subretinal fluid (SRF) in the macular area in the RE. Multimodal imaging included enhanced depth imaging spectral domain optical coherence tomography (EDI SD-OCT) and fundus fluorescein angiography (FFA) which revealed features suggestive of thick choroids BE, ink-blot leak on FFA suggestive of acute CSC in the RE and a pigment epithelial detachment (PED) in the left eye. Focal laser in RE lead to SRF resolution and visual improvement. He was on regular follow up and 5 years later he presented with a recurrence of CSC in the RE for which he underwent a repeat FFA guided focal laser. Conclusion: This case highlights the unique coexistence of CSC in the setting of Nanophthalmos and highlights a probable overlapping pathophysiology of choroidal venous overload as well as the importance of multimodal imaging in differentiating the mimicking entities.

SRTRP-Net: A multi-task learning network for segmentation and prediction of stereotactic radiosurgery treatment response in brain metastases

Before the Stereotactic Radiosurgery (SRS) treatment, it is of great clinical significance to avoid secondary genetic damage and guide the personalized treatment plans for patients with brain metastases (BM) by predicting the response to SRS treatment of brain metastatic lesions. Thus, we developed a multi-task learning model termed SRTRP-Net to provide prior knowledge of BM ROI and predict the SRS treatment response of the lesion. In dual-encoder tumor segmentation Network (DTS-Net), two parallel encoders encode the original and mirrored multi-modal MRI images. The differences in the dual-encoder features between foreground and background are enhanced by the symmetrical visual difference block (SVDB). In the bottom layer of the encoder, a transformer is used to extract local contextual features in the spatial and depth dimensions of low-resolution images. Then, the decoder of DTS-Net provides the prior knowledge for predicting the response to SRS treatment by performing BM segmentation. SRS response prediction network (SRP-Net) directly utilizes shared multi-modal MRI features weighted by the signed distance map (SDM) of the masks. The bidirectional multi-dimensional feature fusion module (BMDF) fuses the shared features and the clinical text information features to obtain comprehensive tumor information for characterizing tumors and predicting SRS treatment response. Experiments based on internal and external clinical datasets have shown that SRTRP-Net achieves comparable or better results. We believe that SRTRP-Net can help clinicians accurately develop personalized first-time treatment regimens for BM patients and improve their survival.

Multi-modal medical image fusion using improved dual-channel PCNN.

This paper proposes a medical image fusion method in the non-subsampled shearlet transform (NSST) domain to combine a gray-scale image with the respective pseudo-color image obtained through different imaging modalities. The proposed method applies a novel improved dual-channel pulse-coupled neural network (IDPCNN) model to fuse the high-pass sub-images, whereas the Prewitt operator is combined with maximum regional energy (MRE) to construct the fused low-pass sub-image. First, the gray-scale image and luminance of the pseudo-color image are decomposed using NSST to find the respective sub-images. Second, the low-pass sub-images are fused by the Prewitt operator and MRE-based rule. Third, the proposed IDPCNN is utilized to get the fused high-pass sub-images from the respective high-pass sub-images. Fourth, the luminance of the fused image is obtained by applying inverse NSST on the fused sub-images, which is combined with the chrominance components of the pseudo-color image to construct the fused image. A total of 28 diverse medical image pairs, 11 existing methods, and nine objective metrics are used in the experiment. Qualitative and quantitative fusion results show that the proposed method is competitive with and even outpaces some of the existing medical fusion approaches. It is also shown that the proposed method efficiently combines two gray-scale images.

Delayed Diagnosis of Right Heart Failure Due to Sinus Venosus Atrial Septal Defect, Persistent Left Superior Vena Cava and Partial Anomalous Pulmonary Venous Return

Sinus venosus atrial septal defects (SVASD) are uncommon and often underdiagnosed as they are visualised poorly on transthoracic echocardiography (TTE). A 59-year-old man was referred following worsening dyspnoea. A TTE demonstrated dilated right chambers, with torrential tricuspid regurgitation and a dilated coronary sinus. Agitated saline test raised the suspicion of both persistent left superior vena cava (PLSVC) and an intracardiac shunt, despite interrogation of the interatrial septum not demonstrating any defects. CT imaging of the thorax was performed revealing a superior SVASD alongside a PLSVC, and partial anomalous pulmonary venous return. SVASD are considerably uncommon, compared to other types of atrial septal defects (ASD). TTE has poor sensitivity in diagnosing SVASD, highlighting the importance of multimodality imaging in such cases. The purpose of this case report is to highlight unique features demonstrated on TTE, especially during the agitated saline study, which showcased the presence of a PLSVC alongside an intracardiac shunt that was undetected through conventional TTE alone.

Application of high intensity focused ultrasound combined with nanomaterials in anti-tumor therapy

High intensity focused ultrasound (HIFU) has demonstrated its safety, efficacy and noninvasiveness in the ablation of solid tumor. However, its further application is limited by its inherent deficiencies, such as postoperative recurrence caused by incomplete ablation and excessive intensity affecting surrounding healthy tissues. Recent research has indicated that the integration of nanomaterials with HIFU exhibits a promising synergistic effect in tumor ablation. The concurrent utilization of nanomaterials with HIFU can help overcome the limitations of HIFU by improving targeting and ablation efficiency, expanding operation area, increasing operation accuracy, enhancing stability and bio-safety during the process. It also provides a platform for multi-therapy and multi-mode imaging guidance. The present review comprehensively expounds upon the synergistic mechanism between nanomaterials and HIFU, summarizes the research progress of nanomaterials as cavitation nuclei and drug carriers in combination with HIFU for tumor ablation. Furthermore, this review highlights the potential for further exploration in the development of novel nanomaterials that enhance the synergistic effect with HIFU on tumor ablation.

On the Actuation Technologies of Biomedical Microrobot: A Summarized Review

In recent years, wireless microrobots have gotten more attention due to their huge potential in the biomedical field, especially drug delivery. Microrobots have several benefits, including small size, low weight, sensitivity, and flexibility. These characteristics have led to microscale improvements in control systems and power delivery with the development of submillimeter-sized robots. Wireless control of individual mobile microrobots has been achieved using a variety of propulsion systems, and improving the actuation and navigation of microrobots will have a significant impact. On the other hand, actuation tools must be integrated and compatible with the human body to drive these untethered microrobots along predefined paths inside biological environments. This study investigated key microrobot components, including medical applications, actuation systems, control systems, and design schemes. The efficiency of a microrobot is impacted by many factors, including the material, structure, and environment of the microrobot. Furthermore, integrating a hybrid actuation system and multimodal imaging can increase the microrobot's navigation effect, imaging algorithms, and working environment. In addition, taking into account the human body's moving distance, autonomous actuating technology could be used to deliver microrobots precisely and quickly to a specific position using a combination of quick approaches.

Multimodal imaging of optic nerve head abnormalities in high myopia.

Highly myopic optic nerve head (ONH) abnormalities encompass a series of complications resulting from the stretching of papillary and peripapillary structures during significant axial elongation. The morphological changes in the ONH typically initiate with disk tilting or rotation, progressing to PHOMS and PPA. Tissue defects in each layer manifest as focal lamina cribrosa defects (FLDs), peripapillary intrachoroidal cavitations (PICCs), and acquired pits of the optic nerve (APON). Anterior vitreous/vascular traction and posterior scleral protrusion may lead to prelaminar schisis as well as paravascular cysts and holes, which can potentially develop into retinoschisis. Traditional color fundus photography (CFP) is often insufficient for visualizing most of these lesions, yet their description and quantification benefit significantly from the advancements in optical coherence tomography (OCT) and OCT angiography (OCTA), complemented by fundus autofluorescence (FAF), indocyanine green angiography (ICGA), and three-dimensional imaging. The effective diagnosis and classification of ONH abnormalities heavily rely on a comprehensive understanding of their multimodal imaging features, as outlined in this review. These findings provide valuable insights into optic neuropathy in high myopia, establishing a solid foundation for future endeavors in disease monitoring and treatment guidance.

Predicting 2-year neurodevelopmental outcomes in preterm infants using multimodal structural brain magnetic resonance imaging with local connectivity

The neurodevelopmental outcomes of preterm infants can be stratified based on the level of prematurity. We explored brain structural networks in extremely preterm (EP; < 28 weeks of gestation) and very-to-late (V-LP; ≥ 28 and < 37 weeks of gestation) preterm infants at term-equivalent age to predict 2-year neurodevelopmental outcomes. Using MRI and diffusion MRI on 62 EP and 131 V-LP infants, we built a multimodal feature set for volumetric and structural network analysis. We employed linear and nonlinear machine learning models to predict the Bayley Scales of Infant and Toddler Development, Third Edition (BSID-III) scores, assessing predictive accuracy and feature importance. Our findings revealed that models incorporating local connectivity features demonstrated high predictive performance for BSID-III subsets in preterm infants. Specifically, for cognitive scores in preterm (variance explained, 17%) and V-LP infants (variance explained, 17%), and for motor scores in EP infants (variance explained, 15%), models with local connectivity features outperformed others. Additionally, a model using only local connectivity features effectively predicted language scores in preterm infants (variance explained, 15%). This study underscores the value of multimodal feature sets, particularly local connectivity, in predicting neurodevelopmental outcomes, highlighting the utility of machine learning in understanding microstructural changes and their implications for early intervention.

Multimodal in-vivo maps as a tool to characterize retinal structural biomarkers for progression in adult-onset Stargardt disease

PurposeTo characterize retinal structural biomarkers for progression in adult-onset Stargardt disease from multimodal retinal imaging in-vivo maps.MethodsSeven adult patients (29–69 years; 3 males) with genetically-confirmed and clinically diagnosed adult-onset Stargardt disease and age-matched healthy controls were imaged with confocal and non-confocal Adaptive Optics Scanning Light Ophthalmoscopy (AOSLO), optical coherence tomography (OCT), fundus infrared (FIR), short wavelength-autofluorescence (FAF) and color fundus photography (CFP). Images from each modality were scaled for differences in lateral magnification before montages of AOSLO images were aligned with en-face FIR, FAF and OCT scans to explore changes in retinal structure across imaging modalities. Photoreceptors, retinal pigment epithelium (RPE) cells, flecks, and other retinal alterations in macular regions were identified, delineated, and correlated across imaging modalities. Retinal layer-thicknesses were extracted from segmented OCT images in areas of normal appearance on clinical imaging and intact outer retinal structure on OCT. Eccentricity dependency in cell density was compared with retinal thickness and outer retinal layer thickness, evaluated across patients, and compared with data from healthy controls.ResultsIn patients with Stargardt disease, alterations in retinal structure were visible in different image modalities depending on layer location and structural properties. The patients had highly variable foveal structure, associated with equally variable visual acuity (-0.02 to 0.98 logMAR). Cone and rod photoreceptors, as well as RPE-like structures in some areas, could be quantified on non-confocal split-detection AOSLO images. RPE cells were also visible on dark field AOSLO images close to the foveal center. Hypo-reflective gaps of non-waveguiding cones (dark cones) were seen on confocal AOSLO in regions with clinically normal CFP, FIR, FAF and OCT appearance and an intact cone inner segment mosaic in three patients.ConclusionDark cones were identified as a possible first sign of retinal disease progression in adult-onset Stargardt disease as these are observed in retinal locations with otherwise normal appearance and outer retinal thickness. This corroborates a previous report where dark cones were proposed as a first sign of progression in childhood-onset Stargardt disease. This also supports the hypothesis that, in Stargardt disease, photoreceptor degeneration occurs before RPE cell death.

Multimodality Imaging of a Patient with Intra-Articular Osteoid Osteoma: A Case Report

Multimodality Imaging of a Patient with Intra-Articular Osteoid Osteoma: A Case Report

Atypical case of choroidal osteoma associated to polypoidal choroidal vasculopathy and preretinal neovascular membrane.

To report a very rare and atypical case of an elderly Caucasian female patient who developed perilesional multiple polypoidal choroidal vasculopathy (PCV) as a probable complication of choroidal osteoma (CO), associated to preretinal neovascular membrane overlying the lesion. Observational case report. A 60-year-old Caucasian woman presented with blurred vision in her right eye (RE). Fundus examination revealed a round white-yellowish calcified deep lesion in the juxta-papillary superior area, measuring 4 disc-diameters, with well-defined scalloped margins and an irregular surface. B-scan ultrasonography and orbital tomography confirmed the diagnosis of choroidal osteoma (CO). Further investigation with multimodal imaging including infracyanine green angiography, fluorescein angiography, swept source optical coherence tomography and angiography highlighted the presence of multiple aneurysmal choroidal dilations around the CO, corresponding to PCV. We also noted the presence of a preretinal neovascular membrane overlying the CO. The patient was monitored with regular follow-up since no signs of activity were detected on multimodal imaging. Our case report represents an exceptional and atypical association between pre-retinal neovascularization, PCV and choroidal osteoma. While the mechanisms underlying the development of PCV and pre-retinal neovascularization in the setting of CO are not well understood, it is imperative for ophthalmologists to recognize this association as a potential cause of sudden vision loss in patients with CO, and to consider appropriate diagnostic and management strategies.

Linking white matter hyperintensities to regional cortical thinning, amyloid deposition, and synaptic density loss in Alzheimer's disease.

We investigated the association between white matter hyperintensities (WMH) and regional cortical thickness, amyloid and tau deposition, and synaptic density in the WMH-connected cortex using multimodal images. We included 107 participants (59 with Alzheimer's disease [AD]; 27 with mild cognitive impairment; 21 cognitively normal controls) with amyloid beta (Aβ) positivity on amyloid positron emission tomography (PET). The cortex connected to WMH was identified using probabilistic tractography. We found that WMH connected to the cortex with more severe regional degeneration as measured by cortical thickness, Aβ and tau deposition, and synaptic vesicle glycoprotein 2 A (SV2A) density using 18F-SynVesT-1 PET. In addition, higher ratios of Aβ in the deep WMH-connected versus WMH-unconnected cortex were significantly related to lower cognitive scores. Last, the cortical thickness of WMH-connected cortex reduced more than WMH-unconnected cortex over 12 months. Our results suggest that WMH may be associated with AD-intrinsic processes of degeneration, in addition to vascular mechanisms. We studied white matter hyperintensities (WMHs) and WMH-connected cortical changes. WMHs are associated with more severe regional cortical degeneration. Findings suggest WMHs may be associated with Alzheimer's disease-intrinsic processes of degeneration.

Glioma segmentation based on dense contrastive learning and multimodal features recalibration

Accurate segmentation of different regions of gliomas from multimodal magnetic resonance (MR) images is crucial for glioma grading and precise diagnosis, but many existing segmentation methods are difficult to effectively utilize multimodal MR image information to recognize accurately the lesion regions with small size, low contrast and irregular shape. To address this issue, this work proposes a novel 3D glioma segmentation model DCL-MANet. DCL-MANet has an architecture of multiple encoders and one single decoder. Each encoder is used to extract MR image features of a given modality. To overcome the entangle problems of multimodal semantic features, a dense contrastive learning (DCL) strategy is presented to extract the modality-specific and common features. Following that, feature recalibration block (RFB) based on modality-wise attention is used to recalibrate the semantic features of each modality, enabling the model to focus on the features that are beneficial for glioma segmentation. These recalibrated features are input into the decoder to obtain the segmentation results. To verify the superiority of the proposed method, we compare it with several state-of-the-art (SOTA) methods in terms of Dice, average symmetric surface distance (ASSD), HD95 and volumetric similarity (Vs). The comparison results show that the average Dice, ASSD, HD95 and Vs of DCL-MANet on all tumor regions are improved at least by 0.66%, 3.47%, 8.94% and 1.07% respectively. For small enhance tumor (ET) region, the corresponding improvement can be up to 0.37%, 7.83%, 11.32%, and 1.35%, respectively. In addition, the ablation results demonstrate the effectiveness of the proposed DCL and RFB, and combining them can significantly increase Dice (1.59%) and Vs (1.54%) while decreasing ASSD (40.51%) and HD95 (45.16%) on ET region. The proposed DCL-MANet could disentangle multimodal features and enhance the semantics of modality-dependent features, providing a potential means to accurately segment small lesion regions in gliomas.

Multimodal Cerebral Imaging is required to Detect Stroke-like Lesions in NDUFS8 Mutation Carriers

Multimodal Cerebral Imaging is required to Detect Stroke-like Lesions in NDUFS8 Mutation Carriers

A novel homozygous splice site variant in ARL2BP causes a syndromic autosomal recessive rod-cone dystrophy with situs inversus, asthenozoospermia, unilateral renal agenesis and microcysts

BackgroundThis report presents a clinical case of syndromic rod-cone dystrophy due to a splice site variant in the ARL2BP gene causing situs inversus, asthenozoospermia, unilateral renal agenesis and microcysts. The presence of renal agenesis and cryptorchidism expands the clinical manifestations due to ARL2BP variants. The detailed, long-term follow-up contributes valuable insights into disease progression, aiding clinical diagnosis and patient management.Case PresentationThe male patient complained of photophobia as the first symptom when he was 20 years old followed by nyctalopia, loss of central visual acuity and peripheral visual field ten years later. Genetic analysis identified a likely pathogenic homozygous variant (c.294-1G > C) involving the splicing acceptor site of intron 4. Reported symptoms together with full-field stimulus threshold testing, electroretinogram and advanced multimodal imaging allowed us to recognize the typical characteristics of a mixed retinal dystrophy. Despite the end-stage retinal disease, this patient still retained a useful residual vision at 63 years and had a slow disease progression during the last 5 years of evaluation.Discussion and conclusionsOur findings underscore the variable clinical presentation of ARL2BP variants, emphasizing the importance of a nuanced approach in diagnosing and managing patients. The presence of renal cysts warrants consideration of a differential diagnosis, particularly with Senior-Loken (SLS), Bardet-Biedl (BBS) and Joubert syndromes (JS) but also with Short Rib Thoracic Dysplasia 9, highlighting the need for careful phenotypic evaluation in these cases.

3D medical image segmentation based on semi-supervised learning using deep co-training

In recent years, artificial intelligence has been applied to 3D COVID-19 medical image diagnosis, which reduces detection costs and missed diagnosis rates with higher predictive accuracy, and diagnostic efficiency. However, the limited size and low quality of clinical 3D medical image samples have hindered the segmentation performance of 3D models. Therefore, we propose a 3D medical image segmentation model based on semi-supervised learning using co-training. Multi-view and multi-modal images are generated using spatial flipping and windowing techniques to enhance the spatial diversity of 3D image samples. A pseudo label generation module based on confidence-weights is employed to generate reliable pseudo labels for non-annotated data, thereby increasing the sample size and reducing overfitting. The proposed approach utilizes a three-stage training process: firstly, training a single network based on annotated data; secondly, incorporating non-annotated data to train a dual-modal network and generate pseudo labels; finally, jointly training six models in three dimensions using both annotated and pseudo labels generated from multi-view and multi-modal images, aiming to enhance segmentation accuracy and generalization performance. Additionally, a consistency regularization loss is applied to reduce noises and accelerate convergence of the training. Moreover, a heatmap visualization method is employed to focus on the attention of features at each stage of training, providing effective reference for clinical diagnosis. Experiments were conducted on an open dataset of 3D COVID-19 CT samples and a non-annotated dataset from TCIA, including 771 NIFTI-format CT images from 661 COVID-19 patients. The results of 5-fold cross-validation show that the proposed model achieves a segmentation accuracy of Dice=73.30 %, ASD=10.633, Sensitivity=63.00 %, and Specificity=99.60 %. Compared to various typical semi-supervised learning 3D segmentation models, it demonstrates better segmentation accuracy and generalization performance.

From sensor fusion to knowledge distillation in collaborative LIBS and hyperspectral imaging for mineral identification

Multimodal spectral imaging offers a unique approach to the enhancement of the analytical capabilities of standalone spectroscopy techniques by combining information gathered from distinct sources. In this manuscript, we explore such opportunities by focusing on two well-known spectral imaging techniques, namely laser-induced breakdown spectroscopy, and hyperspectral imaging, and explore the opportunities of collaborative sensing for a case study involving mineral identification. In specific, the work builds upon two distinct approaches: a traditional sensor fusion, where we strive to increase the information gathered by including information from the two modalities; and a knowledge distillation approach, where the Laser Induced Breakdown spectroscopy is used as an autonomous supervisor for hyperspectral imaging. Our results show the potential of both approaches in enhancing the performance over a single modality sensing system, highlighting, in particular, the advantages of the knowledge distillation framework in maximizing the potential benefits of using multiple techniques to build more interpretable models and paving for industrial applications.

Laser ultrasonic frequency-domain imaging and phase weighted optimization based on full matrix capture

Far-field laser technology has greatly promoted the progress of nondestructive ultrasonic imaging of bulk structures. However, under thermoelastic excitation, the body waves exhibit a relatively low signal-to-noise ratio, resulting in images with low resolution and contrast. Based on the motivation, this paper developed a frequency-domain phase weighted imaging method to improve the quality of laser ultrasonic defect imaging. Firstly, laser ultrasonic scanning was performed on the sample with artificial transverse hole defects. The cylindrical lens focused line source was used to improve the intensity of the body wave signals, and ensure that there was no damage on the material surface under high laser energies. Then, the frequency-domain phase shift migration (PSM) algorithm was used to perform multimode imaging of defects, achieving frequency-domain synthetic aperture focusing technique (F-SAFT) and total focused method (F-TFM) imaging based on full matrix capture. Furthermore, the phase circular statistical vector (PCSV) was proposed for weighted optimization, which improved the image quality, suppressed the background noise and multimode artifacts. Finally, the imaging quality of several algorithms were discussed. The results indicate that frequency-domain images were superior to time-domain results. After phase weighting, the imaging quality can be further improved, and the detection blind zone was significantly reduced. This work will contribute to the rapid and high-quality defect imaging of laser ultrasonic.

MACTFusion: Lightweight Cross Transformer for Adaptive Multimodal Medical Image Fusion.

Multimodal medical image fusion aims to integrate complementary information from different modalities of medical images. Deep learning methods, especially recent vision Transformers, have effectively improved image fusion performance. However, there are limitations for Transformers in image fusion, such as lacks of local feature extraction and cross-modal feature interaction, resulting in insufficient multimodal feature extraction and integration. In addition, the computational cost of Transformers is higher. To address these challenges, in this work, we develop an adaptive cross-modal fusion strategy for unsupervised multimodal medical image fusion. Specifically, we propose a novel lightweight cross Transformer based on cross multi-axis attention mechanism. It includes cross-window attention and cross-grid attention to mine and integrate both local and global interactions of multimodal features. The cross Transformer is further guided by a spatial adaptation fusion module, which allows the model to focus on the most relevant information. Moreover, we design a special feature extraction module that combines multiple gradient residual dense convolutional and Transformer layers to obtain local features from coarse to fine and capture global features. The proposed strategy significantly boosts the fusion performance while minimizing computational costs. Extensive experiments, including clinical brain tumor image fusion, have shown that our model can achieve clearer texture details and better visual quality than other state-of-the-art fusion methods.