Multimodal Image Fusion Research Articles

Simulation of sculpture virtual design system based on multimodal image fusion and intelligent manufacturing

Simulation of sculpture virtual design system based on multimodal image fusion and intelligent manufacturing

Enhanced Preoperative Planning for Intracranial Aneurysms Through Multimodal Image Fusion of Silent/Time-of-Magnetic Resonance Angiography and Computed Tomography Using 3DSlicer: A Comparative Efficacy Analysis With Computed Tomography Angiography.

This study evaluates the effectiveness of multimodal image fusion (MIF) using silent and time-of-flight (TOF) magnetic resonance angiography (MRA) and computed tomography (CT) for preoperative planning in patients with intracranial aneurysms who have contraindications to contrast media. A retrospective study included 40 patients with intracranial aneurysms, diagnosed using three-dimensional computed tomography angiography (CTA). These patients underwent both Silent and TOF MRA scans, followed by a CTA scan. The multi-image fusion (MIF) technique, applied using 3DSlicer software, integrated the silent/TOF-MRA with CT images for preoperative assessment. This study compared the image quality, aneurysm detection sensitivity, and anatomic accuracy of the MIF images with those of three-dimensional CTA. Silent-MRA-CT fusion images demonstrated higher sensitivity (95.5%) and lower false negative rates (4.5%) compared with TOF-MRA-CT. Furthermore, silent-MRA-CT fusion images outperformed TOF-MRA-CT in terms of signal homogeneity, venous signal interference suppression, and aneurysm visibility (all P < 0.05). The interclass correlation coefficient and kappa values for aneurysm morphology and shape indicated superior measurement consistency and shape concordance of silent-MRA-CT with CTA compared with TOF-MRA-CT (all P < 0.01). This study supports the use of silent/TOF-MRA-CT fusion imaging as a reliable alternative to CTA, noting that silent-MRA-CT closely mirrors CTA. Contrast-free MRA-CT fusion images have the potential to be used for preoperative planning in patients with intracranial aneurysms who have contraindications to contrast.

AFDFusion: An adaptive frequency decoupling fusion network for multi-modality image

The multi-modality image fusion goal is to create a single image that provides a comprehensive scene description and conforms to visual perception by integrating complementary information about the merits of the different modalities, e.g., salient intensities of infrared images and detail textures of visible images. Although some works explore decoupled representations of multi-modality images, they struggle with complex nonlinear relationships, fine modal decoupling, and noise handling. To cope with this issue, we propose an adaptive frequency decoupling module to perceive the associative invariant and inherent specific among cross-modality by dynamically adjusting the learnable low frequency weight of the kernel. Specifically, we utilize a contrastive learning loss for restricting the solution space of feature decoupling to learn representations of both the invariant and specific in the multi-modality images. The underlying idea is that: in decoupling, low frequency features, which are similar in the representation space, should be pulled closer to each other, signifying the associative invariant, while high frequencies are pushed farther away, also indicating the intrinsic specific. Additionally, a multi-stage training manner is introduced into our framework to achieve decoupling and fusion. Stage I, MixEncoder and MixDecoder with the same architecture but different parameters are trained to perform decoupling and reconstruction supervised by the contrastive self-supervised mechanism. Stage II, two feature fusion modules are added to integrate the invariant and specific features and output the fused image. Extensive experiments demonstrated the proposed method superiority over the state-of-the-art methods in both qualitative and quantitative evaluation on two multi-modal image fusion tasks.

Adaptive Transformer-Based Multi-Modal Image Fusion for Real-Time Medical Diagnosis and Object Detection

In recent years, medical diagnosis and object detection have been significantly enhanced by the integration of multi-modal image fusion techniques. This study proposes an Adaptive Transformer-Based Multi-Modal Image Fusion (AT-MMIF) framework designed for real-time medical diagnosis and object detection. The framework employs a Transformer architecture to capture both global and local feature correlations across multiple imaging modalities, including MRI, CT, PET, and X-ray, for more accurate diagnostic results and faster object detection in medical imagery. The fusion process incorporates spatial and frequency-domain information to improve the clarity and detail of the output images, enhancing diagnostic accuracy. The adaptive attention mechanism within the Transformer dynamically adjusts to the relevant features of different image types, optimizing fusion in real time. This leads to an improved sensitivity (98.5%) and specificity (96.7%) in medical diagnosis. Additionally, the model significantly reduces false positives and negatives, with an F1 score of 97.2% in object detection tasks. The AT-MMIF framework is further optimized for real-time processing with an average inference time of 120 ms per image and a model size reduction of 35% compared to existing multi-modal fusion models. By leveraging the strengths of Transformer architectures and adaptive learning, the proposed framework offers a highly efficient and scalable solution for real-time medical diagnosis and object detection in various clinical settings, including radiology, oncology, and pathology.

Parallel-way: Multi-modality-based brain tumor segmentation using parallel capsule network

ABSTRACT Brain tumors present a formidable diagnostic challenge due to their aberrant cell growth. Accurate determination of tumor location and size is paramount for effective diagnosis. Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET) are pivotal tools in clinical diagnosis, yet tumor segmentation within their images remains challenging, particularly at boundary pixels, owing to limited sensitivity. Recent endeavors have introduced fusion-based strategies to refine segmentation accuracy, yet these methods often prove inadequate. In response, we introduce the Parallel-Way framework to surmount these obstacles. Our approach integrates MRI and PET data for a holistic analysis. Initially, we enhance image quality by employing noise reduction, bias field correction, and adaptive thresholding, leveraging Improved Kalman Filter (IKF), Expectation Maximization (EM), and Improved Vibe Algorithm (IVib), respectively. Subsequently, we conduct multi-modality image fusion through the Dual-Tree Complex Wavelet Transform (DTWCT) to amalgamate data from both modalities. Following fusion, we extract pertinent features using the Advanced Capsule Network (ACN) and reduce feature dimensionality via Multi-objective Diverse Evolution-based selection. Tumor segmentation is then executed utilizing the Twin Vision Transformer with dual attention mechanism. Implemented our Parallel-Way framework which exhibits heightened model performance. Evaluation across multiple metrics, including accuracy, sensitivity, specificity, F1-Score, and AUC, underscores its superiority over existing methodologies.

Transarterial embolization for anterior cranial fossa dural arteriovenous fistula based on multi-modal three-dimensional imaging.

Dural arteriovenous fistula (DAVF) in the anterior cranial fossa (ACF) is known to show a high risk of intracranial hemorrhage. Recently, multi-modal fusion imaging with computed tomography angiography, computed tomography venography, and three-dimensional (3D) rotation angiography have been used preoperatively to ensure anatomical safety. We report on endovascular treatment as a first-line approach for ACFDAVF based on the understanding of vascular anatomy obtained from multi-modal fusion imaging. All patients with ACF-DAVF treated endovascularly as a first-line approach were included in this study. Analyses took into account complications (particularly visual function), immediate angiographic outcomes, and follow-up findings in consecutive patients with ACF-DAVF treated with interventional treatment based on multi-modal fusion imaging. Five patients with ACF-DAVF underwent six sessions of transarterial embolization (TAE) in our institution. The five male patients (mean age, 74.5 years; range, 60-84 years) were treated with liquid embolic agents (Onyx, four procedures; n-butyl 2-cyanoacrylate, two procedures). No difference was seen between preoperative image evaluation and image evaluation during the endovascular procedure, and in all cases, a microcatheter was navigated into a target artery assumed from preoperative multi-modal imaging, allowing treatment completion in a single procedure. In all cases, the shunt disappeared completely and visual function after procedure was maintained. At the last follow-up, all patients showed a modified Rankin scale score of 0 or 1 with no recurrences. Multi-modal fusion imaging facilitates a 3D understanding of the vascular anatomy, allowing TAE as the first-line treatment for ACF-DAVF.

An ensemble deep learning model for medical image fusion with Siamese neural networks and VGG-19.

Multimodal medical image fusion methods, which combine complementary information from many multi-modality medical images, are among the most important and practical approaches in numerous clinical applications. Various conventional image fusion techniques have been developed for multimodality image fusion. Complex procedures for weight map computing, fixed fusion strategy and lack of contextual understanding remain difficult in conventional and machine learning approaches, usually resulting in artefacts that degrade the image quality. This work proposes an efficient hybrid learning model for medical image fusion using pre-trained and non-pre-trained networks i.e. VGG-19 and SNN with stacking ensemble method. The model leveraging the unique capabilities of each architecture, can effectively preserve the detailed information with high visual quality, for numerous combinations of image modalities in image fusion challenges, notably improved contrast, increased resolution, and lower artefacts. Additionally, this ensemble model can be more robust in the fusion of various combinations of source images that are publicly available from Havard-Medical-Image-Fusion Datasets, GitHub. and Kaggle. Our proposed model performance is superior in terms of visual quality and performance metrics to that of the existing fusion methods in literature like PCA+DTCWT, NSCT, DWT, DTCWT+NSCT, GADCT, CNN and VGG-19.

Current Applications of the Three-Dimensional Printing Technology in Neurosurgery: A Review.

In the recent years, three-dimensional (3D) printing technology has emerged as a transformative tool, particularly in health care, offering unprecedented possibilities in neurosurgery. This review explores the diverse applications of 3D printing in neurosurgery, assessing its impact on precision, customization, surgical planning, and education. A literature review was conducted using PubMed, Web of Science, Embase, and Scopus, identifying 84 relevant articles. These were categorized into spine applications, neurovascular applications, neuro-oncology applications, neuroendoscopy applications, cranioplasty applications, and modulation/stimulation applications. 3D printing applications in spine surgery showcased advancements in guide devices, prosthetics, and neurosurgical planning, with patient-specific models enhancing precision and minimizing complications. Neurovascular applications demonstrated the utility of 3D-printed guide devices in intracranial hemorrhage and enhanced surgical planning for cerebrovascular diseases. Neuro-oncology applications highlighted the role of 3D printing in guide devices for tumor surgery and improved surgical planning through realistic models. Neuroendoscopy applications emphasized the benefits of 3D-printed guide devices, anatomical models, and educational tools. Cranioplasty applications showed promising outcomes in patient-specific implants, addressing biomechanical considerations. The integration of 3D printing into neurosurgery has significantly advanced precision, customization, and surgical planning. Challenges include standardization, material considerations, and ethical issues. Future directions involve integrating artificial intelligence, multimodal imaging fusion, biofabrication, and global collaboration. 3D printing has revolutionized neurosurgery, offering tailored solutions, enhanced surgical planning, and invaluable educational tools. Addressing challenges and exploring future innovations will further solidify the transformative impact of 3D printing in neurosurgical care. This review serves as a comprehensive guide for researchers, clinicians, and policymakers navigating the dynamic landscape of 3D printing in neurosurgery.

Application of 3D-Slicer Software in the Treatment of Gliomas.

To explore the application of 3D-Slicer multimodal image fusion technology in the surgical treatment of gliomas. A retrospective analysis of clinical data from 48 cases of intracranial gliomas treated with 3D-Slicer multimodal image fusion technology from September 2021 to February 2023. Preoperative cranial CT, MRI, TOF-MRA, DTI, and MRV sequence examinations were completed. DICOM format medical imaging data were processed using the 3D-Slicer software. Multimodal fusion and 3-dimensional reconstruction of the tumor and surrounding structures were conducted to clarify the relationship between the tumor and surrounding structures. The surgery path was simulated through 3D virtual visualization images, enabling precise preoperative planning and assisting in surgical treatment. Out of 48 patients, complete tumor resection was achieved in 45 cases. One case encountered severe postoperative cerebral edema, no infections were reported, 1 patient had slightly reduced muscle strength after surgery compared with before, and no cases of severe neurological function damage or surgical death were recorded. 3D-Slicer multimodal image fusion technology can assist in detailed preoperative planning for the surgical treatment of intracranial glioma, reducing surgical complications, improving the rate of total tumor resection, and reducing postoperative complications.

A novel 3D multimodal fusion imaging surgical guidance in microvascular decompression for primary trigeminal neuralgia and hemifacial spasm

BackgroundNeurovascular compression (NVC) is a primary etiology of trigeminal neuralgia (TN) and hemifacial spasm (HFS). Despite Magnetic Resonance Tomographic Angiography (MRTA) being a useful tool for 3D multimodal fusion imaging (MFI) in microvascular decompression (MVD) surgery planning, it may not visualize smaller arterial vessels and veins effectively. We validate a novel computed tomography angiography and venography (CTA/V) - diffusion tensor tractography (DTT) -3D-MFI to enhance the MVD surgical guidance.MethodsIn this prospective study, 80 patients with unilateral primary TN or HFS who underwent MVD surgery were included. Imaging was conducted using CTA/V-DTT-3D-MFI compared with CT-MRTA-3D-MFI in predicting the responsible vessel and assessing the severity of NVC. Surgical outcomes were subsequently analyzed. Neurosurgery residents were provided with questionnaires to evaluate and compare the two approaches.ResultsCTA/V-DTT-3D-MFI significantly improved accuracy in identifying the responsible vessel (kappa = 0.954) and NVC (kappa = 0.969) compared to CT-MRTA-3D-MFI, aligning well with surgical findings. CTA/V-DTT-3D-MFI also exhibited higher sensitivity in identifying responsible vessels (98.0%) and NVC (98.7%) than CT-MRTA-3D-MFI. Additionally, CTA/V-DTT-3D-MFI showed fewer complications, shorter operation times, and lower recurrence after one year (all p < 0.05). Resident neurosurgeons emphasized that CTA/V-DTT-3D-MFI greatly assisted in formulating precise surgical strategies for more accurate identification and protection of responsible vessels and nerves (all p < 0.001).ConclusionCTA/V-DTT-3D-MFI enhances MVD surgery guidance, improving accuracy in identifying responsible vessels and NVC for better outcomes. This advanced imaging plays a crucial role in safer and more effective MVD surgery, as well as in training neurosurgeons.

Adaptive infrared patterns for microscopic surface reconstructions.

Multi-zoom microscopic surface reconstructions of operating sites, especially in ENT surgeries, would allow multimodal image fusion for determining the amount of resected tissue, for recognizing critical structures, and novel tools for intraoperative quality assurance. State-of-the-art three-dimensional model creation of the surgical scene is challenged by the surgical environment, illumination, and the homogeneous structures of skin, muscle, bones, etc., that lack invariant features for stereo reconstruction. An adaptive near-infrared pattern projector illuminates the surgical scene with optimized patterns to yield accurate dense multi-zoom stereoscopic surface reconstructions. The approach does not impact the clinical workflow. The new method is compared to state-of-the-art approaches and is validated by determining its reconstruction errors relative to a high-resolution 3D-reconstruction of CT data. 200 surface reconstructions were generated for 5 zoom levels with 10 reconstructions for each object illumination method (standard operating room light, microscope light, random pattern and adaptive NIR pattern). For the adaptive pattern, the surface reconstruction errors ranged from 0.5 to 0.7 mm, as compared to 1-1.9 mm for the other approaches. The local reconstruction differences are visualized in heat maps. Adaptive near-infrared (NIR) pattern projection in microscopic surgery allows dense and accurate microscopic surface reconstructions for variable zoom levels of small and homogeneous surfaces. This could potentially aid in microscopic interventions at the lateral skull base and potentially open up new possibilities for combining quantitative intraoperative surface reconstructions with preoperative radiologic imagery.

A review of deep learning approaches for multimodal image segmentation of liver cancer.

This review examines the recent developments in deep learning (DL) techniques applied to multimodal fusion image segmentation for liver cancer. Hepatocellular carcinoma is a highly dangerous malignant tumor that requires accurate image segmentation for effective treatment and disease monitoring. Multimodal image fusion has the potential to offer more comprehensive information and more precise segmentation, and DL techniques have achieved remarkable progress in this domain. This paper starts with an introduction to liver cancer, then explains the preprocessing and fusion methods for multimodal images, then explores the application of DL methods in this area. Various DL architectures such as convolutional neural networks (CNN) and U-Net are discussed and their benefits in multimodal image fusion segmentation. Furthermore, various evaluation metrics and datasets currently used to measure the performance of segmentation models are reviewed. While reviewing the progress, the challenges of current research, such as data imbalance, model generalization, and model interpretability, are emphasized and future research directions are suggested. The application of DL in multimodal image segmentation for liver cancer is transforming the field of medical imaging and is expected to further enhance the accuracy and efficiency of clinical decision making. This review provides useful insights and guidance for medical practitioners.

Radiation image reconstruction and uncertainty quantification using a Gaussian process prior

We propose a complete framework for Bayesian image reconstruction and uncertainty quantification based on a Gaussian process prior (GPP) to overcome limitations of maximum likelihood expectation maximization (ML-EM) image reconstruction algorithm. The prior distribution is constructed with a zero-mean Gaussian process (GP) with a choice of a covariance function, and a link function is used to map the Gaussian process to an image. Unlike many other maximum a posteriori approaches, our method offers highly interpretable hyperparamters that are selected automatically with the empirical Bayes method. Furthermore, the GP covariance function can be modified to incorporate a priori structural priors, enabling multi-modality imaging or contextual data fusion. Lastly, we illustrate that our approach lends itself to Bayesian uncertainty quantification techniques, such as the preconditioned Crank–Nicolson method and the Laplace approximation. The proposed framework is general and can be employed in most radiation image reconstruction problems, and we demonstrate it with simulated free-moving single detector radiation source imaging scenarios. We compare the reconstruction results from GPP and ML-EM, and show that the proposed method can significantly improve the image quality over ML-EM, all the while providing greater understanding of the source distribution via the uncertainty quantification capability. Furthermore, significant improvement of the image quality by incorporating a structural prior is illustrated.

MHW-GAN: Multidiscriminator Hierarchical Wavelet Generative Adversarial Network for Multimodal Image Fusion.

Image fusion technology aims to obtain a comprehensive image containing a specific target or detailed information by fusing data of different modalities. However, many deep learning-based algorithms consider edge texture information through loss functions instead of specifically constructing network modules. The influence of the middle layer features is ignored, which leads to the loss of detailed information between layers. In this article, we propose a multidiscriminator hierarchical wavelet generative adversarial network (MHW-GAN) for multimodal image fusion. First, we construct a hierarchical wavelet fusion (HWF) module as the generator of MHW-GAN to fuse feature information at different levels and scales, which avoids information loss in the middle layers of different modalities. Second, we design an edge perception module (EPM) to integrate edge information from different modalities to avoid the loss of edge information. Third, we leverage the adversarial learning relationship between the generator and three discriminators for constraining the generation of fusion images. The generator aims to generate a fusion image to fool the three discriminators, while the three discriminators aim to distinguish the fusion image and edge fusion image from two source images and the joint edge image, respectively. The final fusion image contains both intensity information and structure information via adversarial learning. Experiments on public and self-collected four types of multimodal image datasets show that the proposed algorithm is superior to the previous algorithms in terms of both subjective and objective evaluation.

MixFuse: An iterative mix-attention transformer for multi-modal image fusion

Multi-modal image fusion plays a crucial role in various visual systems. However, existing methods typically involve a multi-stage pipeline, i.e., feature extraction, integration, and reconstruction, which limits the effectiveness and efficiency of feature interaction and aggregation. In this paper, we propose MixFuse, a compact multi-modal image fusion framework based on Transformers. It smoothly unifies the process of feature extraction and integration, As its core, the Mix Attention Transformer Block (MATB) integrates the Cross-Attention Transformer Module (CATM) and the Self-Attention Transformer Module (SATM). The CATM introduces a symmetrical cross-attention mechanism to identify modality-specific and general features, filtering out irrelevant and redundant information. Meanwhile, the SATM is designed to refine the combined features via a self-attention mechanism, enhancing the internal consistency and proper preservation of the features. This successive cross and self-attention modules work together to enhance the generation of more accurate and refined feature maps, which are essential for later reconstruction. Extensive evaluation of MixFuse on five public datasets shows its superior performance and adaptability over state-of-the-art methods. The code and model will be released at https://github.com/Bitlijinfu/MixFuse.

Multimodal registration network with multi-scale feature-crossing.

A critical piece of information for prostate intervention and cancer treatment is provided by the complementary medical imaging modalities of ultrasound (US) and magnetic resonance imaging (MRI). Therefore, MRI-US image fusion is often required during prostate examination to provide contrast-enhanced TRUS, in which image registration is a key step in multimodal image fusion. We propose a novel multi-scale feature-crossing network for the prostate MRI-US image registration task. We designed a feature-crossing module to enhance information flow in the hidden layer by integrating intermediate features between adjacent scales. Additionally, an attention block utilizing three-dimensional convolution interacts information between channels, improving the correlation between different modal features. We used 100 cases randomly selected from The Cancer Imaging Archive (TCIA) for our experiments. A fivefold cross-validation method was applied, dividing the dataset into five subsets. Four subsets were used for training, and one for testing, repeating this process five times to ensure each subset served as the test set once. We test and evaluate our technique using fivefold cross-validation. The cross-validation trials result in a median target registration error of 2.20 mm on landmark centroids and a median Dice of 0.87 on prostate glands, both of which were better than the baseline model. In addition, the standard deviation of the dice similarity coefficient is 0.06, which suggests that the model is stable. We propose a novel multi-scale feature-crossing network for the prostate MRI-US image registration task. A random selection of 100 cases from The Cancer Imaging Archive (TCIA) was used to test and evaluate our approach using fivefold cross-validation. The experimental results showed that our method improves the registration accuracy. After registration, MRI and TURS images were more similar in structure and morphology, and the location and morphology of cancer were more accurately reflected in the images.

The value of multimodal imaging fusion in preoperative visualization assessment of neurovascular relationship in hemifacial spasm: a single-center retrospective study.

The neurovascular conflict (NVC) at the brainstem exit zone of the facial nerve is considered the primary etiology of primary hemifacial spasm (HFS). Therefore, microvascular decompression (MVD) has become the preferred treatment for HFS. Successful neurovascular decompression can achieve significant therapeutic effects, and accurately identifying the site of compression is crucial for the success of this surgery. Detailed diagnostic neuroimaging plays an important role in accurately identifying the site of compression.The purpose of this study is to explore the feasibility and predictive value of preoperative visualization assessment of the neurovascular relationship in HFS using 3D Slicer software based on multimodal imaging fusion. This aims to reduce the omission of responsible vessels and lower the incidence of postoperative complications, thereby potentially improving the efficacy and safety of the surgery. This study retrospectively analyzed 80 patients with HFS who underwent MVD surgery. All patients underwent preoperative cranial MRI scans, including the 3D-FIESTA and the 3D-TOF MRA sequences. Three-dimensional models were reconstructed from the multimodal MRI images using 3D Slicer software. Independent observers, who were blinded to the surgical outcomes, evaluated the neurovascular relationships using both the three-dimensional models and multimodal MRI images. The assessment results were compared with intraoperative findings, and statistical analysis was conducted using SPSS 22.0 software. The agreement between preoperative assessment using the 3D-TOF MRA sequence combined with the 3D-FIESTA sequence and intraoperative findings was represented by a Kappa value of 0.343, while the Kappa value for agreement between three-dimensional reconstruction and intraoperative findings was 0.637. There was a statistically significant difference between the two methods ( X2 = 18.852, P = 0.001 ). The sensitivity and specificity of the 3D-TOF MRA sequence combined with the 3D-FIESTA sequence for evaluating neurovascular relationships were 92.4% and 100%, respectively, while for three-dimensional reconstruction, both were 100%. The Kappa value for agreement between preoperative the 3D-TOF MRA sequence combined with the 3D-FIESTA sequence prediction of offending vessels and intraoperative findings was 0.625, while the Kappa value for agreement between three-dimensional reconstruction and intraoperative findings was 0.938, showing a statistically significant difference ( X2 = 317.798, P = 0.000 ). The Kappa value for agreement between preoperative the 3D-TOF MRA sequence combined with the 3D-FIESTA sequence assessment of the anatomical location of facial nerve involvement in neurovascular compression and intraoperative findings was 0.608, while the Kappa value for agreement between three-dimensional reconstruction and intraoperative findings was 0.918, also showing a statistically significant difference ( X2 = 504.647, P = 0.000 ). The preoperative visualization assessment of neurovascular relationships in HFS using 3D Slicer software based on multimodal imaging fusion has been demonstrated to be reliable. It is more accurate than combining the 3D-TOF MRA sequence with the 3D-FIESTA sequence and shows higher consistency with intraoperative findings. This method provides guidance for surgical procedures and thereby potentially enhances the efficacy and safety of surgeries to a certain extent.

Comparative study of CTA/CTV and MRTA for preoperative simulation of microvascular decompression in neurovascular compression syndromes.

Neurovascular compression syndrome (NVCS), characterized by cranial nerve compression due to adjacent blood vessels at the root entry zone, frequently presents as trigeminal neuralgia (TN), hemifacial spasm (HFS), or glossopharyngeal neuralgia (GN). Despite its prevalence in NVCS assessment, Magnetic Resonance Tomographic Angiography (MRTA)'s limited sensitivity to small vessels and veins poses challenges. This study aims to refine vessel localization and surgical planning for NVCS patients using a novel 3D multimodal fusion imaging (MFI) technique incorporating computed tomography angiography and venography (CTA/CTV). A retrospective analysis was conducted on 76 patients who underwent MVD surgery and were diagnosed with single-site primary TN, HFS, or GN. Imaging was obtained from MRTA and CTA/CTV sequences, followed by image processing and 3D-MFI using FastSurfer and 3DSlicer. The CTA/CTV-3D-MFI showed higher sensitivity than MRTA-3D-MFI in predicting responsible vessels (98.6% vs. 94.6%) and NVC severity (98.6% vs. 90.8%). Kappa coefficients revealed strong agreement with MRTA-3D-MFI (0.855 for vessels, 0.835 for NVC severity) and excellent agreement with CTA/CTV-3D-MFI (0.951 for vessels, 0.952 for NVC). Resident neurosurgeons significantly preferred CTA/CTV-3D-MFI due to its better correlation with surgical reality, clearer depiction of surgical anatomy, and optimized visualization of approaches (p < 0.001). Implementing CTA/CTV-3D-MFI significantly enhanced diagnostic accuracy and surgical planning for NVCS, outperforming MRTA-3D-MFI in identifying responsible vessels and assessing NVC severity. This innovative imaging modality can potentially improve outcomes by guiding safer and more targeted surgeries, particularly in cases where MRTA may not adequately visualize crucial neurovascular structures.

Multimodal medical image fusion based on interval gradients and convolutional neural networks

Many image fusion methods have been proposed to leverage the advantages of functional and anatomical images while compensating for their shortcomings. These methods integrate functional and anatomical images while presenting physiological and metabolic organ information, making their diagnostic efficiency far greater than that of single-modal images. Currently, most existing multimodal medical imaging fusion methods are based on multiscale transformation, which involves obtaining pyramid features through multiscale transformation. Low-resolution images are used to analyse approximate image features, and high-resolution images are used to analyse detailed image features. Different fusion rules are applied to achieve feature fusion at different scales. Although these fusion methods based on multiscale transformation can effectively achieve multimodal medical image fusion, much detailed information is lost during multiscale and inverse transformation, resulting in blurred edges and a loss of detail in the fusion images. A multimodal medical image fusion method based on interval gradients and convolutional neural networks is proposed to overcome this problem. First, this method uses interval gradients for image decomposition to obtain structure and texture images. Second, deep neural networks are used to extract perception images. Three methods are used to fuse structure, texture, and perception images. Last, the images are combined to obtain the final fusion image after colour transformation. Compared with the reference algorithms, the proposed method performs better in multiple objective indicators of QEN\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$Q_{EN}$$\\end{document}, QNIQE\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$Q_{NIQE}$$\\end{document}, QSD\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$Q_{SD}$$\\end{document}, QSSEQ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$Q_{SSEQ}$$\\end{document} and QTMQI\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$Q_{TMQI}$$\\end{document}.

Multi-Source Image Fusion Based on BEMD and Region Sharpness Guidance Region Overlapping Algorithm

Multi-focal image and multi-modal image fusion technology can fully take advantage of different sensors or different times, retaining the image feature information and improving the image quality. A multi-source image fusion algorithm based on bidimensional empirical mode decomposition (BEMD) and a region sharpness-guided region overlapping algorithm are studied in this article. Firstly, source images are decomposed into multi-layer bidimensional intrinsic mode functions (BIMFs) and residuals from high-frequency layer to low-frequency layer by BEMD. Gaussian bidimensional intrinsic mode functions (GBIMFs) are obtained by applying Gaussian filtering operated on BIMF and calculating the sharpness value of segmented regions using an improved weighted operator based on the Tenengrad function, which is the key to comparison selection and fusion. Then, the GBIMFs and residuals selected by sharpness comparison strategy are fused by the region overlapping method, and the stacked layers are weighted to construct the final fusion image. Finally, based on qualitative evaluation and quantitative evaluation indicators, the proposed algorithm is compared with six typical image fusion algorithms. The comparison results show that the proposed algorithm can effectively capture the feature information of images in different states and reduce the redundant information.