Medical Image Fusion Techniques Research Articles

Enhanced multimodal medical image fusion via modified DWT with arithmetic optimization algorithm

Medical image fusion (MIF) techniques are proficient in combining medical images in distinct morphologies to obtain a reliable medical analysis. A single modality image could not offer adequate data for an accurate analysis. Therefore, a novel multimodal MIF-based artificial intelligence (AI) method has been presented. MIF approaches fuse multimodal medical images for exact and reliable medical recognition. Multimodal MIF improves diagnostic accuracy and clinical decision-making by combining complementary data in different imaging modalities. This article presents a new multimodal medical image fusion model utilizing Modified DWT with an Arithmetic Optimization Algorithm (MMIF-MDWTAOA) approach. The MMIF-MDWTAOA approach aims to generate a fused image with the significant details and features from each modality, leading to an elaborated depiction for precise interpretation by medical experts. The bilateral filtering (BF) approach is primarily employed for noise elimination. Next, the image decomposition process uses a modified discrete wavelet transform (MDWT) approach. However, the approximation coefficient of modality_1 and the detailed coefficient of modality_2 can be fused interchangeably. Furthermore, a fusion rule is derived from combining the multimodality data, and the AOA model is enforced to ensure the optimum selection of the fusion rule parameters. A sequence of simulations is accomplished to validate the enhanced output of the MMIF-MDWTAOA technique. The investigational validation of the MMIF-MDWTAOA technique showed the highest entropy values of 7.568 and 7.741 bits/pixel over other approaches.

A multibranch and multiscale neural network based on semantic perception for multimodal medical image fusion

Medical imaging is indispensable for accurate diagnosis and effective treatment, with modalities like MRI and CT providing diverse yet complementary information. Traditional image fusion methods, while essential in consolidating information from multiple modalities, often suffer from poor image quality and loss of crucial details due to inadequate handling of semantic information and limited feature extraction capabilities. This paper introduces a novel medical image fusion technique leveraging unsupervised image segmentation to enhance the semantic understanding of the fusion process. The proposed method, named DUSMIF, employs a multi-branch, multi-scale deep learning architecture that integrates advanced attention mechanisms to refine the feature extraction and fusion processes. An innovative approach that utilizes unsupervised image segmentation to extract semantic information is introduced, which is then integrated into the fusion process. This not only enhances the semantic relevance of the fused images but also improves the overall fusion quality. The paper proposes a sophisticated network structure that extracts and fuses features at multiple scales and across multiple branches. This structure is designed to capture a comprehensive range of image details and contextual information, significantly improving the fusion outcomes. Multiple attention mechanisms are incorporated to selectively emphasize important features and integrate them effectively across different modalities and scales. This approach ensures that the fused images maintain high quality and detail fidelity. A joint loss function combining content loss, structural similarity loss, and semantic loss is formulated. This function not only guides the network in preserving image brightness and texture but also ensures that the fused image closely resembles the source images in both content and structure. The proposed method demonstrates superior performance over existing fusion techniques in objective assessments and subjective evaluations, confirming its effectiveness in enhancing the diagnostic utility of fused medical images.

CFIFusion: Dual‐Branch Complementary Feature Injection Network for Medical Image Fusion

ABSTRACTThe goal of fusing medical images is to integrate the diverse information that multimodal medical images hold. However, the challenges lie in the limitations of imaging sensors and the issue of incomplete modal information retention, which make it difficult to produce images encompassing both functional and anatomical information. To overcome these obstacles, several medical image fusion techniques based on CNN or transformer architectures have been presented. Nevertheless, CNN technique struggles to establish extensive dependencies between the fused and source images, and transformer architecture often overlooks shallow complementary features. To augment both the feature extraction capacity and the stability of the model, we introduce a framework, called dual‐branch complementary feature injection fusion (CFIFusion) technique, a for multimodal medical image fusion framework that combines unsupervised models of CNN model and transformer techniques. Specifically, in our framework, the entire source image and segmented source image are input into an adaptive backbone network to learn global and local features, respectively. To further retain the source images' complementary information, we design a multi‐scale complementary feature extraction framework as an auxiliary module, focusing on calculating feature differences at each level to capture the shallow complementary information. Then, we design a shallow information preservation module tailored for sliced image characteristics. Experimental results on the Harvard whole brain atlas dataset demonstrate that CFIFusion shows greater benefits than recent state‐of‐the‐art algorithms in terms of both subjective and objective evaluations.

Advancing multimodal medical image fusion: an adaptive image decomposition approach based on multilevel Guided filtering.

With the rapid development of medical imaging methods, multimodal medical image fusion techniques have caught the interest of researchers. The aim is to preserve information from diverse sensors using various models to generate a single informative image. The main challenge is to derive a trade-off between the spatial and spectral qualities of the resulting fused image and the computing efficiency. This article proposes a fast and reliable method for medical image fusion depending on multilevel Guided edge-preserving filtering (MLGEPF) decomposition rule. First, each multimodal medical image was divided into three sublayer categories using an MLGEPF decomposition scheme: small-scale component, large-scale component and background component. Secondly, two fusion strategies-pulse-coupled neural network based on the structure tensor and maximum based-are applied to combine the three types of layers, based on the layers' various properties. The three different types of fused sublayers are combined to create the fused image at the end. A total of 40 pairs of brain images from four separate categories of medical conditions were tested in experiments. The pair of images includes various case studies including magnetic resonance imaging (MRI) , TITc, single-photon emission computed tomography (SPECT) and positron emission tomography (PET). We included qualitative analysis to demonstrate that the visual contrast between the structure and the surrounding tissue is increased in our proposed method. To further enhance the visual comparison, we asked a group of observers to compare our method's outputs with other methods and score them. Overall, our proposed fusion scheme increased the visual contrast and received positive subjective review. Moreover, objective assessment indicators for each category of medical conditions are also included. Our method achieves a high evaluation outcome on feature mutual information (FMI), the sum of correlation of differences (SCD), Qabf and Qy indexes. This implies that our fusion algorithm has better performance in information preservation and efficient structural and visual transferring.

New Approach for Multimodal Medical Image Fusion using PDE-based Technique

Accurate diagnosis of diseases is essential for effective medical treatment, and medical image fusion plays a crucial role in achieving this goal. This article proposes a novel technique for multimodal medical image fusion based on fourth order PDE-based techniques. The proposed technique uses inputted CT and brain MRI images, and applies a PDE-based technique for noise reduction and edge preservation. Subsequently, a computed fusion rule using principal component analysis is applied to generate a fused image of the brain that presents clear and essential components for medical diagnosis and analysis. The article provides a comparative study of the proposed technique with other existing techniques, demonstrating its optimal performance. The proposed technique can significantly enhance medical image diagnosis and analysis for accurate disease detection and treatment.

Medical Diagnosis with Multimodal Image Fusion Techniques

Image Fusion is an effective approach utilized to draw out all the significant information from the source images, which supports experts in evaluation and quick decision making. Multi modal medical image fusion produces a composite fused image utilizing various sources to improve quality and extract complementary information. It is extremely challenging to gather every piece of information needed using just one imaging method. Therefore, images obtained from different modalities are fused Additional clinical information can be gleaned through the fusion of several types of medical image pairings. This study's main aim is to present a thorough review of medical image fusion techniques which also covers steps in fusion process, levels of fusion, various imaging modalities with their pros and cons, and the major scientific difficulties encountered in the area of medical image fusion. This paper also summarizes the quality assessments fusion metrics. The various approaches used by image fusion algorithms that are presently available in the literature are classified into four broad categories i) Spatial fusion methods ii) Multiscale Decomposition based methods iii) Neural Network based methods and iv) Fuzzy Logic based methods. the benefits and pitfalls of the existing literature are explored and Future insights are suggested. Moreover, this study is anticipated to create a solid platform for the development of better fusion techniques in medical applications.

Comprehensive performance analysis of different medical image fusion techniques for accurate healthcare diagnosis applications

Comprehensive performance analysis of different medical image fusion techniques for accurate healthcare diagnosis applications

A Lightweight Multimode Medical Image Fusion Method Using Similarity Measure Between Intuitionistic Fuzzy Sets Joint Laplacian Pyramid

Medical image fusion combines the multiple features of human tissue from different source images, which will be conducive to clinical diagnosis. Because of the human visual perception and mechanisms of medical imaging, abundant fuzzy features exist in medical image fusion with remarkable influence. To address this issue, we introduce the similarity measure of fuzzy set theory into medical image fusion to abstract and measure these fuzzy features. In this study, we present a new similarity measure of intuitionistic fuzzy set theory to represent image features and present a lightweight medical image fusion technique on the basis of this new measure. First, the new similarity measure is proposed according to geometric modeling technique and verified by mathematical justification. Second, a set of high frequency sub-images and a low frequency sub-image are produced by Laplacian pyramid decomposition to separate different image features. Third, our proposed similarity measure is utilized to describe and extract the detailed image features. Finally, two fusion rules and inverse Laplacian pyramid decomposition are sequentially used to get the fused image. This research shows that the similarity measure of fuzzy set theory can get excellent performance in medical image fusion. Experiments reveal that our similarity measure and medical image fusion method both have the superior performance to most of the existing methods.

Medical image fusion based on multi-scale decomposition using hybrid deep learning network model

ABSTRACT In medical applications, effective medical diagnosis is developed by combining medical images from various morphologies using medical image fusion techniques. An accurate diagnosis cannot be made from a single modality image, and the existing approaches need better efficiency due to poor image quality and inconsistent performance. To solve this problem, this research proposes an effective artificial intelligence model based on multi-modal medical image fusion. This research develops a new method based on medical images using deep residual neural networks (ResNet-50) and a DarkNet-19. The optimised discrete wavelet transform (ODWT) is used for image decomposing. The original image’s high- and low-frequency coefficients are decomposed by using ODWT. The low-frequency coefficient is then fused using the Modified ResNet-50 model. After that, DarkNet-19 fuses the high-frequency coefficients, and the input stimulus for DarkNet-19 is a high-frequency coefficient’s modified average gradient. The inverse ODWT is employed to reconstruct the fused image finally. The proposed fusion model’s effectiveness is assessed using the various CT-MRI, CT-PET, and MRI-SPECT imaging datasets. The proposed medical image fusion approach attains a maximum performance for PSNR of 40.03 dB, Fusion Factor of 6.1025, Fusion Symmetry of 0.0869, and Visual Information Fidelity of 0.121.

Multimodality Medical Image Fusion Using Clustered Dictionary Learning in Non-Subsampled Shearlet Transform.

Imaging data fusion is becoming a bottleneck in clinical applications and translational research in medical imaging. This study aims to incorporate a novel multimodality medical image fusion technique into the shearlet domain. The proposed method uses the non-subsampled shearlet transform (NSST) to extract both low- and high-frequency image components. A novel approach is proposed for fusing low-frequency components using a modified sum-modified Laplacian (MSML)-based clustered dictionary learning technique. In the NSST domain, directed contrast can be used to fuse high-frequency coefficients. Using the inverse NSST method, a multimodal medical image is obtained. Compared to state-of-the-art fusion techniques, the proposed method provides superior edge preservation. According to performance metrics, the proposed method is shown to be approximately 10% better than existing methods in terms of standard deviation, mutual information, etc. Additionally, the proposed method produces excellent visual results regarding edge preservation, texture preservation, and more information.

Water wave optimized nonsubsampled shearlet transformation technique for multimodal medical image fusion

SummaryMedical image fusion has advanced to the point that it is now possible to combine multiple medical images for accurate disease diagnosis and treatment. The state‐of‐art techniques based on spatial and transform domains suffer from different limitations such as low fused image quality, spectral degradation, contrast reduction, low edge information preserving, lack of shift‐invariance, high computational complexity, classification accuracy, and sensitivity to noise. The main motivation of this work is to generate a single image with excellent visual clarity that retains the features of the source images. This article proposes a water wave optimized nonsubsampled shearlet transformation technique (NSST) for multimodal medical image fusion, in which the water wave optimization (WWO) algorithm is used to allocate the weights of the NSST approach's high‐frequency subbands. The NSST approach is primarily used in this work due to its ability to withstand shift‐invariance and its potential to improve the visual clarity of the fused multimodal image by preserving the essential features present in the image's various directions and edges. We combined the NSST technique with the WWO algorithm, which processes the edges, details, and contourlets of medical images using a max selection strategy based on the fitness function, to improve image quality and computational costs. The WWO algorithm is mainly applied to the NSST to minimize the L1 distance between the fused and the source images. Hence to overcome this problem a condition CNN optimized with a hybrid tunicate swarm memetic (TSM) algorithm is used to incorporate both the benefits offered by the condition CNN–TSM algorithm and NSST. The TSM optimized condition CNN architecture is used to preserve the coefficients of the image and improve the perceiving capability of the high‐frequency sub‐bands. An inverse NSST is used for fused frequency sub‐band integration. Finally, the efficiency of the proposed methodology is evaluated in terms of enhanced visual feature quality, edge detection, contour detection, and computational performance.

Non Sub-Sampled Contourlet with Joint Sparse Representation Based Medical Image Fusion

Medical Image Fusion is the synthesizing technology for fusing multimodal medical information using mathematical procedures to generate better visual on the image content and high-quality image output. Medical image fusion represents an indispensible role in fixing major solutions for the complicated medical predicaments, while the recent research results have an enhanced affinity towards the preservation of medical image details, leaving color distortion and halo artifacts to remain unaddressed. This paper proposes a novel method of fusing Computer Tomography (CT) and Magnetic Resonance Imaging (MRI) using a hybrid model of Non Sub-sampled Contourlet Transform (NSCT) and Joint Sparse Representation (JSR). This model gratifies the need for precise integration of medical images of different modalities, which is an essential requirement in the diagnosing process towards clinical activities and treating the patients accordingly. In the proposed model, the medical image is decomposed using NSCT which is an efficient shift variant decomposition transformation method. JSR is exercised to extricate the common features of the medical image for the fusion process. The performance analysis of the proposed system proves that the proposed image fusion technique for medical image fusion is more efficient, provides better results, and a high level of distinctness by integrating the advantages of complementary images. The comparative analysis proves that the proposed technique exhibits better-quality than the existing medical image fusion practices.

A Brief Analysis of Multimodal Medical Image Fusion Techniques

Recently, image fusion has become one of the most promising fields in image processing since it plays an essential role in different applications, such as medical diagnosis and clarification of medical images. Multimodal Medical Image Fusion (MMIF) enhances the quality of medical images by combining two or more medical images from different modalities to obtain an improved fused image that is clearer than the original ones. Choosing the best MMIF technique which produces the best quality is one of the important problems in the assessment of image fusion techniques. In this paper, a complete survey on MMIF techniques is presented, along with medical imaging modalities, medical image fusion steps and levels, and the assessment methodology of MMIF. There are several image modalities, such as Computed Tomography (CT), Positron Emission Tomography (PET), Magnetic Resonance Imaging (MRI), and Single Photon Emission Computed Tomography (SPECT). Medical image fusion techniques are categorized into six main categories: spatial domain, transform fusion, fuzzy logic, morphological methods, and sparse representation methods. The MMIF levels are pixel-level, feature-level, and decision-level. The fusion quality evaluation metrics can be categorized as subjective/qualitative and objective/quantitative assessment methods. Furthermore, a detailed comparison between obtained results for significant MMIF techniques is also presented to highlight the pros and cons of each fusion technique.

DFENet: A dual-branch feature enhanced network integrating transformers and convolutional feature learning for multimodal medical image fusion

In recent times, several medical image fusion techniques based on the convolutional neural network (CNN) have been proposed for various medical imaging fusion tasks. However, these methods cannot model the long-range dependencies between the fused image and the source images. To address this limitation, we propose DFENet, a multimodal medical image fusion framework that integrates CNN feature learning and vision transformer feature learning using self-supervised learning. DFENet is based on an encoder-decoder network, which can be trained on large-scale natural image dataset without the need for carefully collated ground truth fusion images. The proposed network consists of an encoder, a feature fuser, and a decoder. The encoder is composed of a CNN module and a transformer module, which is used to extract local and global features of images. In order to avoid the use of simple up-sampling and concatenate processing, a new global semantic information aggregation module is proposed to efficiently aggregate the multi-scale features obtained by the transformer module, which enhances the quality of the reconstructed images. The decoder is composed of six convolution layers with two skip connections, which are used for the reconstruction from fused features. We also propose a fusion strategy combining local energy and gradient information for the feature fusion process of magnetic resonance imaging and functional medical images. Compared to conventional fusion rules, our fusion strategy is more robust to noisy images. And compared with the existing competitive methods, our method retains more texture details of the original images and outputs a more natural and realistic fused image.

Convolutional Neural Networks (CNN) with Quantum-Behaved Particle Swarm Optimization (QPSO)-Based Medical Image Fusion

Medical imaging fusion is the process of combining pictures from various imaging modalities to create a single image that may be used in clinical settings. Robust methods for merging image data from several modalities are being developed in the field of multimodal medical imaging. Deep learning (DL) has been widely researched in two areas: pattern recognition and image processing. We will demonstrate a multimodal image fusion with DL implementation that considers the characteristics of medical diagnostic imaging as well as the demands of clinical practice. For the past three years, pixel-level picture fusion has been a hot topic. This paper proposes a new multimodal medical picture fusion technique for a wide range of medical diagnostic challenges. Image fusion is crucial in biomedical research and clinical diagnostics for biomedical image processing and therapy planning. The most convincing argument for fusion is obtaining a significant amount of critical information from the input photographs. We show how a well-organized multimodal medical image fusion technique can be utilized to integrate computed tomography (CT) and magnetic resonance imaging (MRI) data in this study. Using convolutional neural networks (CNNs), the quantum-behaved particle swarm optimization (QPSO) algorithm was used to create a method for integrating multimodal medical pictures. In order to improve the overall quality and efficiency of QPSO, it was chosen to add the metrics of image entropy, standard deviation, average gradient (AG), spatial frequency (SF), and visual information fidelity (VIF). In experiments, multimodal medical images are utilized to evaluate a variety of parameters, including performance and algorithm stability. When compared to the other possibilities, the recommended technique outperformed them in the evaluations. On a range of quantitative metrics, this method outperforms the alternatives.

Multimodal Medical Image Fusion with Improved Multi-Objective Meta-Heuristic Algorithm with Fuzzy Entropy

Medical image fusion enhances the significant and the valuable information such as exact abnormality localisation of the multimodal medical images. In the field of clinical environment, medical imaging acts as an important role in helping the doctors/radiologists. The information available in the images is important during the diagnosis. This can be enhanced using the multimodal medical image fusion technique through the integration of the information from several imaging modalities. Nowadays, several methodologies are proposed for fusing the medical images. Yet, the multimodal medical image fusion remains as a challenging task owing to the deprivation of medical images at the phase of acquisition. To handle this problem, this paper plans to develop the enhanced multi-objective medical image fusion model. Before initiating the fusion process, both the images to be fused are split into high-frequency sub-bands and low-frequency sub-bands by the improved Fast Discrete Curvelet Transform (FDCuT). Here, the fusion of low-frequency sub-images is accomplished by the averaging method, and high-frequency sub-images are fused by the optimised Type-2 fuzzy entropy. Both the FDCuT and Type-2 fuzzy entropy are enhanced by the multi-objective meta-heuristic algorithm by Adaptive Electric fish optimisation (A-EFO). The multi-objective function focuses on the “Peak Signal to Noise Ratio (PSNR), Structural SIMilarity (SSIM), Feature SIMilarity (FSIM)”. The comparison of the developed methodology over the traditional approaches observes enhanced performance with respect to visual quality measures.

Multi-modality image fusion for medical assistive technology management based on hybrid domain filtering

Medical image fusion technique combines images from different imaging modalities to enhance the reliability and accuracy of the medical diagnoses that play an increasingly significant role in clinical applications. In the domain of medical image fusion, it has made huge advancement in the span of past few years. However medical image fusion stills remain a challenging task since the recent methods still suffers from colour distortion, blurring and noise in their fusion results. In order to address the shortcomings of recent methods, this paper presents a novel method to fuse multimodal medical images by employing hybridization of cross bilateral filter and edge aware filtering using pixel-based fusion rule. This method proposed a novel algorithm for computation of weights for final fusion rule. First, a cross bilateral filter (CBF) is employed to both source images which considers both geometric closeness and gray level of neighbourhood pixels by avoiding smoothing the edges and utilizes one image to find the kernel and another to filter and vice versa. Detail images are obtained after subtraction of CBF output from the corresponding source images. With the help of domain transform filter, small scale features of the detail images are retrieved from the surrounding of large-scale structures of these images. These detail images are further processed via rolling guidance filter (RGF) which eliminates small-scale structures and refine edges to preserve large-scale structures of detail images. The weights are computed while measuring the strength of details in the detail images. The computed weights are then directly multiplied with source images followed by weight normalization in order to obtain final fusion result. Qualitative and quantitative experiments are performed on the publicly available datasets, where the results demonstrate that the proposed method outperforms the state-of-the-art, in terms of both visual effect and objective evaluation metrics. Our method generates clear and clean fused images which do not suffers from noise and artifacts while preserving essential information of the multimodal images.

Bio-Medical Image Segmentation using Wavelet Based Fusion Technique

In recent years, bio-medical image segmentation is established itself as base for image analysis. This article proposes a new method in developing a robust wavelet based medical image fusion technique for image segmentation. A GLCM (Gray Level Co-occurrence Matrix) based statistical method is used to extracts the texture features of the image decomposed at single level and the image is segmented based on region growing method. The combination of these two along with fusion technique gives a new segmented image. The results indicate the efficiency of the proposed method in segmenting the both normal cell images as well as darker cell images.

Multimodal fusion of tomographic sequences of medical images: MRE spatially enhanced by MRI

Background and Objective:In biomedical fields, image analysis is often necessary for an accurate diagnosis. In order to obtain all the information needed to form an in-depth clinical picture, it may be useful to combine the contents of images taken under different diagnostic modes. Multimodal medical image fusion techniques enable complementary information acquired by different imaging devices to be automatically combined into a unique image.Methods:In this paper, multimodal medical images fusion method based on multiresolution analysis (MRA) is proposed, with the aim to combine the high geometric content of magnetic resonance imaging (MRI) and the elasticity information of magnetic resonance elastography (MRE), simultaneously acquired on the same organs of a patient. First, the slices of MRE are volumetrically interpolated to exactly overlap, each with a slice of MRI. Then, the spatial details of MRI are extracted by means of MRA and injected into the corresponding slices of MRE. Due to the intrinsic dissimilarity between corresponding slices of MRE and MRI, the spatial details of MRI are modulated by local or global matching functions.Results:The performance of the proposed method is quantitatively assessed considering radiometric and geometric consistency of the fused images with respect to their originals, in a comparison with two popular methods from the literature. For a qualitative evaluation, a visual inspection is carried out.Conclusions:The results show that the proposed method enables an effective MRI-MRE fusion that allows the elasticity information and geometric details of the examined organs to be evaluated in a single image.

Multimodal Classification on PET/CT Image Fusion for Lung Cancer: A Comprehensive Survey

Medical image fusion has become essential for accurate diagnosis. For example, a lung cancer diagnosis is currently conducted with the help of multimodality image fusion to find anatomical and functional information about the tumor and metabolic measurements to identify the lung cancer stage and metastatic information of the disease. Generally, the success of multimodality imaging for lung cancer diagnosis is due to the combination of PET and CT imaging advantages while minimizing their respective limitations. However, medical image fusion involves the registration of two different modalities, which is time-consuming and technically challenging, and it is a cause of concern in a clinical setting. Therefore, the paper's main objective is to identify the most efficient medical image fusion techniques and the recent advances by conducting a collective survey. In addition, the study delves into the impact of deep learning techniques for image fusion and their effectiveness in automating the image fusion procedure with better image quality while preserving essential clinical information.