Multimodal Medical Image Fusion Algorithms Research Articles

CDRNet: Cascaded dense residual network for grayscale and pseudocolor medical image fusion

ObjectiveMultimodal medical fusion images have been widely used in clinical medicine, computer-aided diagnosis and other fields. However, the existing multimodal medical image fusion algorithms generally have shortcomings such as complex calculations, blurred details and poor adaptability. To solve this problem, we propose a cascaded dense residual network and use it for grayscale and pseudocolor medical image fusion. MethodsThe cascaded dense residual network uses a multiscale dense network and a residual network as the basic network architecture, and a multilevel converged network is obtained through cascade. The cascaded dense residual network contains 3 networks, the first-level network inputs two images with different modalities to obtain a fused Image 1, the second-level network uses fused Image 1 as the input image to obtain fused Image 2 and the third-level network uses fused Image 2 as the input image to obtain fused Image 3. The multimodal medical image is trained through each level of the network, and the output fusion image is enhanced step-by-step. ResultsAs the number of networks increases, the fusion image becomes increasingly clearer. Through numerous fusion experiments, the fused images of the proposed algorithm have higher edge strength, richer details, and better performance in the objective indicators than the reference algorithms. ConclusionCompared with the reference algorithms, the proposed algorithm has better original information, higher edge strength, richer details and an improvement of the four objective SF, AG, MZ and EN indicator metrics.

Multimodal medical image fusion algorithm based on pulse coupled neural networks and nonsubsampled contourlet transform

Combining two medical images from different modalities is more helpful for using the resulting image in the healthcare field. Medical image fusion means combining two or more images coming from multiple sensors. This technology obtains an output image that presents more effective and useful information from two images. This paper proposes a multi-modal medical image fusion algorithm based on the nonsubsampled contourlet transform (NSCT) and pulse coupled neural networks (PCNN) methods. The input images are decomposed using the NSCT method into low- and high-frequency subbands. The PCNN is a fusion rule for integrating both low- and high-frequency subbands. The inverse of the NSCT method is to reconstruct the fused image. The results of medical image fusion help doctors with disease diagnosis and patient treatment. The proposed algorithm is tested on six groups of multi-modal medical images using 100 pairs of input images. The proposed algorithm is compared with eight fusion methods. We evaluate the performance of the proposed algorithm using the fusion metrics: peak signal to noise ratio (PSNR), mutual information (MI), entropy (EN), weighted edge information (Q^{AB/F}), nonlinear correlation information entropy (Q_{ncie}), standard deviation (SD), and average gradient (AG). Experimental results show that the proposed algorithm can perform better than other medical image fusion methods and achieve promising results.Graphical abstract

An Adaptive Image Fusion Algorithm in the NSST Based on CDF 9/7 for Neurodegenerative Diseases

The neurodegenerative disease such as: Parkinson's disease (PD), mild Alzheimer’s affects many people and has a serious influence on their life, With the quick advancement of computer-aided diagnostic (CAD) methods, early detection is crucial since effective treatment halts the spread of the disease. Image fusion is useful for medical diagnostics. In this paper we propose a multi-modality medical image fusion algorithm in NSST domain. Shearlets (NSST) are decomposed similarly to contourlets (NSCT), except that instead of applying the Laplacian pyramid followed by directional filtering, shearlets use a shear matrix. In this article the Biorthogonal CDF9/7 filter is applied in the shift-invariant shearlet filter banks, then the coefficients of low frequency bands are selected using maximum rule, and using the gradient in each subband high frequency image to motivate the modified pulse coupled neural networks (Modified PCNN). Finally reverse IHS to get the fused color image, all this to optimize the calculation performance and improve the characteristics of the fused image for medical diagnosis. Our approach was validated with several brain diseases modalities: Alzheimer’s…etc. The findings reveal that the suggested image fusion technique has a higher quality than those fused by previous algorithms existing.

MULTIMODAL FUSION ANALYSIS OF MEDICAL IMAGES BY DISCRETE WAVELET TRANSFORMATION

Medical image fusion is the development of registry and fusion of different pots of one or more imaging modalities in order to improve image value and reduce uncertainty and concern, to improve the applicability of the medical image for clinical opinion or the evaluation of medical care. difficulties. Multimodal medical image fusion algorithms and devices have achieved a remarkable result in improving the veracity of clinical decision support in medical images. The main objective is to improve the understanding of medical images using Discrete Wavelet Transform technology. DWT mainly uses blending rules involving the average pixel. Discrete wavelet transform has been implemented using fusion technology for medical image fusion. Fusion Power is calculated based on PSNR, MSE, and Total Progression Moment. The result demonstrates the success of the fusion scheme on wavelet transform imaging (MRI) and positron production tomography (PET). Other variants of MRI (magnetic resonance imaging) and PET (positron emission tomography) performed for medical diagnosis. Keywords: Multimodal fusion of medical images, fusion rules, magnetic signs,PET.

Multi-Modal Medical Image Fusion With Geometric Algebra Based Sparse Representation.

Multi-modal medical image fusion can reduce information redundancy, increase the understandability of images and provide medical staff with more detailed pathological information. However, most of traditional methods usually treat the channels of multi-modal medical images as three independent grayscale images which ignore the correlation between the color channels and lead to color distortion, attenuation and other bad effects in the reconstructed image. In this paper, we propose a multi-modal medical image fusion algorithm with geometric algebra based sparse representation (GA-SR). Firstly, the multi-modal medical image is represented as a multi-vector, and the GA-SR model is introduced for multi-modal medical image fusion to avoid losing the correlation of channels. Secondly, the orthogonal matching pursuit algorithm based on geometric algebra (GAOMP) is introduced to obtain the sparse coefficient matrix. The K-means clustering singular value decomposition algorithm based on geometric algebra (K-GASVD) is introduced to obtain the geometric algebra dictionary, and update the sparse coefficient matrix and dictionary. Finally, we obtain the fused image by linear combination of the geometric algebra dictionary and the coefficient matrix. The experimental results demonstrate that the proposed algorithm outperforms existing methods in subjective and objective quality evaluation, and shows its effectiveness for multi-modal medical image fusion.

Multimodal Medical Image Fusion Based on Content-based and PCA-sigmoid.

The objective of any multimodal medical image fusion algorithm is to assist a radiologist for better decision-making during the diagnosis and therapy by integrating the anatomical (magnetic resonance imaging) and functional (positron emission tomography/ single-photon emission computed tomography) information. We proposed a new medical image fusion method based on content-based decomposition, Principal Component Analysis (PCA), and sigmoid function. We considered Empirical Wavelet Transform (EWT) for content-based decomposition purposes since it can preserve crucial medical image information such as edges and corners. PCA is used to obtain initial weights corresponding to each detail layer. In our experiments, we found that direct usage of PCA for detail layer fusion introduces severe artifacts into the fused image due to weight scaling issues. In order to tackle this, we considered using the sigmoid function for better weight scaling. We considered 24 pairs of MRI-PET and 24 pairs of MRI-SPECT images for fusion, and the results are measured using four significant quantitative metrics. Finally, we compared our proposed method with other state-of-the-art transformbased fusion approaches, using traditional and recent performance measures. An appreciable improvement is observed in both qualitative and quantitative results compared to other fusion methods.

Nursing Analysis Based on Medical Imaging Technology before and after Coronary Angiography in Cardiovascular Medicine.

With the advancement of technology, medical imaging technology has been greatly improved. This article mainly studies the nursing before and after coronary angiography in cardiovascular medicine based on medical imaging technology. This paper proposes a multimodal medical image fusion algorithm based on multiscale decomposition and convolution sparse representation. The algorithm first decomposes the preregistered source medical image by NSST, takes the subimages of different scales as training images, and optimizes the subdictionaries of different scales; then convolution and sparse the subimages on each scale encoding to obtain the sparse coefficients of different subimages; secondly, the combination of improved L1 norm and improved spatial frequency (novel sum-modified SF (NMSF)) is used for high-frequency subimage coefficients, and the fusion of low-frequency subimages improved the rule of combining the L1 norm and the regional energy; finally, the final fused image is obtained by inverse NSST of the fused low-frequency subband and high-frequency subband. Experimental analysis found that the bifurcation angle has nothing to do with the damage of the branch vessels after the main branch stent is placed. The bifurcation angle greater than 50° is an independent predictor of MACE after stent extrusion for bifurcation lesions. Experimental results show that the proposed method has good performance in contrast enhancement, detail extraction, and information retention, and it improves the quality of the fusion image.

Two-Scale Multimodal Medical Image Fusion Based on Structure Preservation

Medical image fusion has an indispensable value in the medical field. Taking advantage of structure-preserving filter and deep learning, a structure preservation-based two-scale multimodal medical image fusion algorithm is proposed. First, we used a two-scale decomposition method to decompose source images into base layer components and detail layer components. Second, we adopted a fusion method based on the iterative joint bilateral filter to fuse the base layer components. Third, a convolutional neural network and local similarity of images are used to fuse the components of the detail layer. At the last, the final fused result is got by using two-scale image reconstruction. The contrast experiments display that our algorithm has better fusion results than the state-of-the-art medical image fusion algorithms.

Multimodal Medical Image Fusion Based on Multiple Latent Low-Rank Representation.

A multimodal medical image fusion algorithm based on multiple latent low-rank representation is proposed to improve imaging quality by solving fuzzy details and enhancing the display of lesions. Firstly, the proposed method decomposes the source image repeatedly using latent low-rank representation to obtain several saliency parts and one low-rank part. Secondly, the VGG-19 network identifies the low-rank part's features and generates the weight maps. Then, the fused low-rank part can be obtained by making the Hadamard product of the weight maps and the source images. Thirdly, the fused saliency parts can be obtained by selecting the max value. Finally, the fused saliency parts and low-rank part are superimposed to obtain the fused image. Experimental results show that the proposed method is superior to the traditional multimodal medical image fusion algorithms in the subjective evaluation and objective indexes.

DSAGAN: A generative adversarial network based on dual-stream attention mechanism for anatomical and functional image fusion

In recent years, extensive multimodal medical image fusion algorithms have been proposed. However, existing methods are primarily based on specific transformation theories. There are many problems with existing algorithms, such as poor adaptability, low efficiency and blurry details. To address these problems, this paper proposes a generative adversarial network based on dual-stream attention mechanism (DSAGAN) for anatomical and functional image fusion. The dual-stream architecture and multiscale convolutions are utilized to extract deep features. In addition, the attention mechanism is utilized to further enhance the fused features. Then, the fusion images and multimodal input images are put into the discriminator. In the update stage of the discriminator, we expect to judge the multimodal images as real, and to judge the fusion images as fake. Furthermore, the fusion images are expected to be judged as real in the update stage of the generator, forcing the generator to improve the fusion quality. The training process continues until the generator and discriminator reach a Nash equilibrium. After training, the fusion images can be obtained directly after inputting anatomical and functional images. Compared with the reference algorithms, DSAGAN consumes less fusion time and achieves better objective metrics in terms of QAG, QEN and QNIQE.

Multi-modal Medical Image Fusion Algorithm Based on Spatial Frequency Motivated PA-PCNN in the NSST Domain.

Image fusion has been grown as an effectual method in diseases related diagnosis schemes. In this paper, a new method for combining multimodal medical images using spatial frequency motivated parameter-adaptive PCNN (SF-PAPCNN) is suggested. The multi- modal images are disintegrated into frequency bands by using decomposition NSST. The coefficients of low frequency bands are selected using maximum rule. The coefficients of high frequency bands are combined by SF-PAPCNN. In this paper, a new method for combining multimodal medical images using spatial frequency motivated parameter-adaptive PCNN (SF-PAPCNN) is suggested. The multi-modal images are disintegrated into frequency bands by using decomposition NSST. The coefficients of low frequency bands are selected using maximum rule. The coefficients of high frequency bands are combined by SF-PAPCNN. The fused medical images is obtained by applying INSST to above coefficients. The quality metrics such as entropy ENT, fusion symmetry FS, deviation STD, mutual information QMI and edge strength QAB/F are used to validate the efficacy of suggested scheme.

Multi-modal brain image fusion based on multi-level edge-preserving filtering

Recently, multi-modal medical imaging technology and its collaborative diagnosis technology are developing rapidly. The application of medical image fusion technology in medical diagnosis becomes more important. In this paper, a multi-modal medical image fusion algorithm based on multi-level edge-preserving filtering (MLEPF) decomposition model is proposed. Firstly, an MLEPF model based on weighted mean curvature filtering is presented and used to decompose the multi-modal medical image into three types of layers: fine-structure (FS), coarse-structure (CS), and base (BS) layers. Secondly, a gradient domain pulse-coupled neural network (PCNN) fusion strategy is used to merge the FS and CS layers, and an energy attribute fusion strategy is used to merge the BS layers. Finally, the fused image is obtained by combining the three types of fused layers. The experiments are performed on six different disease datasets and one normal dataset, which contains more than 100 image pairs. Qualitative and quantitative evaluation testify that the proposed algorithm is superior to some excellent algorithms and can achieve close result to some state-of-the-art algorithms.

Multimodal Medical Image Fusion using Rolling Guidance Filter with CNN and Nuclear Norm Minimization.

Medical image fusion is very important for the diagnosis and treatment of diseases. In recent years, there have been a number of different multi-modal medical image fusion algorithms that can provide delicate contexts for disease diagnosis more clearly and more conveniently. Recently, nuclear norm minimization and deep learning have been used effectively in image processing. A multi-modality medical image fusion method using a rolling guidance filter (RGF) with a convolutional neural network (CNN) based feature mapping and nuclear norm minimization (NNM) is proposed. At first, we decompose medical images to base layer components and detail layer components by using RGF. In the next step, we get the basic fused image through the pretrained CNN model. The CNN model with pre-training is used to obtain the significant characteristics of the base layer components. And we can compute the activity level measurement from the regional energy of CNN-based fusion maps. Then, a detail fused image is gained by NNM. That is, we use NNM to fuse the detail layer components. At last, the basic and detail fused images are integrated into the fused result. From the comparison with the most advanced fusion algorithms, the results of experiments indicate that this fusion algorithm has the best effect in visual evaluation and objective standard. The fusion algorithm using RGF and CNN-based feature mapping, combined with NNM, can improve fusion effects and suppress artifacts and blocking effects in the fused results.

A novel multi-modal medical image fusion algorithm

In order to improve the contrast of image fusion and highlight the unique characteristics of medical images, a multi-modal medical image fusion algorithm in the framework of non-subsampled contourlet transform (NSCT) is proposed in this paper. Firstly, the computed tomography images and magnetic resonance image are decomposed into low- and high-frequency sub-bands through the NSCT of multi-scale geometric transformation; secondly, for the low-frequency sub-band, the local area standard deviation method is selected or the fusion, while for the high-frequency sub-band, an adaptive pulse coupling neural network model is constructed and the fusion rules are set by the cumulative ignition times of iterative operation in the network; finally, the fusion image is obtained through image reconstruction. Experimental results show that the fusion results of the algorithm in this paper can improve the image fusion quality significantly and it has certain advantages in both visual effects and objective evaluation indexes, which provides a more reliable basis for clinical diagnosis and treatment of diseases.

Multimodal medical image fusion algorithm in the era of big data

AbstractIn image-based medical decision-making, different modalities of medical images of a given organ of a patient are captured. Each of these images will represent a modality that will render the examined organ differently, leading to different observations of a given phenomenon (such as stroke). The accurate analysis of each of these modalities promotes the detection of more appropriate medical decisions. Multimodal medical imaging is a research field that consists in the development of robust algorithms that can enable the fusion of image information acquired by different sets of modalities. In this paper, a novel multimodal medical image fusion algorithm is proposed for a wide range of medical diagnostic problems. It is based on the application of a boundary measured pulse-coupled neural network fusion strategy and an energy attribute fusion strategy in a non-subsampled shearlet transform domain. Our algorithm was validated in dataset with modalities of several diseases, namely glioma, Alzheimer’s, and metastatic bronchogenic carcinoma, which contain more than 100 image pairs. Qualitative and quantitative evaluation verifies that the proposed algorithm outperforms most of the current algorithms, providing important ideas for medical diagnosis.

Enhancement of Hybrid Multimodal Medical Image Fusion Techniques for Clinical Disease Analysis

Multimodal medical image fusion is one the most significant and useful disease analytic techniques. This research article proposes the hybrid multimodality medical image fusion methods and discusses the most essential advantages and disadvantages of these methods. The hybrid multimodal medical image fusion algorithms are used to improve the quality of fused multimodality medical image. Magnetic resonance imaging, positron emission tomography, and single photon emission computed tomography are the input multimodal therapeutic images used for fusion process. An experimental result of proposed hybrid fusion techniques provides the fused multimodal medical images of highest quality, shortest processing time, and best visualization. Both traditional and hybrid multimodal medical image fusion algorithms are evaluated using several quality metrics. Compared with existing techniques the proposed result gives the better processing performance in both qualitative and quantitative evaluation criteria. This is favorable, especially for helping in accurate clinical disease analysis.

Multimodal Medical Image Fusion Using Various Hybrid Fusion Techniques For clinical Treatment Analysis

Medical image fusion is one the most significant and useful disease analytic techniques. This research paper proposed and examines some of the hybrid multimodality medical image fusion methods and discusses the most essential advantages and disadvantages of these methods to develop hybrid multimodal image fusion algorithms that improve the feature of merged multimodality therapeutic image. Computed Tomography, Magnetic Resonance Imaging, Positron Emission Tomography and Single Photon Emission Computed Tomography are the input multimodal therapeutic images used for fusion process. An experimental results of proposed all hybrid fusion techniques provides the best fused multimodal medical images of highest quality, highest details, shortest processing time, and best visualization. Both traditional and hybrid multimodal medical image fusion algorithms are evaluated using several quality metrics. Compared with other existing techniques the proposed technique experimental results demonstrate the better processing performance and results in both subjective and objective evaluation criteria. This is favorable, especially for helping in accurate clinical disease analysis.

Multimodal Medical Image Fusion Using Various Hybrid Fusion Techniques For clinical Treatment Analysis

Medical image fusion is one the most significant and useful disease analytic techniques. This research paper proposed and examines some of the hybrid multimodality medical image fusion methods and discusses the most essential advantages and disadvantages of these methods to develop hybrid multimodal image fusion algorithms that improve the feature of merged multimodality therapeutic image. Computed Tomography, Magnetic Resonance Imaging, Positron Emission Tomography and Single Photon Emission Computed Tomography are the input multimodal therapeutic images used for fusion process. An experimental results of proposed all hybrid fusion techniques provides the best fused multimodal medical images of highest quality, highest details, shortest processing time, and best visualization. Both traditional and hybrid multimodal medical image fusion algorithms are evaluated using several quality metrics. Compared with other existing techniques the proposed technique experimental results demonstrate the better processing performance and results in both subjective and objective evaluation criteria. This is favorable, especially for helping in accurate clinical disease analysis.

Detail-enhanced multimodality medical image fusion based on gradient minimization smoothing filter and shearing filter.

In this paper, a detail-enhanced multimodality medical image fusion algorithm is proposed by using proposed multi-scale joint decomposition framework (MJDF) and shearing filter (SF). The MJDF constructed with gradient minimization smoothing filter (GMSF) and Gaussian low-pass filter (GLF) is used to decompose source images into low-pass layers, edge layers, and detail layers at multiple scales. In order to highlight the detail information in the fused image, the edge layer and the detail layer in each scale are weighted combined into a detail-enhanced layer. As directional filter is effective in capturing salient information, so SF is applied to the detail-enhanced layer to extract geometrical features and obtain directional coefficients. Visual saliency map-based fusion rule is designed for fusing low-pass layers, and the sum of standard deviation is used as activity level measurement for directional coefficients fusion. The final fusion result is obtained by synthesizing the fused low-pass layers and directional coefficients. Experimental results show that the proposed method with shift-invariance, directional selectivity, and detail-enhanced property is efficient in preserving and enhancing detail information of multimodality medical images. Graphical abstract The detailed implementation of the proposed medical image fusion algorithm.

An improved multimodal medical image fusion algorithm based on fuzzy transform

Multimodal medical image fusion has become a powerful tool in clinical applications. The main aim is to fuse different multimodal medical images, obtained from different imaging modalities, into a single fused image that is extensively used by the physicians for explicit diagnosis and treatment of diseases. In this paper, an improved multimodal medical image fusion algorithm based on fuzzy transform (FTR) is proposed. The core idea behind the proposed algorithm is to improve the performance of multimodal medical image fusion algorithm by taking into consideration the error images obtained using FTR pair. Subjective as well as objective evaluations demonstrate that the fusion quality in terms of edge strength, standard deviation, feature mutual information, fusion factor, feature similarity and structural similarity has significantly improved in the proposed algorithm as compared to other state-of-art multimodal medical image fusion algorithms.