Multimodal Medical Image Fusion Method Research Articles

Fine grained medical image fusion using type-2 fuzzy logic

<p>In recent years, many fast-growing technologies coupled with wide volume of medical data for the digitalization of that data. Thus, researchers have shown their immense interest on Multi-sensor image fusion technologies which convey image information based on data from various sensor modalities into a single image. The image fusion technique is a widespread technique for the diagnosis of medical instrumentation and measurement. Therefore, in this paper we have introduced a novel multimodal sensor medical image fusion method based on type-2 fuzzy logic is proposed using Sugeno model. Moreover, a Gaussian smoothen filter is introduced to extract the detailed information of an image using sharp feature points.Type-2 fuzzy algorithm is used to achieve highly efficient feature points from both the b images to provide high visually classified resultant image. The experimental results demonstrate that the proposed method can achieve better performance than the state-of-the- art methods in terms of visual quality and objective evaluation.</p>

A Phase Congruency and Local Laplacian Energy Based Multi-Modality Medical Image Fusion Method in NSCT Domain

Multi-modality image fusion provides more comprehensive and sophisticated information in modern medical diagnosis, remote sensing, video surveillance, and so on. This paper presents a novel multi-modality medical image fusion method based on phase congruency and local Laplacian energy. In the proposed method, the non-subsampled contourlet transform is performed on medical image pairs to decompose the source images into high-pass and low-pass subbands. The high-pass subbands are integrated by a phase congruency-based fusion rule that can enhance the detailed features of the fused image for medical diagnosis. A local Laplacian energy-based fusion rule is proposed for low-pass subbands. The local Laplacian energy consists of weighted local energy and the weighted sum of Laplacian coefficients that describe the structured information and the detailed features of source image pairs, respectively. Thus, the proposed fusion rule can simultaneously integrate two key components for the fusion of low-pass subbands. The fused high-pass and low-pass subbands are inversely transformed to obtain the fused image. In the comparative experiments, three categories of multi-modality medical image pairs are used to verify the effectiveness of the proposed method. The experiment results show that the proposed method achieves competitive performance in both the image quantity and computational costs.

Medical Image Fusion via PCNN Based on Edge Preservation and Improved Sparse Representation in NSST Domain

Medical image fusion integrates image features of different modalities to provide comprehensive information for clinical diagnosis, treatment planning, and image-guided surgery. The information of the fused image is richer and clearer, which makes up for the defect of the single-mode medical image and retains the characteristic information of the source image. This paper proposes a novel multi-modality medical image fusion method based on gray medical images and color medical images. In the proposed method, the nonsubsampled shearlet transform (NSST) method is first used to decompose the source image into a low frequency sub-band and several high frequency sub-bands. The improved sparse representation is utilized to fuse the low frequency sub-band, which can remove the detail features through the sobel operator and the guided filter to improve the ability to preserve energy effectively. Meanwhile, the high frequency sub-bands are fused by a pulse coupled neural network (PCNN) based on edge preservation. This method fully considers the imaging characteristics of different medical modalities, which can process edge information well and process image details better. Finally, the fused low frequency sub-band and high frequency sub-bands are inversely transformed to obtain the final fused image. Seven different categories of medical images and seven fusion methods are used to verify the effectiveness of the proposed method. In addition, the medical images of three different modalities are merged to testify the influence of the fusion sequence. The experimental results show that the proposed method is superior to existing state-of-art methods in subjective visual performance and objective evaluation.

Medical Image Fusion With Parameter-Adaptive Pulse Coupled Neural Network in Nonsubsampled Shearlet Transform Domain

As an effective way to integrate the information contained in multiple medical images with different modalities, medical image fusion has emerged as a powerful technique in various clinical applications such as disease diagnosis and treatment planning. In this paper, a new multimodal medical image fusion method in nonsubsampled shearlet transform (NSST) domain is proposed. In the proposed method, the NSST decomposition is first performed on the source images to obtain their multiscale and multidirection representations. The high-frequency bands are fused by a parameter-adaptive pulse-coupled neural network (PA-PCNN) model, in which all the PCNN parameters can be adaptively estimated by the input band. The low-frequency bands are merged by a novel strategy that simultaneously addresses two crucial issues in medical image fusion, namely, energy preservation and detail extraction. Finally, the fused image is reconstructed by performing inverse NSST on the fused high-frequency and low-frequency bands. The effectiveness of the proposed method is verified by four different categories of medical image fusion problems [computed tomography (CT) and magnetic resonance (MR), MR-T1 and MR-T2, MR and positron emission tomography, and MR and single-photon emission CT] with more than 80 pairs of source images in total. Experimental results demonstrate that the proposed method can obtain more competitive performance in comparison to nine representative medical image fusion methods, leading to state-of-the-art results on both visual quality and objective assessment.

Multimodal Medical Image Fusion Based on Fuzzy Discrimination With Structural Patch Decomposition.

Multimodal medical image fusion, emerging as a hot topic, aims to fuse images with complementary multi-source information. In this paper, we propose a novel multimodal medical image fusion method based on structural patch decomposition (SPD) and fuzzy logic technology. First, the SPD method is employed to extract two salient features for fusion discrimination. Next, two novel fusion decision maps called an incomplete fusion map and supplemental fusion map are constructed from salient features. In this step, the supplemental map is constructed by our defined two different fuzzy logic systems. The supplemental and incomplete maps are then combined to construct an initial fusion map. The final fusion map is obtained by processing the initial fusion map with a Gaussian filter. Finally, a weighted average approach is adopted to create the final fused image. Additionally, an effective color medical image fusion scheme that can effectively prevent color distortion and obtain superior diagnostic effects is also proposed to enhance fused images. Experimental results clearly demonstrate that the proposed method outperforms state-of-the-art methods in terms of subjective visual and quantitative evaluations.

Enhancement of Hybrid Multimodal Medical Image Fusion Techniques for Clinical Disease Analysis

Multimodal medical image fusion is one the most significant and useful disease analytic techniques. This research article proposes the hybrid multimodality medical image fusion methods and discusses the most essential advantages and disadvantages of these methods. The hybrid multimodal medical image fusion algorithms are used to improve the quality of fused multimodality medical image. Magnetic resonance imaging, positron emission tomography, and single photon emission computed tomography are the input multimodal therapeutic images used for fusion process. An experimental result of proposed hybrid fusion techniques provides the fused multimodal medical images of highest quality, shortest processing time, and best visualization. Both traditional and hybrid multimodal medical image fusion algorithms are evaluated using several quality metrics. Compared with existing techniques the proposed result gives the better processing performance in both qualitative and quantitative evaluation criteria. This is favorable, especially for helping in accurate clinical disease analysis.

Multimodal Medical Image Fusion Using Various Hybrid Fusion Techniques For clinical Treatment Analysis

Medical image fusion is one the most significant and useful disease analytic techniques. This research paper proposed and examines some of the hybrid multimodality medical image fusion methods and discusses the most essential advantages and disadvantages of these methods to develop hybrid multimodal image fusion algorithms that improve the feature of merged multimodality therapeutic image. Computed Tomography, Magnetic Resonance Imaging, Positron Emission Tomography and Single Photon Emission Computed Tomography are the input multimodal therapeutic images used for fusion process. An experimental results of proposed all hybrid fusion techniques provides the best fused multimodal medical images of highest quality, highest details, shortest processing time, and best visualization. Both traditional and hybrid multimodal medical image fusion algorithms are evaluated using several quality metrics. Compared with other existing techniques the proposed technique experimental results demonstrate the better processing performance and results in both subjective and objective evaluation criteria. This is favorable, especially for helping in accurate clinical disease analysis.

Multimodal Medical Image Fusion Using Various Hybrid Fusion Techniques For clinical Treatment Analysis

Medical image fusion is one the most significant and useful disease analytic techniques. This research paper proposed and examines some of the hybrid multimodality medical image fusion methods and discusses the most essential advantages and disadvantages of these methods to develop hybrid multimodal image fusion algorithms that improve the feature of merged multimodality therapeutic image. Computed Tomography, Magnetic Resonance Imaging, Positron Emission Tomography and Single Photon Emission Computed Tomography are the input multimodal therapeutic images used for fusion process. An experimental results of proposed all hybrid fusion techniques provides the best fused multimodal medical images of highest quality, highest details, shortest processing time, and best visualization. Both traditional and hybrid multimodal medical image fusion algorithms are evaluated using several quality metrics. Compared with other existing techniques the proposed technique experimental results demonstrate the better processing performance and results in both subjective and objective evaluation criteria. This is favorable, especially for helping in accurate clinical disease analysis.

IGM‐based perceptual multimodal medical image fusion using free energy motivated adaptive PCNN

AbstractMultimodal medical image fusion merges two medical images to produce a visual enhanced fused image, to provide more accurate comprehensive pathological information to doctors for better diagnosis and treatment. In this article, we present a perceptual multimodal medical image fusion method with free energy (FE) motivated adaptive pulse coupled neural network (PCNN) by employing Internal Generative Mechanism (IGM). First, source images are divided into predicted layers and detail layers with Bayesian prediction model. Then to retain human visual system inspired features, FE is used to motivate the PCNN for processing detail layers, and large firing times are selected as coefficients. The predicted layers are fused with the averaging strategy as activity level measurement. Finally, the fused image is reconstructed by merging coefficients in both fused layers. Experimental results visually and quantitatively show that the proposed fusion strategy is superior to the state‐of‐the‐art methods.

Multimodal medical image fusion by cloud model theory

Image fusion can provide more extensive information since it combines two or more different images. Cloud model is a recently proposed theory in artificial intelligence and has the advantage of taking the randomness and fuzziness into account. In this paper, we introduce a novel multimodal medical image fusion method by cloud model theory. The proposed method fits the histograms of input images using the high-order spline function firstly and then divides intervals in line with the valley point of the fitted curve. On this basis, cloud models are generated adaptively through the reverse cloud generator. Finally, cloud reasoning rules are designed to achieve the fused image. Experimental results demonstrate that the fused images by proposed method show more image details and lesion regions than existing methods. The objective image quality assessment metrics on the fused images also show the superiority of the proposed method.

Adopting Quaternion Wavelet Transform to Fuse Multi-Modal Medical Images

Medical image fusion plays an important role in clinical applications such as image-guided surgery, image-guided radiotherapy, noninvasive diagnosis, and treatment planning. In this paper, we propose a novel multi-modal medical image fusion method based on simplified pulse-coupled neural network and quaternion wavelet transform. The proposed fusion algorithm is capable of combining not only pairs of computed tomography (CT) and magnetic resonance (MR) images, but also pairs of CT and proton-density-weighted MR images, and multi-spectral MR images such as T1 and T2. Experiments on six pairs of multi-modal medical images are conducted to compare the proposed scheme with four existing methods. The performances of various methods are investigated using mutual information metrics and comprehensive fusion performance characterization (total fusion performance, fusion loss, and modified fusion artifacts criteria). The experimental results show that the proposed algorithm not only extracts more important visual information from source images, but also effectively avoids introducing artificial information into fused medical images. It significantly outperforms existing medical image fusion methods in terms of subjective performance and objective evaluation metrics.

Multimodal medical image fusion based on discrete Tchebichef moments and pulse coupled neural network

ABSTRACTMultimodal medical image fusion plays a vital role in clinical diagnoses and treatment planning. In many image fusion methods‐based pulse coupled neural network (PCNN), normalized coefficients are used to motivate the PCNN, and this makes the fused image blur, detail loss, and decreases contrast. Moreover, they are limited in dealing with medical images with different modalities. In this article, we present a new multimodal medical image fusion method based on discrete Tchebichef moments and pulse coupled neural network to overcome the aforementioned problems. First, medical images are divided into equal‐size blocks and the Tchebichef moments are calculated to characterize image shape, and energy of blocks is computed as the sum of squared non‐DC moment values. Then to retain edges and textures, the energy of Tchebichef moments for blocks is introduced to motivate the PCNN with adaptive linking strength. Finally, large firing times are selected as coefficients of the fused image. Experimental results show that the proposed scheme outperforms state‐of‐the‐art methods and it is more effective in processing medical images with different modalities. © 2017 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 27, 57–65, 2017

Contextual Information Driven Multi-modal Medical Image Fusion

ABSTRACTTo utilize context correlation between coefficients in contourlet domain, a novel multi-modal medical image fusion method based on contextual information is proposed. First, the context information of contourlet coefficients are calculated to capture the strong dependencies of coefficients. Second, hidden Markov model based on context information for the contourlet coefficients (C-CHMM) is constructed to describe the characteristics of medical image in a small number of parameters. Further, low-pass subband coefficients are combined by magnitude maximum rule and high-pass subband coefficients are fused by a new C-CHMM driven multi-strategy fusion rule. Finally, the fused image is obtained by inverse contourlet transform. Experimental results demonstrate that the proposed fusion method can effectively suppress the color distortion and provide a better fusion quality compared with some typical fusion methods.

Research advances in multi-modality medical image registration and fusion methods and their clinical application

Multi-modality medical image processing has become a hot topic for research in the field of image processing and plays an important role in clinical diagnosis and treatment. Images with different modalities provide different information on patients. Anatomical images (such as computed tomography and magnetic resonance imaging) provide information on anatomical morphology and the structure of human body, and functional images (such as single-photon emission computed tomography and positron emission tomography) provide the functional information on the distribution of radioactive concentration within human body. Such information needs to be fused to obtain comprehensive fusion images, and the images with different modalities need to be registered to obtain useful fusion images. This article reviews several image registration and fusion techniques used in the medical field, points out their advantages and shortcomings, and introduces the application of various processing techniques in clinical practice. Key words: Image processing; Image registration; Image fusion

Multimodal Sensor Medical Image Fusion Based on Type-2 Fuzzy Logic in NSCT Domain

Multimodal medical image fusion plays a vital role in different clinical imaging sensor applications. This paper presents a novel multimodal medical image fusion method that adopts a multiscale geometric analysis of the nonsubsampled contourlet transform (NSCT) with type-2 fuzzy logic techniques. First, the NSCT was performed on preregistered source images to obtain their high- and low-frequency subbands. Next, an effective type-2 fuzzy logic-based fused rule is proposed for fusion of the high-frequency subbands. In the presented fusion approach, the local type-2 fuzzy entropy is introduced to automatically select high-frequency coefficients. However, for the low-frequency subbands, they were fused by a local energy algorithm based on the corresponding image’s local features. Finally, the fused image was constructed by the inverse NSCT with all composite subbands. Both subjective and objective evaluations showed better contrast, accuracy, and versatility in the proposed approach compared with state-of-the-art methods. Besides, an effective color medical image fusion scheme is also given in this paper that can inhibit color distortion to a large extent and produce an improved visual effect.

Medical image fusion using discrete fractional wavelet transform

A multimodal medical image fusion method based on discrete fractional wavelet (DFRWT) is presented in this paper. With a change in p order in domain (0,1], source medical images are decomposed by DFRWT in different p order. The sparsity character of the mode coefficients in subband images changes. According to the method, to enhance the correlation between subband coefficients, the non-sparsity character of the mode coefficients in low p order should be utilized. The coefficients of the all subbands are fused using the weighted regional variance rule. Finally, inverse DFRWT is applied to obtain a fused image. Subjective and objective analyses of the results and comparisons with other multiresolution domain techniques show the effectiveness of the proposed scheme in fusing multimodal medical images.

Multi-modal Medical Image Fusion Based on Non-subsampled Shearlet Transform

In order to provide more comprehensive and effective information for cancer diagnosis and tumor treatment planning, it is necessary to fuse multi-modal medical images, such as CT/MRI image, CT/PET image, MRI/SPECT image and so on. In this paper, a multimodal medical image fusion method based on Non-subsampled Shearlet Transform is proposed. Firstly, in this method, source images are decomposed into low-pass and highpass subbands by NSST. Then, due to the characteristic features--large sparsity and strong contrast, the high-frequency and low-frequency coefficients of the images are fused by the different fusion rules. Finally, the image is reconstructed by the inverse nonsampled shearlet transform. In the method, the fusion rules are designed based on the regional energy and the average gradient; the image entropy, relative quality, average gradient, standard deviation and spatial frequency were used to evaluate the fusion results objectively. In the experiments, CT and MRI images are chosen to verify the method. Both the visual and the objective analysis show that the proposed method is better than the conventional Wavelet-based and non-subsampled Contourlet-based methods.

Robust multi-modal medical image fusion via anisotropic heat diffusion guided low-rank structural analysis

This paper proposes a novel and robust multi-modal medical image fusion method, which is built upon a novel framework comprising multi-scale image decomposition based on anisotropic heat kernel design, scale-aware salient information extraction based on low-rank analysis, and scale-specific fusion rules. Our framework respects multi-scale structure features, while being robust to complex noise perturbation. First, anisotropic heat kernel is computed by constructing an image pyramid and embedding multi-level image properties into 2D manifolds in a divide-and-conquer way, consequently, multi-scale structure-preserving image decomposition can be accommodated. Second, to extract meaningfully scale-aware salient information, we conduct low-rank analysis over the image layer groups obtained in the first step, and employ the low-rank components to form the scale space of the salient features, wherein the underlying noise can be synchronously decoupled in a natural way. Third, to better fuse the complementary salient information extracted from multi-modal images, we design an S-shaped weighting function to fuse the large-scale layers, and employ the maximum selection principle to handle the small-scale layers. Moreover, we have conducted extensive experiments on MRI and PET/SPECT images. The comprehensive and quantitative comparisons with state-of-the-art methods demonstrate the informativeness, accuracy, robustness, and versatility of our novel approach.

A novel multi-modal medical image fusion method based on shift-invariant shearlet transform

Medical image fusion plays an important role in clinical applications, such as image-guided surgery, image-guided radiotherapy, non-invasive diagnosis and treatment planning. Shearlet is a novel multi-scale geometric analysis (MGA) tool proposed recently. In order to overcome the drawback of the shearlet-based fusion methods that the pseudo-Gibbs phenomenon is easily caused around the singularities of the fused image, a new multi-modal medical image fusion method is proposed in shift-invariant shearlet transform domain. First, the original images are decomposed into lowpass sub-bands and highpass sub-bands; then, the lowpass sub-bands and high sub-bands are combined according to the fusion rules, respectively. All the operations are performed in shift-invariant shearlet domain. The final fused image is obtained by directly applying inverse shift-invariant shearlet transform to the fused lowpass sub-bands and highpass sub-bands. Experimental results demonstrate that the proposed method can not only suppress the pseudo-Gibbs phenomenon efficiently, but perform better than the popular wavelet transform-based method, contourlet transform-based method and non-subsampled contourlet transform-based method.

NSCT-based multimodal medical image fusion using pulse-coupled neural network and modified spatial frequency

In this article, a novel multimodal medical image fusion (MIF) method based on non-subsampled contourlet transform (NSCT) and pulse-coupled neural network (PCNN) is presented. The proposed MIF scheme exploits the advantages of both the NSCT and the PCNN to obtain better fusion results. The source medical images are first decomposed by NSCT. The low-frequency subbands (LFSs) are fused using the 'max selection' rule. For fusing the high-frequency subbands (HFSs), a PCNN model is utilized. Modified spatial frequency in NSCT domain is input to motivate the PCNN, and coefficients in NSCT domain with large firing times are selected as coefficients of the fused image. Finally, inverse NSCT (INSCT) is applied to get the fused image. Subjective as well as objective analysis of the results and comparisons with state-of-the-art MIF techniques show the effectiveness of the proposed scheme in fusing multimodal medical images.