Multimodal Image Fusion Algorithm Research Articles

MFHOD: Multi-modal image fusion method based on the higher-order degradation model

The task of multimodal image fusion aims to preserve the respective advantages of each modality, such as the detailed texture information from visible light images and the salient target information from infrared images. However, images in real environments are often not perfect high-quality images but are affected by various degradation factors such as noise, blur, and glare. Existing unsupervised multimodal image fusion algorithms use undegraded source images as input to fusion networks and generate high-quality fusion images through complex fusion strategies. This single linear learning approach from high-quality to high-quality cannot be applied to the fusion task of degraded multimodal images in real environments. To address this issue, this paper proposes an end-to-end multimodal image fusion network based on a high-order local degradation model (MFHOD). Firstly, inspired by the idea of probabilistic degradation, we propose a high-order local random degradation model (HODM), which inputs the source multimodal images into the degradation model to obtain degraded images before feeding them into the network. Secondly, we design a simple and efficient dual-branch feature extraction encoder to extract deep features from images. Then, from the perspectives of image pixels, brightness, and gradients, we propose an improved composite loss composed of multiple loss functions to constrain network training. Finally, we propose a L2-norm fusion strategy for preserving brightness information in low-light nighttime images. Our MFHOD demonstrates good performance on infrared and visible light image datasets as well as medical image datasets. Experimental results show that MFHOD can effectively suppress the effects of glare, noise, and smoke in adverse environments, and also improve the quality of fusion images in low-light and nighttime environments.

Multimodal brain image fusion based on error texture elimination and salient feature detection.

As an important clinically oriented information fusion technology, multimodal medical image fusion integrates useful information from different modal images into a comprehensive fused image. Nevertheless, existing methods routinely consider only energy information when fusing low-frequency or base layers, ignoring the fact that useful texture information may exist in pixels with lower energy values. Thus, erroneous textures may be introduced into the fusion results. To resolve this problem, we propose a novel multimodal brain image fusion algorithm based on error texture removal. A two-layer decomposition scheme is first implemented to generate the high- and low-frequency subbands. We propose a salient feature detection operator based on gradient difference and entropy. The proposed operator integrates the gradient difference and amount of information in the high-frequency subbands to effectively identify clearly detailed information. Subsequently, we detect the energy information of the low-frequency subband by utilizing the local phase feature of each pixel as the intensity measurement and using a random walk algorithm to detect the energy information. Finally, we propose a rolling guidance filtering iterative least-squares model to reconstruct the texture information in the low-frequency components. Through extensive experiments, we successfully demonstrate that the proposed algorithm outperforms some state-of-the-art methods. Our source code is publicly available at https://github.com/ixilai/ETEM.

CSID: A Novel Multimodal Image Fusion Algorithm for Enhanced Clinical Diagnosis.

Technology-assisted clinical diagnosis has gained tremendous importance in modern day healthcare systems. To this end, multimodal medical image fusion has gained great attention from the research community. There are several fusion algorithms that merge Computed Tomography (CT) and Magnetic Resonance Images (MRI) to extract detailed information, which is used to enhance clinical diagnosis. However, these algorithms exhibit several limitations, such as blurred edges during decomposition, excessive information loss that gives rise to false structural artifacts, and high spatial distortion due to inadequate contrast. To resolve these issues, this paper proposes a novel algorithm, namely Convolutional Sparse Image Decomposition (CSID), that fuses CT and MR images. CSID uses contrast stretching and the spatial gradient method to identify edges in source images and employs cartoon-texture decomposition, which creates an overcomplete dictionary. Moreover, this work proposes a modified convolutional sparse coding method and employs improved decision maps and the fusion rule to obtain the final fused image. Simulation results using six datasets of multimodal images demonstrate that CSID achieves superior performance, in terms of visual quality and enriched information extraction, in comparison with eminent image fusion algorithms.

Multi-modal Image Fusion Algorithm based on Variable Parameter Fractional Difference Enhancement

Abstract Multi-modal image fusion can more accurately describe the features of a scene than a single image. Because of the different imaging mechanisms, the difference between multi-modal images is great, which leads to poor contrast of the fused images. Therefore, a simple and effective spatial domain fusion algorithm based on variable parameter fractional difference enhancement is proposed. Based on the characteristics of fractional difference enhancement, a variable parameter fractional difference is introduced, the multi-modal images are repeatedly enhanced, and multiple enhanced images are obtained. A correlation coefficient is applied to constrain the number of enhancement cycles. In addition, an energy contrast is used to extract the contrast features of the image, and the tangent function is simultaneously used to obtain the fusion weight to attain multiple contrast-enhanced initialization fusion images. Finally, the weighted average is applied to obtain the final fused image. Experimental results demonstrate that the proposed fusion algorithm can effectively preserve the contrast features between images and improve the quality of fused images.

Multimodal image fusion based on point-wise mutual information

Multimodal image fusion aims to generate a fused image from different signals that captured by multimodal sensors. Although the images obtained by multimodal sensors have different appearances, the information included in these images might be redundant and noisy. In the previous studies, the fusion rule and their properties that guiding how to merge the features from multiple images is relatively simple functions such as choose-max or weighted average. However, merging the features with redundant information based on these fusion rules may lead to brightness distortion or extra noises, since these fusion rules ignore the spatial consistency of feature selections. In this paper, we propose a novel multimodal image fusion algorithm that focuses on both the transferring the salient structures and maintaining spatial consistency that we consider as fundamental to build our proposed architecture. The proposed algorithm selects features to be transferred into fusion results by a graph cut algorithm, in which the spatial varying smoothness cost is formulated based on the independence between local features measured by point-wise mutual information (PMI). Experiment results demonstrate that, with the straightforward gradient features, the proposed method can obtain state-of-the-art performance on several publicly available multimodal image databases.

Color Medical Imaging Fusion Based on Principle Component Analysis and F-Transform

In last years, various medical image fusion algorithms have been proposed to fuse medical image. But, most of them focus on fusing grayscale images. This paper proposes a qualified algorithm for the fusion of multimodal color medical images. The technique of F-transforms has mainly been employed as a fusion technique for images obtained from equal or different modalities. The restriction of fused color mixing RGB, substitution method is resolved by incorporating F-transform and color mixing RGB. The proposed method significantly outperforms the traditional methods in terms of both visual quality and objective evaluation, with improved contrast and overall intensity. The proposed method provides better visual information than the gray ones and more adaptable to human vision. Additional, PCA is functional on the two-level decomposition to maximize the spatial resolution. Experimental evaluation demonstrates that the proposed algorithm qualitatively outperforms many existing state-of-the-art multimodal image fusion algorithms.

Multimodal Medical Image Fusion Using Various Hybrid Fusion Techniques For clinical Treatment Analysis

Medical image fusion is one the most significant and useful disease analytic techniques. This research paper proposed and examines some of the hybrid multimodality medical image fusion methods and discusses the most essential advantages and disadvantages of these methods to develop hybrid multimodal image fusion algorithms that improve the feature of merged multimodality therapeutic image. Computed Tomography, Magnetic Resonance Imaging, Positron Emission Tomography and Single Photon Emission Computed Tomography are the input multimodal therapeutic images used for fusion process. An experimental results of proposed all hybrid fusion techniques provides the best fused multimodal medical images of highest quality, highest details, shortest processing time, and best visualization. Both traditional and hybrid multimodal medical image fusion algorithms are evaluated using several quality metrics. Compared with other existing techniques the proposed technique experimental results demonstrate the better processing performance and results in both subjective and objective evaluation criteria. This is favorable, especially for helping in accurate clinical disease analysis.

Multimodal Medical Image Fusion Using Various Hybrid Fusion Techniques For clinical Treatment Analysis

Medical image fusion is one the most significant and useful disease analytic techniques. This research paper proposed and examines some of the hybrid multimodality medical image fusion methods and discusses the most essential advantages and disadvantages of these methods to develop hybrid multimodal image fusion algorithms that improve the feature of merged multimodality therapeutic image. Computed Tomography, Magnetic Resonance Imaging, Positron Emission Tomography and Single Photon Emission Computed Tomography are the input multimodal therapeutic images used for fusion process. An experimental results of proposed all hybrid fusion techniques provides the best fused multimodal medical images of highest quality, highest details, shortest processing time, and best visualization. Both traditional and hybrid multimodal medical image fusion algorithms are evaluated using several quality metrics. Compared with other existing techniques the proposed technique experimental results demonstrate the better processing performance and results in both subjective and objective evaluation criteria. This is favorable, especially for helping in accurate clinical disease analysis.

Multi-modal Color Medical Image Fusion Using Quaternion Discrete Fourier Transform

Multimodal image fusion is a process of combining multiple images, generated by identical or diverse imaging modalities, to get precise inside information about the same body organ. In recent years, various multimodal image fusion algorithms have been proposed to fuse medical image. However, most of them focus on fusing grayscale images. This paper proposes a novel algorithm for the fusion of multimodal color medical images. The proposed algorithm divides source images into blocks, converts each RGB block into quaternion representation and transforms them from special domain to frequency domain by applying quaternion discrete Fourier transform. The fused coefficients are obtained by calculating and comparing contrast values of corresponding coefficients in transformed blocks. The resultant fused image is reconstructed by merging all the blocks after applying inverse quaternion discrete Fourier transform on each block. Experimental evaluation demonstrates that the proposed algorithm qualitatively outperforms many existing state-of-the-art multimodal image fusion algorithms.

Region-Based Multimodal Image Fusion Using ICA Bases

In this paper, we present a novel multimodal image fusion algorithm in the independent component analysis (ICA) domain. Region-based fusion of ICA coefficients is implemented, where segmentation is performed in the spatial domain and ICA coefficients from separate regions are fused separately. The ICA coefficients from given regions are consequently weighted using the Piella fusion metric in order to maximize the quality of the fused image. The proposed method exhibits significantly higher performance than the basic ICA algorithm and also shows improvement over other state-of-the-art algorithms