Visible Image Fusion Algorithm Research Articles

TGFuse: An Infrared and Visible Image Fusion Approach Based on Transformer and Generative Adversarial Network.

The end-to-end image fusion framework has achieved promising performance, with dedicated convolutional networks aggregating the multi-modal local appearance. However, long-range dependencies are directly neglected in existing CNN fusion approaches, impeding balancing the entire image-level perception for complex scenario fusion. In this paper, therefore, we propose an infrared and visible image fusion algorithm based on the transformer module and adversarial learning. Inspired by the global interaction power, we use the transformer technique to learn the effective global fusion relations. In particular, shallow features extracted by CNN are interacted in the proposed transformer fusion module to refine the fusion relationship within the spatial scope and across channels simultaneously. Besides, adversarial learning is designed in the training process to improve the output discrimination via imposing competitive consistency from the inputs, reflecting the specific characteristics in infrared and visible images. The experimental performance demonstrates the effectiveness of the proposed modules, with superior improvement against the state-of-the-art, generalising a novel paradigm via transformer and adversarial learning in the fusion task.

Infrared and visible image fusion of generative adversarial network based on multi-channel encoding and decoding

Image fusion can overcome the shortcomings of unclear, inaccurate and incomplete target description of single sensor image, achieve more accurate expression of target, and lay a foundation for subsequent image analysis and processing. Aiming at the existing algorithms based on generative adversarial networks only relying on the adversarial training of discriminators to achieve image fusion, and the balance of the respective features of infrared and visible images is not comprehensive or even fine enough, an infrared and visible image fusion algorithm based on multi-channel encoding and decoding generative adversarial network was proposed in this paper. Compared with the existing image decomposition methods, a triple codec structure is designed for feature extraction, which can use the generator itself to get a fusion image with extraordinary effect, and then use the discriminator to improve the quality of the fusion result. Secondly, a contrast enhancement module is designed after the infrared path structure of the encoder in the generator to enhance the contrast information of the target area. In addition, a dual discriminator model is designed, and in order to improve the feature learning ability of the generator, an incentive module is designed at the end of the discriminator. Finally, gradient loss and contrast loss are designed, and the training speed is improved by combining specific functions. Experimental results show that compared with the eleven classical algorithms, almost all of the proposed algorithms have better visual effects, and their quantitative indicators are also better.

An ISAR and Visible Image Fusion Algorithm Based on Adaptive Guided Multi-Layer Side Window Box Filter Decomposition

Traditional image fusion techniques generally use symmetrical methods to extract features from different sources of images. However, these conventional approaches do not resolve the information domain discrepancy from multiple sources, resulting in the incompleteness of fusion. To solve the problem, we propose an asymmetric decomposition method. Firstly, an information abundance discrimination method is used to sort images into detailed and coarse categories. Then, different decomposition methods are proposed to extract features at different scales. Next, different fusion strategies are adopted for different scale features, including sum fusion, variance-based transformation, integrated fusion, and energy-based fusion. Finally, the fusion result is obtained through summation, retaining vital features from both images. Eight fusion metrics and two datasets containing registered visible, ISAR, and infrared images were adopted to evaluate the performance of the proposed method. The experimental results demonstrate that the proposed asymmetric decomposition method could preserve more details than the symmetric one, and performed better in both objective and subjective evaluations compared with the fifteen state-of-the-art fusion methods. These findings can inspire researchers to consider a new asymmetric fusion framework that can adapt to the differences in information richness of the images, and promote the development of fusion technology.

An Infrared and Visible Image Fusion Algorithm Method Based on a Dual Bilateral Least Squares Hybrid Filter

Infrared and visible images of the same scene are fused to produce a fused image with richer information. However, most current image-fusion algorithms suffer from insufficient edge information retention, weak feature representation, and poor contrast, halos, and artifacts, and can only be applied to a single scene. To address these issues, we propose a novel infrared and visual image fusion algorithm based on a bilateral–least-squares hybrid filter (DBLSF) with the least-squares and bilateral filter hybrid model (BLF-LS). The proposed algorithm utilizes the residual network ResNet50 and the adaptive fusion strategy of the structure tensor to fuse the base and detail layers of the filter decomposition, respectively. Experiments on 32 sets of images from the TNO image-fusion dataset show that, although our fusion algorithm sacrifices overall time efficiency, the Combination 1 approach can better preserve image edge information and image integrity; reduce the loss of source image features; suppress artifacts and halos; and compare favorably with other algorithms in terms of structural similarity, feature similarity, multiscale structural similarity, root mean square error, peak signal-to-noise ratio, and correlation coefficient by at least 2.71%, 1.86%, 0.09%, 0.46%, 0.24%, and 0.07%; and the proposed Combination 2 can effectively improve the contrast and edge features of the fused image and enrich the image detail information, with an average improvement of 37.42%, 26.40%, and 26.60% in the three metrics of average gradient, edge intensity, and spatial frequency compared with other algorithms.

Infrared and visible image fusion algorithm based on structure-texture decomposition

Infrared and visible image fusion algorithm based on structure-texture decomposition

MGFuse: An Infrared and Visible Image Fusion Algorithm Based on Multiscale Decomposition Optimization and Gradient-Weighted Local Energy

Existing image fusion algorithms have difficulty in effectively preserving valuable target features in infrared and visible images, which easily introduces blurry edges and unremarkable notable targets during their fusion process. We propose the MGFuse algorithm as a solution to this problem, which is a novel fusion algorithm that utilizes multiscale decomposition optimization and gradient-weighted local energy. Initially, non-subsampled shearlet transform (NSST) is applied to partition both the infrared and visible images into several high-frequencies and low-frequencies components. Subsequently, the acquired low frequencies continue to be decomposed via proposed optimization function to get base layers and texture layers, which can optimize the quality of image edges and preserve fine-grained details, respectively. In addition, we have formulated an intrinsic attribute-based energy (IAE) fusion scheme to merge the two base layers. The texture layers and high-frequencies are extracted by gradient-weighted local energy (GE) operator based on structure tensor, which are employed to construct the fusion strategy for these parts. At last, the acquired texture and base parts are linearly combined to get the integrated low-frequency layer on which the final image is acquired using inverse NSST. Numerous experimental observations demonstrate that our MGFuse algorithm achieve superior fusion capability than the reference nine advanced algorithms in both qualitative and quantitative assessment, and robustness to noisy images with different noise levels.

Infrared and visible image fusion algorithm based on spatial domain and image features.

Multi-scale image decomposition is crucial for image fusion, extracting prominent feature textures from infrared and visible light images to obtain clear fused images with more textures. This paper proposes a fusion method of infrared and visible light images based on spatial domain and image features to obtain high-resolution and texture-rich images. First, an efficient hierarchical image clustering algorithm based on superpixel fast pixel clustering directly performs multi-scale decomposition of each source image in the spatial domain and obtains high-frequency, medium-frequency, and low-frequency layers to extract the maximum and minimum values of each source image combined images. Then, using the attribute parameters of each layer as fusion weights, high-definition fusion images are through adaptive feature fusion. Besides, the proposed algorithm performs multi-scale decomposition of the image in the spatial frequency domain to solve the information loss problem caused by the conversion process between the spatial frequency and frequency domains in the traditional extraction of image features in the frequency domain. Eight image quality indicators are compared with other fusion algorithms. Experimental results show that this method outperforms other comparative methods in both subjective and objective measures. Furthermore, the algorithm has high definition and rich textures.

Infrared and visible image fusion algorithm based on split-attention residual networks

在红外和可见光图像融合算法中, 图像信息的丢失始终是制约融合图像质量提升的关键问题, 为此, 提出了一种基于拆分注意力残差网络的红外和可见光图像融合算法, 使用带有拆分注意力模块的深层残差网络拓展感受野和提高跨通道信息融合能力, 运用平滑最大值单元函数作为激活函数进一步提升网络性能; 特征提取后运用零相位分量分析和归一化算法得到融合权重后完成图像融合。实验结果表明, 融合后的图像细节丰富, 边缘锐利; 在峰值信噪比、结构相似性指数度量和基于梯度的融合性能等指标上与经典的6种算法相比均有不同程度提升。

Fusion of Infrared and Visible Images Based on Optimized Low-Rank Matrix Factorization with Guided Filtering

In recent years, image fusion has been a research hotspot. However, it is still a big challenge to balance the problems of noiseless image fusion and noisy image fusion. In order to improve the weak performance and low robustness of existing image fusion algorithms in noisy images, an infrared and visible image fusion algorithm based on optimized low-rank matrix factorization with guided filtering is proposed. First, the minimized error reconstruction factorization is introduced into the low-rank matrix, which effectively enhances the optimization performance, and obtains the base image with good filtering performance. Then using the base image as the guide image, the source image is decomposed into the high-frequency layer containing detail information and noise, and the low-frequency layer containing energy information through guided filtering. According to the noise intensity, the sparse reconstruction error is adaptively obtained to fuse the high-frequency layers, and the weighted average strategy is utilized to fuse the low-frequency layers. Finally, the fusion image is obtained by reconstructing the pre-fused high-frequency layer and the pre-fused low-frequency layer. The comparative experiments show that the proposed algorithm not only has good performance for noise-free images, but more importantly, it can effectively deal with the fusion of noisy images.

Infrared and visible image fusion algorithm based on a cross-layer densely connected convolutional network.

To preserve the saliency of targets in infrared images and the textures in visible images, a novel infrared and visible image fusion method, to the best of our knowledge, is proposed. First, we design a densely connected convolutional network that contains an encoder, fusion, and decoder to minimize the omission of source image effective information. Then, a loss function based on the variational model is designed to retain the thermal radiation information of the infrared image and the details of the visible image to the greatest extent. The experimental results show that the proposed method outperforms state-of-the-art methods in terms of six metrics and better preserves the clear target and textures of infrared and visible images.

An infrared and visible image fusion algorithm based on ResNet-152

An infrared and visible image fusion algorithm based on ResNet-152

End to End Infrared and Visible Image Fusion With Texture Details and Contrast Information

Infrared and visible image fusion combine data information from different sensors to achieve a richer description of the same scene. In order to highlight the salient features of the infrared image and the visible image in the fusion image and obtain a fusion image with good performance, an end-to-end infrared and visible image fusion algorithm is proposed in this paper. The contrast attention module and visible image cascade part are introduced in the generator, so that the fusion image can focus on the detail information in the visible image and the contrast information in the infrared image. And in order to retain more structural contour information in the original image, the contour loss is added to the content loss function. In addition, the contrast and detail information in infrared and visible images are balanced by two discriminators. And a goal-guided reward function is introduced into the discriminator, which further facilitates the generator to produce effective fused images. Finally, extensive fusion experiments on public datasets verify the advantages of the proposed algorithm compared with other classical algorithms, and ablation experiments demonstrate the effectiveness of the improved part of the algorithm.

Injected Infrared and Visible Image Fusion via $L_{1}$ Decomposition Model and Guided Filtering

In this paper, an infrared (IR) and visible (VIS) image fusion algorithm is designed for the injection of the IR objects into the VIS background in a perceptual manner. It consists of four parts: image decomposition, layer fusion, image reconstruction, and image refinement. An edge-preserving filter is constructed for image decomposition, in which an <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$L_1$</tex-math></inline-formula> regularization term and a fractional gradient are newly introduced. The resulting filter is capable of not only preserving edges, but also attenuating the influence of the IR background. A two-layer fusion rule is adopted, which consists of a routine weighted-average fusion rule and an injected fusion rule. It ensures that the fused image is with both rich background information of the VIS image and the salient features of the IR image. After image reconstruction, the guided filter is applied again to the IR image to refine the fused image, such that the final version of the fused image is with satisfactory human visual perception under even dim lights. The effectiveness and superiority of our fusion algorithm are illustrated by the results of ablation studies and comparative experiments.

Infrared and Visible Image Fusion Algorithm Based on Dynamic Range Compression Enhancement and NSST

Infrared and Visible Image Fusion Algorithm Based on Dynamic Range Compression Enhancement and NSST

Unsupervised Infrared Image and Visible Image Fusion Algorithm Based on Deep Learning

Unsupervised Infrared Image and Visible Image Fusion Algorithm Based on Deep Learning

Electrical Fault Detection Equipment Based on Infrared Image Fusion

Maintaining the safe and stable operation of power equipment is the first task of power system operation and maintenance. The equipment in normal operation may have various faults, so it needs to be located, detected and repaired. In order to improve the on-line detection efficiency of electrical equipment, diagnose the operation state of equipment in time and locate faults, this project proposes a multi focus infrared and visible image fusion algorithm based on NSST and adaptive PCNN. Taking the power equipment as the research object, the infrared image of the power equipment is compared with the visible image, and the feature points are selected, so as to realize the image registration. After image registration, the registered visible and infrared images are fused and reconstructed, and the effective features of the two kinds of images are integrated into one image. The target image constructed by fusion not only has thermal energy information, but also has object texture, highlights the target, and can detect the fault location without power failure. It has certain practical significance for quickly removing the fault point of power equipment.

Infrared and Visible Image Fusion via Fast Approximate Bilateral Filter and Local Energy Characteristics

Image fusion is to effectively enhance the accuracy, stability, and comprehensiveness of information. Generally, infrared images lack enough background details to provide an accurate description of the target scene, while visible images are difficult to detect radiation under adverse conditions, such as low light. People hoped that the richness of image details can be improved by using effective fusion algorithms. In this paper, we propose an infrared and visible image fusion algorithm, aiming to overcome some common defects in the process of image fusion. Firstly, we use fast approximate bilateral filter to decompose the infrared image and visible image to obtain the small-scale layers, large-scale layer, and base layer. Then, the fused base layer is obtained based on local energy characteristics, which avoid information loss of traditional fusion rules. The fused small-scale layers are acquired by selecting the absolute maximum, and the fused large-scale layer is obtained by summation rule. Finally, the fused small-scale layers, large-scale layer, and base layer are merged to reconstruct the final fused image. Experimental results show that our method retains more detailed appearance information of the fused image and achieves good results in both qualitative and quantitative evaluations.

Infrared and visible image fusion through hybrid curvature filtering image decomposition

To improve the saliency of the target in the integrated image and the ability to retain the information of detail, an infrared and visible image fusion algorithm based on image decomposition of hybrid curvature filtering is proposed. Firstly, we introduce the hybrid curvature filtering to decompose the input source images into three levels, and three image layers are acquired. According to the different attributes of the image layers, this paper adopts different fusion strategies to fuse the image layers. Finally, the fusion image is obtained by way of adding the image layers. Experimental results demonstrate that compared with other algorithms, the fusion image generated from this algorithm is more conducive to human visual perception and computer analysis.

Multi-scale saliency measure and orthogonal space for visible and infrared image fusion

For infrared and visible image fusion technology, it has always been a challenge to effectively select useful information from the source image and integrate them because the imaging principle of infrared and visible images are widely different. To solve this problem, a novel infrared and visible image fusion algorithm are proposed, which includes the following contributions: (i) an infrared visual saliency extracting scheme using global measurement are presented, (ii) a visible visual saliency measure scheme by a local measurement strategy are proposed, and (iii) a fusion rule based on orthogonal space is designed to combine the extracted saliency maps. Specifically, in order to make humans pay attention to infrared targets, coarse-scale decomposition is performed. Then a global measurement strategy is utilized to get saliency maps. In addition, since visible images have rich textures, fine-scale decomposition can make the visual system pay attention to tiny details. Next, the visual saliency is measured by a local measurement strategy. Different from the general fusion rules, the orthogonal space is constructed to integrate the saliency maps, which can remove the correlation of saliency maps to avoid mutual interference. Experiments on public databases demonstrate that the fusion results of the proposed fusion algorithm are better than other comparison algorithms in qualitative and quantitative assessment.

An Efficient Algorithm to Highlight Details in Infrared and Visible Image Fusion

To improve the fusion quality of infrared and visible images and highlight target and scene details, in this paper, a novel infrared and visible image fusion algorithm is proposed. First, a method for combining dynamic range compression and contrast restoration based on a guided filter is adopted to enhance the contrast of visible source images. Second, guided filter-based image multiscale decomposition is used to decompose images into base layers and detail layers. For base layer fusion, a fusion strategy based on the detail and energy measurements of the source image is proposed to determine the pixel value of the fused image base layer such that the energy loss of the fusion can be reduced and the texture detail features are highlighted to obtain more source image details. Finally, recursive separation and weighted histogram equalization methods are applied to optimize the fused image. Experimental results show that the fusion algorithm and fusion strategy proposed in this paper can effectively improve fusion image clarity, while more detailed target and scene information can still be retained.