An Infrared and Visible Image Fusion Algorithm Based on LSWT-NSST

Li Junwu,Binhua Li,Yaoxi Jiang

doi:10.1109/access.2020.3028088

Abstract

Regarding the problems of image distortion, edge blurring, Gibbs phenomena in the traditional wavelet transform algorithm and the loss of subtle features in the Non-Subsampled Shearlet Transform (NSST), and considering the physical characteristics of infrared and visible images, an infrared and visible image fusion algorithm based on the Lifting Stationary Wavelet Transform (LSWT) and Non-Subsampled Shearlet Transform is proposed in this paper. First, since LSWT can quickly calculate and has all advantages of traditional WT, it is utilized to decompose infrared and visible images to obtain low-frequency coefficients and multi-scale and multi-directional high-frequency coefficients, respectively. Second, NSST multi-scale decomposition is used to extract the target features and detailed features of the image from the high and low-frequency sub-bands to obtain new high and low-frequency sub-bands. Third, according to the physical characteristics that low and high-frequency coefficients represent, different fusion rules are designed. Discrete Cosine Transform (DCT) and Local Spatial Frequency (LSF) are introduced in the low-frequency sub-band, and LSF adaptive weighted fusion rules are used in the DCT domain. The fusion strategy improves the regional contrast in the high-frequency sub-band with the spectral characteristics of human vision. Finally, the Inverse Lifting Stationary Wavelet Transform (ILSWT) is used to reconstruct the fusion coefficients to obtain the final fused images. To verify the advantages of the proposed algorithm in this paper, the classic and advanced 9 IR and VI fusion algorithms are selected for subjective and objective comparison. In the objective evaluation, a comprehensive ranking index is designed based on 9 classical indicators. Simulation experiments with 10 IR and VI fusion algorithms prove that the proposed algorithm has better performance and flexibility. The results show that the proposed algorithm in this paper fuses the images with clear edges, prominent targets, and good visual perception, and it outperforms state-of-the-art image fusion algorithms.

Highlights

Image fusion is a technology that performs registration on images obtained by different sensors on the same target; it utilizes certain algorithms to remove redundant information and integrate complementary information to generate more suitable fusion images for human visual perception [1]
Inspired by the above discussion, and by integrating the advantages of the image multi-scale frequency domain transform and the characteristics of Lifting Stationary Wavelet Transform (LSWT) and Non-Subsampled Shearlet Transform (NSST), this paper proposes an infrared and visible image fusion algorithm based on LSWT-NSST
The third part introduces the infrared and visible image fusion algorithm based on LSWT-NSST in detail

Summary

INTRODUCTION

Image fusion is a technology that performs registration on images obtained by different sensors on the same target; it utilizes certain algorithms to remove redundant information and integrate complementary information to generate more suitable fusion images for human visual perception [1]. Traditional image fusion strategies mostly obtain high and low-frequency fusion decision maps by filtering the source image or performing calculations on the decomposition coefficients, and they have achieved good fusion results This method ignores the physical characteristics of the high and low-frequency sub-bands, which causes the loss of details and reduces the effect of the fused image. Inspired by the above discussion, and by integrating the advantages of the image multi-scale frequency domain transform and the characteristics of LSWT and NSST, this paper proposes an infrared and visible image fusion algorithm based on LSWT-NSST. The sub-sampling operation in the Multi-directional shear filter makes Shearlet Transform not have shift invariance and cause spectral aliasing To avoid this defect and retain the advantages of multiscale decomposition, Easley proposed the NSST algorithm [35]. NLSP and Shearlet filters are used to make NSST algorithm multi-scale, multi-directional and shift invariant, which can effectively characterize the details of the source image and avoid Gibbs and eclipse phenomena

DISCRETE COSINE TRANSFORM

2) OBJECTIVE EVALUATION

3) EXPERIMENTAL RESULTS AND DISCUSSION

CONCLUSION