An Optimal Algorithm for Fusion of Passive Millimeter Wave and Visible Images Based on Non-subsampled Shearlet Transform and Improved Spiking Cortical Model

Abstract

The purpose of this paper is to propose an algorithm and a novel method for fusion of Passive Millimeter Wave (PMMW) images with their visible images by using Non-subsampled Shearlet Transform (NSST) based on the Spiking Cortical Model (SCM) fusion rule. The parameters of NSST and Improved SCM (ISCM) are selected and optimized based on time consumption and the output of the optimization function like QAB/F. In addition, an effective thresholding method in contourlet transform for PMMW and visible images fusion, which is proposed in our previous paper, is applied to the fusion procedure. Furthermore, NSST is used for analysis of the initial images in different resolutions and directions, followed by using of the ISCM neural network as the fusion rule. Combination of the proposed fusion and thresholding processes resulted in better output images containing visual information close to that of the visible image as well as the hidden object in the millimeter wave image. Moreover, a new evaluation criterion is proposed which does not suffer the shortcomings of the available criteria and improves the previous results for detecting hidden objects by up to 23% with 5% extra time consumption. Finally, the obtained results are evaluated with the available and standard metrics, and all the images exhibited better values than those of the other methods. In fact, the value obtained for the gun image is also acceptable. The results indicate that the proposed fusion method realizes almost all the considered objectives, and the fused image show the objects distinctly in an image with visible details.