A mimic fusion method based on difference feature association falling shadow for infrared and visible video

Abstract

Optimal selection of fusion strategy based on image difference information is an important way to improve the adaptive performance of infrared photoelectric detection systems. Heterogeneous difference feature amplitudes often have different dimensions and orders of magnitude, so the traditional fusion model cannot adjust the fusion algorithm effectively according to the difference feature attribute value, resulting in poor fusion effect or even fusion failure. Furthermore, the existing fusion methods often ignore the association between the different feature information, which leads to problems such as weak semantic interpretation of the fusion process and difficulty in improving the fusion effect. As a result, we draw on the multi-mimetic idea of mimic octopus, and propose a mimic fusion method based on difference feature association falling shadows for infrared and visible video. Firstly, we select six difference features to quantitatively describe the amplitudes of three types of significant complementary information in bimodal video. Secondly, the difference feature frequency distribution is obtained by KNNE. Thirdly, construct the difference feature comprehensive weight to coordinate the relationship between the multiple attributes of difference feature, so as to determine the main difference feature of each frame according to the result. Then use PCCs to calculate the correlation between the any two difference features comprehensive weights to obtain the feature association matrix. Finally, the association synthesis rules of the heterogeneous difference feature fusion validity are established, and the mimic algorithm variables are optimized and selected according to the association falling shadow results to realize the mimic fusion for infrared and visible video. The experimental results show that the method in this paper has achieved remarkable results in preserving the typical infrared target and visible light structural details in the whole video, and is significantly better than other single fusion methods in quantitative analysis and qualitative evaluation.