MLF: A mimic layered fusion method for infrared and visible video

Abstract

Optimal selection of fusion strategy based on image difference information is an important way to improve the adaptive performance of infrared photoelectric detection systems. Aiming at the problem that the current infrared and visible video fusion model cannot be adjusted dynamically according to the difference features of video intra-frame, resulting in poor fusion effect or even fusion failure. In this paper, a mimic layered fusion method for infrared and visible video is proposed. Firstly, by comparing the general video fusion process and polymorphic process of the mimic octopus, we establish the corresponding relationship of them, and determine the four-layer variants of video mimic layered fusion. Secondly, we divide the region of interest in each frame of the video sequence, frames with significant changes in the region of interest are selected as salient frames. Thirdly, the magnitudes of various difference features of the region of interest of the salient frames are calculated respectively. Then, and a fusion effectiveness function is constructed based on cosine similarity and weighting idea to analyze and compare the fusion effects of fusion algorithms, fusion rules, fusion parameters and fusion structures on various difference features layer by layer, so as to select the optimal mimetic variant layer by layer, the mimic layered fusion is realized based on layered fusion discrimination mechanism through the optimal combination of variants. Finally, the experimental results show that our method in this paper has achieved remarkable results in preserving the typical infrared target and visible structural details in the whole video, and is significantly better than other single fusion methods in quantitative analysis and qualitative evaluation.