Infrared and Visible Image Fusion Methods for Unmanned Surface Vessels with Marine Applications

Abstract

Infrared and visible image fusion is a very effective way to solve the degradation of sea images for unmanned surface vessels (USVs). Fused images with more clarity and information are useful for the visual system of USVs, especially in harsh marine environments. In this work, three novel fusion strategies based on adaptive weight, cross bilateral filtering, and guided filtering are proposed to fuse the feature maps that are extracted from source images. First, the infrared and visible cameras equipped on the USV are calibrated using a self-designed calibration board. Then, pairs of images containing water scenes are aligned and used as experimental data. Finally, each proposed strategy is inserted into the neural network as a fusion layer to verify the improvements in quality of water surface images. Compared to existing methods, the proposed method based on adaptive weight provides a higher spatial resolution and, in most cases, less spectral distortion. The experimental results show that the visual quality of fused images obtained based on an adaptive weight strategy is superior compared to other strategies, while also providing an acceptable computational load.