Thermal UAV image super-resolution via bimodal image feature fusion

Abstract

ABSTRACT With the wide application of unmanned aerial vehicles (UAVs), UAV thermal imaging technology has been brought to the attention of numerous experts and scholars. At present, the application of UAV thermal imaging technology is limited by the problem of low resolution of the images captured by the thermal imaging system itself. In order to solve this problem, this paper proposes a UAV thermal image super-resolution reconstruction method based on bimodal image feature fusion, which improves the UAV thermal image super-resolution characterization capability by making use of the feature information provided by the UAV visible image to compensate for the lack of texture and edge information perception in thermal images. The proposed methodology consists of two tasks: The first task involves a bimodal image feature fusion network that extracts texture and edge information from visible images. This addresses the lack of detailed feature information in thermal images with a differential complementary modal feature fusion module, ultimately producing a fused image with high-quality feature information. The second task entails the integration of a gated multilevel feature fusion module into the UAV thermal image reconstruction network as an attention mechanism, thereby improving the use of fused image feature information for generating high-resolution UAV thermal images. Finally, a joint adaptive training strategy is utilized to further enhance the super-resolution effect of UAV thermal images. The experimental results show that the proposed method can effectively extract the visible image information to compensate for the characterization information in the UAV thermal image super-resolution, and achieves the optimal values of 31.53 dB and 0.9116 in PSNR and SSIM metrics, respectively, providing a new solution for the thermal image super-resolution of UAVs.