Thermal UAV Image Super-Resolution Guided by Multiple Visible Cues

Abstract

Unmanned aerial vehicle (UAV) thermal-imaging has received much attention, but the insufficient image resolution caused by thermal imaging systems is still a crucial problem that limits the understanding of thermal UAV images. However, high-resolution visible images are relatively easy to access, and it is thus valuable for exploring useful information from visible image to assist thermal UAV image super-resolution (SR). In this article, we propose a novel multiconditioned guidance network (MGNet) to effectively mine the information of visible images for thermal UAV image SR. High-resolution visible UAV images usually contain salient appearance, semantic, and edge information, which plays a critical role in boosting the performance of thermal UAV image SR. Therefore, we design an effective multicue guidance module (MGM) to leverage the appearance, edge, and semantic cues from visible images to guide thermal UAV image SR. In addition, we build the first benchmark dataset for the task of thermal UAV image SR guided by visible images. It is collected by a multimodal UAV platform and composes of 1025 pairs of manually aligned visible and thermal images. Extensive experiments on the built dataset show that our MGNet can effectively leverage useful information from visible images to improve the performance of thermal UAV image SR and perform well against several state-of-the-art methods. The dataset is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/mmic-lcl/Datasets-and-benchmark-code</uri> .