Thermal image visualization using multi-discriminator CycleGAN with unpaired thermal–visible image training set

Abstract

Thermal imaging utilizes the differences in thermal radiation within a scene to capture images, effectively compensating for the limited imaging capabilities of visible-light cameras in low-light environments. In order to further enhance thermal imaging, we propose a novel approach that transforms the visualization of thermal images into an image conversion task. Our method, named TIVNet (Thermal Images Visualization Network), is based on a multi-discriminator cycle-consistent adversarial network (CycleGAN) and operates in the YUV color space to accurately colorize and represent thermal infrared images. The utilization of color and texture discriminators in our framework ensures that the converted images exhibit appropriate color styles and capture fine details, respectively. In the generator, we employed a Dense Residual Module (DRM) to facilitate the reuse of features and enhance the flow of information. By utilizing the DRM, we aim to enhance the overall performance and efficiency of the generator network. Additionally, we incorporated the Dual-channel Self-Attention Module(DCSA) to enhance the feature representation. By leveraging self-attention, the generator can effectively learn and exploit the inherent relationships between different regions, leading to improved feature representation and, consequently, more accurate colorization and representation of thermal infrared images. Notably, our proposed method leverages unsupervised learning, eliminating the need for a fully paired thermal infrared-visible image dataset. Through extensive experimentation on publicly available datasets as well as our own, we demonstrate that our approach outperforms state-of-the-art methods in thermal image visualization. Furthermore, the converted images produced by TIVNet exhibit a high level of realism, thereby enhancing the visual impact of thermal imaging. This improvement is particularly beneficial for human perception and subsequent target recognition tasks. Our findings suggest that the application of our method yields superior results both quantitatively and qualitatively, showcasing its potential for advancing thermal imaging capabilities.