Abstract
Thermal cameras work well in harsh environments, but the quality of infrared images is not as high as visible light. Thermal to visible image translation can get rid of the image modal differences caused by various spectral characteristics. Nowadays, Generative Adversarial Network (GAN) can transform the images from one domain to another domain, but the generated images are still in a single channel in case of facial thermal to visible. In this paper, we propose a claw connection-based generative adversarial networks framework named ClawGAN for the facial thermal images to RGB visible images translation. We proposed the mismatch metric (MM) to measure the mapping relationship of paired images and use template matching to reduce MM of the dataset. Based on the CycleGAN framework, the synthesized loss and the generative reconstructed loss are added to the adversarial loss and the cycle-consistency loss to form a new objective function. And a claw-connected network is invoked to replace the U-net network structure of the generator for more feature preservation. The model is judged from subjective evaluation and objective evaluation based on image quality metrics such as PSNR (Peak Signal to Noise Ratio), SSIM (Structural Similarity), FID (Fréchet inception distance), and face recognition accuracy. We divided the open datasets into bright light and dark light to research the effect of illumination. The experiments show that the proposed method has the lowest FID and the highest face recognition accuracy compared to the state-of-the-art methods. The proposed ClawGAN retains the structural features of thermal images while not only enhancing the quality of images but also effectively improving the observability of image translation results in both bright and dark light. The code is available at https://github.com/Luoyi3819/ClawGAN.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.