PAS-GAN: A GAN based on the Pyramid Across-Scale module for visible-infrared image transformation

Abstract

With the rapid development of computer vision, there is a recent trend for intelligent image understanding based on deep learning. RGB images and infrared images have complementary information in image capture tasks in complex environments due to their different imaging modalities. Therefore, the combination of the two plays an important role in improving video surveillance and target detection capabilities. However, large publicly available infrared image datasets are lacking and acquiring infrared images can be resource-intensive. The lack of samples can lead to a breakdown in the training of deep models. In this paper, to overcome this challenge, we construct a GAN-based visible-infrared image transformation model. The model uses existing visible data to generate infrared images by training an end-to-end generative network. We innovatively propose a lightweight PAS feature extraction module applied to the generator. It enriches the image detail representation of the feature domain from multiple dimensions and greatly improves the model feature representation capability. And the image gradient calculation is used to limit the direction of model optimization. We evaluated our model on three different publicly available datasets, evaluating the quality of the generated images in terms of both visual effects and objective numerical assessments. The experimental results show that the network exhibits excellent results in both qualitative and quantitative evaluation compared to the current state-of-the-art image generation methods. The lightweight feature extraction module also gives PAS-GAN a significant advantage in inference speed.