Thermal Face Generation Using StyleGAN

Gabriel Hermosilla,Gonzalo Farias Castro,Hector Allende-Cid,Esteban Vera,Diego-Ignacio Henriquez Tapia

doi:10.1109/access.2021.3085423

Gabriel Hermosilla, Gonzalo Farias Castro + Show 3 more

Open Access

https://doi.org/10.1109/access.2021.3085423

Copy DOI

Abstract

This article proposes the use of generative adversarial networks (GANs) via StyleGAN2 to create high-quality synthetic thermal images and obtain training data to build thermal face recognition models using deep learning. We employed different variants of StyleGAN2, incorporating the new improved version of StyleGAN that uses adaptive discriminator augmentation (ADA). In addition, three different thermal databases from the literature were employed to train a thermal face detector based on YOLOv3 and to train StyleGAN2 and its variants, evaluating different metrics. The synthetic thermal database was built using GANSpace to manipulate the intermediate latent space w of StyleGAN2 and obtain images with different characteristics, such as eyeglasses, rotation, beards, etc. We carried out the training of 6 pretrained deep learning models for face recognition to validate the use of our synthetic thermal database, obtaining 99.98% accuracy for classifying synthetic thermal face images.

Highlights

Artificial intelligence techniques have been extensively utilized in recent years due to a variety of new algorithms in fields such as face detection, face recognition, and image synthesis, which are mainly based on machine learning and deep learning
This article proposes the use of generative adversarial networks using StyleGAN2 to create high-quality synthetic thermal images and obtain training data to build face recognition models in deep learning
To train StyleGAN2 and its variants, it was necessary to search for thermal databases from the literature, to which a face detector based on YOLOv3 was applied, which was trained and achieved a high detection performance (99.37%)

Summary

INTRODUCTION

Artificial intelligence techniques have been extensively utilized in recent years due to a variety of new algorithms in fields such as face detection, face recognition, and image synthesis, which are mainly based on machine learning and deep learning. The above has been used successfully in medical applications through the use of convolutional neural networks, transfer learning and combinations of machine learning and deep learning models [4], [5] [6] This data requirement is a major problem in face recognition when thermal imaging is utilized because few thermal databases are available in the literature, and they have a limited number of images and low resolution. The following advances in the state of the art of GANs are notable: the Deep Convolutional GAN (DCGAN) [21], which improved the resolution and image quality results obtained with original GANs, and the CoupledGAN (CoGAN) [22], which instead of using a single generator and a discriminator, uses two generators and a discriminator In this way, generators learn to create images with different characteristics; for example, faces with different hair color, eye color, skin, etc. The section will show various databases that are used to train StyleGAN2

THERMAL FACE DATABASES

CARL DATABASE

Findings

CONCLUSION