Learning to Recognize Masked Faces by Data Synthesis

Abstract

Face coverings have become the new normal for people living through the global COVID-19 pandemic crisis. While wearing a mask is a necessary public health measure, the social phenomenon raises new challenges to existing face recognition models. In this work, we evaluate deep neural network approaches for the masked face recognition task. We find that current deep networks can not generalize successfully to recognizing faces with masks. To address this issue, we investigate the use of images of faces with simulated masks to train a deep neural network model for face recognition. We train our model using a collection of two face recognition datasets: the Labeled Faces in the Wild (LFW) dataset, the Real-world Masked Face Recognition (RMFR) dataset and the Simulated Masked Face Recognition (SMFR) dataset. We find that the data sampling strategy during training plays a significant role when the number of simulated examples is much greater than that of available real instances. We show that the model trained using a combination of real and simulated data accurately classifies masked faces with an accuracy of 99%.