Masked face reconstruction and recognition through principle components analysis

Abstract

Due to the Covid-19 virus infecting humans since 2019, the traditional methods of face recognition are not suitable anymore, for the reason that lots of countries and regions request people to wear masks in public places, whereas the masks will add additional noise in the recognition process. To solve this problem, we suggest a new method named masked face recognition, whose principle is first converting the masked face image to an unmasked reconstruction image by applying Principle Components Analysis. We call it the Reconstruction Process; then using Convolutional Neural Network to exert face recognition on unmasked reconstruction images, we call it the Recognition Process. During the Reconstruction Process, we first detect the unmasked area, then linearly fit the unmasked area with Eigenfaces generated from the PCA process and a set of coefficients, which are the objectives we optimize. After the optimization, we are able to obtain the reconstruction image by the linear combination of the Eigenface with its corresponding coefficient. Experimental results clearly show the high effectiveness of our method, and the accuracy of being recognized correctly is increased from about 80% if we use the original masked images to 96.36% if we use the unmasked reconstruction images.