Face Verification Accuracy Research Articles

Masked face recognition using domain adaptation

Wearing facial masks has become a must in our daily life due to the global COVID-19 pandemic. However, the performance of a face recognition system is severely degraded due to the fact that the face images in the gallery are unmasked faces while the probe face images captured by the camera are masked faces, making the probe face images different from gallery face images in the activated region and the distribution domain. In this paper, we propose a novel face recognition system to address the issue. The system is integrated with a domain adaptation layer and a feature refinement layer. The feature refinement layer is based on the structure of the self-attention mechanism to align activated regions of unmasked faces with those of masked faces. The domain adaptation layer works by adapting the system from the unmasked face domain to the synthetically masked face domain and the real- world masked face domain. The system is tested on real-world data through face verification and face identification. The face verification accuracy is improved by 6.83% for the RMFD_FV dataset and 4.2% for the MFR2 dataset, and the face identification accuracy is improved by 15.43% for the MFRFI dataset.

GFAM: A Gender-Preserving Face Aging Model for Age Imbalance Data

The objective of face aging is to generate facial images that present the effects of aging. The existing one-hot encoding method for aging and/or rejuvenation patterns overlooks the personalized patterns for different genders and races, causing errors such as a male beard appearing on an aged female face. A gender-preserving face aging model is proposed to address these issues, termed GFAM. GFAM employs a generative adversarial network and includes several subnetworks that simulate the aging process between two adjacent age groups to learn specific aging effects. Specifically, the proposed model introduces a gender classifier and gender loss function that uses gender information as a self-guiding mechanism for maintaining gender attributes. To maintain the identity information of synthetic faces, the proposed model also introduces an identity-preserving module. Additionally, age balance loss is used to mitigate the impact of imbalanced age distribution and enhance the accuracy of aging predictions. Moreover, we construct a dataset with balanced age distribution for the task of face age progression, referred to as Age_FR. This dataset is expected to facilitate current research efforts. Ablation studies have been conducted to extensively evaluate the performance improvements achieved by our method. We obtained relative improvements of 3.75% higher than the model without the gender preserving module. The experimental results provide evidence of the effectiveness of the proposed method, both through qualitative and quantitative analyses. Notably, the mean face verification accuracy for the age-progressed groups (0–20, 31–40, 41–50, and 51–60) was found to be 100%, 99.83%, 99.79%, and 99.11%, respectively, highlighting the robustness of our approach across various age ranges.

LTT-GAN: Looking Through Turbulence by Inverting GANs

In many applications of long-range imaging, we are faced with a scenario where a person appearing in the captured imagery is often degraded by atmospheric turbulence. However, restoring such degraded images for face verification is difficult since the degradation causes images to be geometrically distorted and blurry. To mitigate the turbulence effect, in this paper, we propose the first turbulence mitigation method that makes use of visual priors encapsulated by a well-trained GAN. Based on the visual priors, we propose to learn to preserve the identity of restored images on a Spatial Periodic Contextual Distance (SPCX). Such a distance takes both the realism of restored images from the GAN and identity difference as the objective of the network learning. In addition, Hierarchical Pseudo Connections (HPC) are proposed for facilitating the identity-preserving learning by introducing more appearance variance without identity changing. Extensive experiments show that our method significantly outperforms prior art in both the visual quality and face verification accuracy of restored results.

Analysis of Real-Time Face-Verification Methods for Surveillance Applications.

In the last decade, face-recognition and -verification methods based on deep learning have increasingly used deeper and more complex architectures to obtain state-of-the-art (SOTA) accuracy. Hence, these architectures are limited to powerful devices that can handle heavy computational resources. Conversely, lightweight and efficient methods have recently been proposed to achieve real-time performance on limited devices and embedded systems. However, real-time face-verification methods struggle with problems usually solved by their heavy counterparts-for example, illumination changes, occlusions, face rotation, and distance to the subject. These challenges are strongly related to surveillance applications that deal with low-resolution face images under unconstrained conditions. Therefore, this paper compares three SOTA real-time face-verification methods for coping with specific problems in surveillance applications. To this end, we created an evaluation subset from two available datasets consisting of 3000 face images presenting face rotation and low-resolution problems. We defined five groups of face rotation with five levels of resolutions that can appear in common surveillance scenarios. With our evaluation subset, we methodically evaluated the face-verification accuracy of MobileFaceNet, EfficientNet-B0, and GhostNet. Furthermore, we also evaluated them with conventional datasets, such as Cross-Pose LFW and QMUL-SurvFace. When examining the experimental results of the three mentioned datasets, we found that EfficientNet-B0 could deal with both surveillance problems, but MobileFaceNet was better at handling extreme face rotation over 80 degrees.

Perturbation-insensitive cross-domain image enhancement for low-quality face verification

Low-quality face images present two typical problems: perceptual distortion and a significant reduction in verification accuracy. Prevailing works address these two problems by face image enhancement. Generally, they require low- and high-quality image pairs or facial priors, e.g., facial landmarks, which are relatively rare in real-world scenarios. In this paper, we introduce a fresh perspective to address these problems, by hypothesizing that there is a positive correlation between cross-domain image enhancement and adversarial defense. With the incorporation of cross-domain image enhancement, paired images or facial priors are no longer necessary. Furthermore, we introduce two types of adversarial perturbations, namely appearance and semantic perturbations, and defending these perturbations aims to solve the aforementioned two typical problems. Defending appearance perturbations decreases perceptual distortion and improve image quality, while defending semantic perturbations promotes identity preservation during the image enhancement process, which improves verification accuracy. To this end, we propose a collaborative face enhancement module (COFEM) for face verification based on two types of adversarial perturbation examples. COFEM incorporates three components. First, an adversarial example generator attacks high-quality images (source domain) in two different ways to obtain appearance and semantic perturbation examples. Next, an image enhancement network denoises these perturbation examples and enhances the quality of low-quality images (target domain). Then, an image reconstruction network is utilized to preserve the identity of the enhanced image such that it is consistent with that of the corresponding input. Unlike prevailing image enhancement models which mainly focus on high perceptual quality, COFEM emphasizes identity-related feature preservation, which is vital to face verification. Combined with COFEM, we also design a face verification module to form a low-quality face verification approach. Extensive experiments demonstrate the effectiveness of our approach in improving the low-quality face image quality and verification accuracy.

Dynamic face authentication systems: Deep learning verification for camera close-Up and head rotation paradigms

The popularity of face-authentication systems has also generated interest in the study of malicious authentication attempts, such as face spoofing attacks. In this study we investigate two dynamic face-authentication challenges: the camera close-up, and head-rotation paradigms. For each paradigm we developed an ML-based face-authentication system that performs the tasks of liveness detection and face verification. In order to generate structured data-representations from the videos collected in the wild, we designed feature representations that extract three-dimensional and spatial characteristics of a face, while also capturing the particular liveness cues of the requested challenge-based movements. Furthermore, a set of Neural-Network models that employ Convolutional Neural Networks and Siamese Neural Network architectures were proposed. To train and test our models we collected a dataset of 177 live videos recorded by 41 different subjects and a set of 243 attack attempts in uncontrolled scenarios. The resulting NN models yield good performance against multiple types of media-based attacks (printed-attacks, screen-attacks, 2D-masks, videos acquired from public social media, deep fakes). The camera close-up system presented an overall liveness detection accuracy of 97.7% and a face verification accuracy of 97.6%. On the other hand, the evaluation of the head-rotation system resulted in a liveness detection accuracy of 92.4% and face verification accuracy of 98.1%. Face authentication methods based on dynamic user-challenges constitute a scalable approach that does not require specialized hardware to increase the security of face authentication systems in realistic usage scenarios. The proposed methods not only make it harder for attackers to generate spoofing attacks, but they also constitute a practical complement to static biometric authentication systems.

DDAT: Dual domain adaptive translation for low-resolution face verification in the wild

Low-resolution (LR) face verification has received much attention because of its wide applicability in real scenarios, especially in long-distance surveillance. However, the poor quality and scarcity of training data make the accuracy far from satisfactory. In this paper, we propose an end-to-end LR face translation and verification framework to improve the generation quality of face images and face verification accuracy simultaneously. We design a dual domain adaptive structure to generate high-quality images. On one hand, the structure can reduce the domain gap between training data and test data. On the other hand, the structure preserves identity consistency and low-level attributes. Meanwhile, in order to make the whole model more robust, we treat the generated images of the target domain as an extension of the training data. We conduct extensive comparative experiments on multiple benchmark data sets. Experimental results verify that our method achieves improved results in high-quality face generation and LR face verification. In particular, our model DDAT reduces FID to 18.63 and 39.55 on the source and the target domain from 254.7 and 206.19 of the up-sampling results, respectively. Our method outperforms competing approaches by more than 10 percentage points in terms of face verification accuracy on multiple surveillance benchmarks.

Deep Face Rectification for 360° Dual-Fisheye Cameras.

Rectilinear face recognition models suffer from severe performance degradation when applied to fisheye images captured by 360° back-to-back dual fisheye cameras. We propose a novel face rectification method to combat the effect of fisheye image distortion on face recognition. The method consists of a classification network and a restoration network specifically designed to handle the non-linear property of fisheye projection. The classification network classifies an input fisheye image according to its distortion level. The restoration network takes a distorted image as input and restores the rectilinear geometric structure of the face. The performance of the proposed method is tested on an end-to-end face recognition system constructed by integrating the proposed rectification method with a conventional rectilinear face recognition system. The face verification accuracy of the integrated system is 99.18% when tested on images in the synthetic Labeled Faces in the Wild (LFW) dataset and 95.70% for images in a real image dataset, resulting in an average accuracy improvement of 6.57% over the conventional face recognition system. For face identification, the average improvement over the conventional face recognition system is 4.51%.

A deep learning framework for face verification without alignment

Most of the CNN (convolutional neural networks) methods require alignment, which will affect the efficiency of verification. This paper proposes a deep face verification framework without alignment. First and foremost, the framework consists of two training stages and one testing stage. In the first training stage, the CNN is fully trained on the large face dataset. In the second training stage, embedding triplet is adopted to fine-tune the models. Furthermore, in the testing stage, SIFT (scale invariant feature transform) descriptors are extracted from intermediate pooling results for cascading verification, which effectively improves the accuracy of face verification without alignment. Last but not least, two CNN architectures are designed for different scenarios. The CNN1 (convolutional neural networks 1), with fewer layers and parameters, requires a small amount of memory and computation in training and testing, so it is suitable for real-time system. The CNN2 (convolutional neural networks 2), with more layers and parameters, has excellent face verification. Through the long-term training on WEB-face dataset and experiments on the LFW (labled faces in the wild), YTB (YouTube) datasets, the results show that the proposed method has superior performance compared with some state-of-the-art methods.

Learning face similarities for face verification using hybrid convolutional neural networks

<span>Face verification focuses on the task of determining whether two face images belong to the same identity or not. For unrestricted faces in the wild, this is a very challenging task. Besides significant degradation due to images that have large variations in pose, illumination, expression, aging, and occlusions, it also suffers from large-scale ever-expanding data needed to perform one-to-many recognition task. In this paper, we propose a face verification method by learning face similarities using a Convolutional Neural Networks (ConvNet). Instead of extracting features from each face image separately, our ConvNet model jointly extracts relational visual features from two face images in comparison. We train four hybrid ConvNet models to learn how to distinguish similarities between the face pair of four different face portions and join them at top-layer classifier level. We use binary-class classifier at top-layer level to identify the similarity of face pairs which includes a conventional Multi-Layer Perceptron (MLP), Support Vector Machines (SVM), Native Bayes, and another ConvNet. There are 3 face pairing configurations discussed in this paper. Results from experiments using Labeled face in the Wild (LFW) and CelebA datasets indicate that our hybrid ConvNet increases the face verification accuracy by as much as 27% when compared to individual ConvNet approach. We also found that Lateral face pair configuration yields the best LFW test accuracy on a very strict test protocol without any face alignment using MLP as top-layer classifier at 87.89%, which on-par with the state-of-the-arts. We showed that our approach is more flexible in terms of inferencing the learned models on out-of-sample data by testing LFW and CelebA on either model.</span>

Disentangled representation learning and residual GAN for age-invariant face verification

One of the challenges in cross-age face recognition and verification is to effectively model the facial aging process. Despite the rapid advances in face-related areas, it is still very difficult for existing methods to simultaneously achieve accurate facial feature preservation and reliable aging during aging modeling. Aiming to address this issue, we introduce a Disentangled Representation learning and Residual Generative Adversarial Network (DR-RGAN) that represents the facial features without age interference, which is achieved by explicitly disentangling the facial features and age variation. An encoder-decoder structured generator produces aging images from unstructured facial representations by using the age characteristics provided separately. It can thus take the disentangled representation to preserve personal identity for face verification. Considering that pixel-based errors may cause a loss of detail, a VGG based content loss is further equipped to preferably preserve facial features. The discriminator is trained to distinguish the real from generated faces, carry out identification prediction, and leverage an age estimator to boost the aging accuracy. It is beneficial for obtaining more photorealistic and desirable aging effects, as well as more consistent face verification results. Extensive experiments on the CACD and LFW datasets demonstrate that our DR-RGAN generates pleasing aging imageries and achieves a high accuracy of face verification.

Learning based Facial Image Compression with semantic fidelity metric

Surveillance and security scenarios usually require high efficient facial image compression scheme for face recognition and identification. While either traditional general image codecs or special facial image compression schemes only heuristically refine codec separately according to face verification accuracy metric. We propose a Learning based Facial Image Compression (LFIC) framework with a novel Regionally Adaptive Pooling (RAP) module whose parameters can be automatically optimized according to gradient feedback from an integrated hybrid semantic fidelity metric, including a successfully exploration to apply Generative Adversarial Network (GAN) as metric directly in image compression scheme. The experimental results verify the framework’s efficiency by demonstrating performance improvement of 71.41%, 48.28% and 52.67% Bit Rate Saving separately over JPEG2000, WebP and neural network-based codecs under the same face verification accuracy distortion metric. We also evaluate LFIC’s superior performance gain compared with latest specific facial image codecs. Visual experiments also show some interesting insight on how LFIC can automatically capture the information in critical areas based on semantic distortion metrics for optimized compression, which is quite different from the heuristic way of optimization in traditional image compression algorithms.

A Normalized Light CNN for Face Recognition

In recent years, achievements of deep learning in face recognition have exceeded those of traditional machine learning methods do. To achieve better recognition performance, the architectures of the neural network tend to be deeper and more complex, which wastes the time and space. Aiming at this issue, a series of light convolutional neural network (Light CNN) has been proposed, which consume less hardware resources while has excellent performance. In this paper, we propose a Normalized Light CNN model modified from a Light CNN model. We introduced a normalized layer to this model in both training and test phase. The normalized layer normalizes the output features, making it represent images better. We evaluate our model on LFW dataset. The accuracy of face verification reaches 98.46%, which is better than the original model.

Dual Minkowski Loss for Face Verification of Convolutional Network

Despite face recognition and verification have achieved great success in recent years, these achievements are experimental results on fixed data sets. Implementing these outstanding technologies in the field of undeveloped data sets presents serious challenges. We adopt three state-of-the-art pre-trained models on an entire new dataset University Test System Database (UTSD), however the results are far from satisfactory. Therefore, two methods are adopted to solve this problem. The first way is data augmentation including horizontal flipping, cropping and RGB channels transform, which can solve the imbalance of label pairs. The second way is the combination of Manhattan Distance and Euclidean Distance, we call it Dual Minkowski Loss (DML). Through the implementation of photo augmentation and innovative method on UTSD, the accuracy of face verification has been significantly improved, achieving the best 99.3%.

Deep Correlation Feature Learning for Face Verification in the Wild

Convolutional neural networks (CNNs) commonly uses the softmax loss function as the supervision signal. In order to enhance the discriminative power of the deeply learned features, this letter proposes a new supervision signal, called correlation loss, for face verification task. Specifically, the correlation loss encourages the large correlation between the deep feature vectors and their corresponding weight vectors in softmax loss. With the joint supervision of softmax loss and correlation loss, the deep correlation feature learning (DCFL) network can learn the deep features with both the interclass separability and the intraclass compactness, which are highly discriminative for face verification. More importantly, by applying the weight vector of softmax function as the class prototype, the proposed correlation loss function is easy to be optimized during the backpropatation of CNN. Finally, the DCFL method achieves 99.55% and 96.06% face verification accuracy using a 64-layer ResNet on the labeled face in-the-Wild (LFW) and you-tube face (YTF) benchmark, respectively.

Deeply vulnerable: a study of the robustness of face recognition to presentation attacks

The vulnerability of deep-learning-based face-recognition (FR) methods, to presentation attacks (PA), is studied in this study. Recently, proposed FR methods based on deep neural networks (DNN) have been shown to outperform most other methods by a significant margin. In a trustworthy face-verification system, however, maximising recognition-performance alone is not sufficient – the system should also be capable of resisting various kinds of attacks, including PA. Previous experience has shown that the PA vulnerability of FR systems tends to increase with face-verification accuracy. Using several publicly available PA datasets, the authors show that DNN-based FR systems compensate for variability between bona fide and PA samples, and tend to score them similarly, which makes such FR systems extremely vulnerable to PAs. Experiments show the vulnerability of the studied DNN-based FR systems to be consistently higher than 90%, and often higher than 98%.

Probabilistic Elastic Part Model: A Pose-Invariant Representation for Real-World Face Verification

Pose variation remains to be a major challenge for real-world face recognition. We approach this problem through a probabilistic elastic part model. We extract local descriptors (e.g., LBP or SIFT) from densely sampled multi-scale image patches. By augmenting each descriptor with its location, a Gaussian mixture model (GMM) is trained to capture the spatial-appearance distribution of the face parts of all face images in the training corpus, namely the probabilistic elastic part (PEP) model. Each mixture component of the GMM is confined to be a spherical Gaussian to balance the influence of the appearance and the location terms, which naturally defines a part. Given one or multiple face images of the same subject, the PEP-model builds its PEP representation by sequentially concatenating descriptors identified by each Gaussian component in a maximum likelihood sense. We further propose a joint Bayesian adaptation algorithm to adapt the universally trained GMM to better model the pose variations between the target pair of faces/face tracks, which consistently improves face verification accuracy. Our experiments show that we achieve state-of-the-art face verification accuracy with the proposed representations on the Labeled Face in the Wild (LFW) dataset, the YouTube video face database, and the CMU MultiPIE dataset.

Collaborative Random Faces-Guided Encoders for Pose-Invariant Face Representation Learning.

Learning discriminant face representation for pose-invariant face recognition has been identified as a critical issue in visual learning systems. The challenge lies in the drastic changes of facial appearances between the test face and the registered face. To that end, we propose a high-level feature learning framework called "collaborative random faces (RFs)-guided encoders" toward this problem. The contributions of this paper are three fold. First, we propose a novel supervised autoencoder that is able to capture the high-level identity feature despite of pose variations. Second, we enrich the identity features by replacing the target values of conventional autoencoders with random signals (RFs in this paper), which are unique for each subject under different poses. Third, we further improve the performance of the framework by incorporating deep convolutional neural network facial descriptors and linking discriminative identity features from different RFs for the augmented identity features. Finally, we conduct face identification experiments on Multi-PIE database, and face verification experiments on labeled faces in the wild and YouTube Face databases, where face recognition rate and verification accuracy with Receiver Operating Characteristic curves are rendered. In addition, discussions of model parameters and connections with the existing methods are provided. These experiments demonstrate that our learning system works fairly well on handling pose variations.

Face Behind Makeup

In this work, we propose a novel automatic makeup detector and remover framework. For makeup detector, a locality-constrained low-rank dictionary learning algorithm is used to determine and locate the usage of cosmetics. For the challenging task of makeup removal, a locality-constrained coupled dictionary learning (LC-CDL) framework is proposed to synthesize non-makeup face, so that the makeup could be erased according to the style. Moreover, we build a stepwise makeup dataset (SMU) which to the best of our knowledge is the first dataset with procedures of makeup. This novel technology itself carries many practical applications, e.g. products recommendation for consumers; user-specified makeup tutorial; security applications on makeup face verification. Finally, our system is evaluated on three existing (VMU, MIW, YMU) and one own-collected makeup datasets. Experimental results have demonstrated the effectiveness of DL-based method on makeup detection. The proposed LC-CDL shows very promising performance on makeup removal regarding on the structure similarity. In addition, the comparison of face verification accuracy with presence or absence of makeup is presented, which illustrates an application of our automatic makeup remover system in the context of face verification with facial makeup.

Face Verification Subject to Varying (Age, Ethnicity, and Gender) Demographics Using Deep Learning

Human facial appearance is strongly influenced by demographical characteristics such as categorical age, ethnicity, and gender with each category further partitioned into classes-Black, White, Male, Female, Young (18-30), Middle Age (30-50), and Old (50-70)-and groups−mix of classes. Most subjects share a more similar appearance with their own demographic class than with other classes. We evaluate here the accuracy of automatic facial verification for subjects belonging to varying age, ethnicity, and gender categories. Towards that end, we use a convolutional neural network for feature extraction and show that our method yields better performance on individual demographics compared to a commercial face recognition engine. For one-class demographic groups, we corroborate empirical findings that biometric performance on verification is relatively lower for females, young subjects in the 18-30 age group, and blacks. We then expand the scope of our method and evaluate the accuracy of face verification for several multiclass demographic groups. We discuss the results and make suggestions for improving face verification across varying demographics.