Commercial Face Recognition System Research Articles

Fusion-Based 2.5D Face Recognition System

Face recognition is the dominant biometrics system used to authenticate an individual’s identity in various applications. Most commercial face recognition systems rely on 2D face images, but the changes in the environment lighting and a person's posture affect the accuracy of the 2D face recognition systems. Hence, the 2.5D face recognition system arises as the solution to eliminate the drawbacks of the 2D face recognition system. The depth feature in the 2.5D data (depth image) provides additional information that can help to improve the accuracy and robustness of 2.5D face recognition systems, particularly in challenging scenarios. This paper proposes a fusion-based approach for the 2.5D face recognition system to enhance the system’s performance, where feature fusion involves the combination of features extracted from the depth image. Gabor-based Region Covariance Matrices (GRCMs) that serve as face identifiers combine the depth and texture images in the structure of a covariance matrix. Several experiments on different fusions have been conducted in the Face Recognition Grand Challenge version 2 (FRGC v2.0) database. This paper shows that the max-min fusion applied to the surface normal (y-direction) and the mean curvature has achieved the best accuracy rate of 93.66% among the other fusion approaches used.

Two-Face: Adversarial Audit of Commercial Face Recognition Systems

Computer vision applications like automated face detection are used for a variety of purposes ranging from unlocking smart devices to tracking potential persons of interest for surveillance. Audits of these applications have revealed that they tend to be biased against minority groups which result in unfair and concerning societal and political outcomes. Despite multiple studies over time, these biases have not been mitigated completely and have in fact increased for certain tasks like age prediction. While such systems are audited over benchmark datasets, it becomes necessary to evaluate their robustness for adversarial inputs. In this work, we perform an extensive adversarial audit on multiple systems and datasets, making a number of concerning observations- there has been a drop in accuracy for some tasks on CelebSET dataset since a previous audit. While there still exists a bias in accuracy against individuals from minority groups for multiple datasets, a more worrying observation is that these biases tend to get exorbitantly pronounced with adversarial inputs toward the minority group. We conclude with a discussion on the broader societal impacts in light of these observations and a few suggestions on how to collectively deal with this issue.

PRNU‐based detection of facial retouching

Nowadays, many facial images are acquired using smart phones. To ensure the best outcome, users frequently retouch these images before sharing them, e.g. via social media. Modifications resulting from used retouching algorithms might be a challenge for face recognition technologies. Towards deploying robust face recognition as well as enforcing anti-photoshop legislations, a reliable detection of retouched face images is needed. In this work, the effects of facial retouching on face recognition are investigated. A qualitative assessment of 32 beautification apps is conducted. Based on this assessment five apps are chosen which are used to create a database of 800 beautified face images. Biometric performance is measured before and after retouching using a commercial face recognition system. Subsequently, a retouching detection system based on the analysis of photo response non-uniformity (PRNU) is presented. Specifically, scores obtained from analysing spatial and spectral features extracted from PRNU patterns across image cells are fused. In a scenario, in which unaltered bona fide images are compressed to the average sizes of the retouched images using JPEG, the proposed PRNU-based detection scheme is shown to robustly distinguish between bona fide and retouched images achieving an average detection equal error rate of 13.7% across all retouching algorithms.

Unconstrained Kinect video face database

Unconstrained face recognition poses several challenges to existing algorithms and to address these variations, different methodologies are proposed, including utilizing video and 3D (depth) information. With the introduction of consumer level depth capturing devices such as Microsoft Kinect, research has been performed in utilizing low cost RGB-D depth data for characterizing and matching faces. Next generation of Kinect device, the Kinect version 2, has also been released which provides higher resolution color, depth and near infrared images at a comparable sensor cost. Such multiple information obtained from a single sensor can be extremely helpful in designing novel algorithms and systems for object and face recognition. This paper introduces the KaspAROV RGB-D video face database which contains face videos and images from both versions of the Kinect device for over 100 subjects which will be made available to the research community. The database captured in visible and NIR spectrum (along with depth) encompasses face images and videos with challenges such as pose, distance, and illumination. The database can be used to evaluate the performance of face recognition algorithms which utilize either uni-spectrum or multi-spectrum information. We provide standard experimental protocols for ease of comparative evaluation on the database and also include baseline results using several existing algorithms including a commercial face recognition system and deep learning algorithm.

Impact of eye detection error on face recognition performance

The locations of the eyes are the most commonly used features to perform face normalisation (i.e. alignment of facial features), which is an essential preprocessing stage of many face recognition systems. In this study, the authors study the sensitivity of open source implementations of five face recognition algorithms to misalignment caused by eye localisation errors. They investigate the ambiguity in the location of the eyes by comparing the difference between two independent manual eye annotations. They also study the error characteristics of automatic eye detectors present in two commercial face recognition systems. Furthermore, they explore the impact of using different eye detectors for training/enrolment and query phases of a face recognition system. These experiments provide an insight into the influence of eye localisation errors on the performance of face recognition systems and recommend a strategy for the design of training and test sets of a face recognition algorithm.

Facial aging and asymmetry decomposition based approaches to identification of twins

A reliable and accurate biometric identification system must be able to distinguish individuals even in situations where their biometric signatures are very similar. However, the strong similarity in the facial appearance of twins has complicated facial feature based recognition and has even compromised commercial face recognition systems. This paper addresses the above problem and proposes two novel methods to distinguish identical twins using (1) facial aging features and (2) asymmetry decomposition features. Facial aging features are extracted using Gabor filters from regions of the face that typically exhibit wrinkles and laugh lines, while Facial asymmetry decomposition based features are obtained by projecting the difference between the two left sides (consisting of the left half of the face and its mirror) and two right sides (consisting of the right half of the face and its mirror) of a face onto a subspace. Feature vectors obtained using these methods were used for classification. Experiments conducted on images of five types of twins from the University of Notre Dame ND-Twins database indicate that both our proposed approaches achieve high identification rates and are hence quite promising at distinguishing twins.

IJARCCE - Computer and Communication Engineering

2 Abstract: Biometric identification system must be able to distinguish between individuals even in situation where the biometric signature may be similar, such as in the identical twins. Main focus is on twin based image retrieval which retrieves all image from database by matching some of the basic features of identical twins like nose, lips, eyes, forehead and face features. With the advancement of computer technology Image Retrieval based on Identical twin face recognition is a challenging task because of the high degree of similarity in their overall facial appearance. Commercial face recognition systems exhibit poor performance in differentiating between identical twins under practical conditions as they cannot be discriminated based on DNA.

On Recognizing Faces in Videos Using Clustering-Based Re-Ranking and Fusion

Due to widespread applications, availability of large intra-personal variations in video and limited information content in still images, video-based face recognition has gained significant attention. Unlike still face images, videos provide abundant information that can be leveraged to address variations in pose, illumination, and expression as well as enhance the face recognition performance. This paper presents a video-based face recognition algorithm that computes a discriminative video signature as an ordered list of still face images from a large dictionary. A three-stage approach is proposed for optimizing ranked lists across multiple video frames and fusing them into a single composite ordered list to compute the video signature. This signature embeds diverse intra-personal variations and facilitates in matching two videos with large variations. For matching two videos, a discounted cumulative gain measure is utilized, which uses the ranking of images in the video signature as well as the usefulness of images in characterizing the individual in the video. The efficacy of the proposed algorithm is evaluated under different video-based face recognition scenarios such as matching still face images with videos and matching videos with videos. The efficacy of the proposed algorithm is demonstrated on the YouTube faces database and the MBGC v2 video challenge database that comprise different types of video-based face recognition challenges such as matching still face images with videos and matching videos with videos. Performance comparison with the benchmark results on both the databases and a commercial face recognition system shows the efficiency of the proposed algorithm for video-based face recognition.

Digital Face Recognition Techniques for Matching with Sketches

Face recognition is a well studied problem in many application domains. However, matching sketches with digital face images is a very important law enforcement application that has received relatively less attention. The state-of art face recognition algorithms cannot be used directly and require additional processing to address the non-linear variations present in sketches and digital face images. One of the important clues in solving crimes and apprehending criminals is matching sketches with digital face images. This paper presents an automated algorithm that extracts discriminating information from local regions of both sketches and digital face images. Structural information along with minute details present in local facial regions are encoded using multi scale circular Weber’s local descriptor. Further, an evolutionary Memetic Optimization is proposed to assign optimal weights to every local facial region to boost the identification performance. Since, forensic sketches or digital face images can be of poor quality, a pre-processing technique is used to enhance the quality of images and improve the identification performance. Comprehensive experimental evaluation on different sketch databases show that the proposed algorithm yields better identification performance compared to existing face recognition algorithms and two commercial face recognition systems.

Recognizing Surgically Altered Face Images Using Multiobjective Evolutionary Algorithm

Widespread acceptability and use of biometrics for person authentication has instigated several techniques for evading identification. One such technique is altering facial appearance using surgical procedures that has raised a challenge for face recognition algorithms. Increasing popularity of plastic surgery and its effect on automatic face recognition has attracted attention from the research community. However, the nonlinear variations introduced by plastic surgery remain difficult to be modeled by existing face recognition systems. In this research, a multiobjective evolutionary granular algorithm is proposed to match face images before and after plastic surgery. The algorithm first generates non-disjoint face granules at multiple levels of granularity. The granular information is assimilated using a multiobjective genetic approach that simultaneously optimizes the selection of feature extractor for each face granule along with the weights of individual granules. On the plastic surgery face database, the proposed algorithm yields high identification accuracy as compared to existing algorithms and a commercial face recognition system.

Matching Composite Sketches to Face Photos: A Component-Based Approach

The problem of automatically matching composite sketches to facial photographs is addressed in this paper. Previous research on sketch recognition focused on matching sketches drawn by professional artists who either looked directly at the subjects (viewed sketches) or used a verbal description of the subject's appearance as provided by an eyewitness (forensic sketches). Unlike sketches hand drawn by artists, composite sketches are synthesized using one of the several facial composite software systems available to law enforcement agencies. We propose a component-based representation (CBR) approach to measure the similarity between a composite sketch and mugshot photograph. Specifically, we first automatically detect facial landmarks in composite sketches and face photos using an active shape model (ASM). Features are then extracted for each facial component using multiscale local binary patterns (MLBPs), and per component similarity is calculated. Finally, the similarity scores obtained from individual facial components are fused together, yielding a similarity score between a composite sketch and a face photo. Matching performance is further improved by filtering the large gallery of mugshot images using gender information. Experimental results on matching 123 composite sketches against two galleries with 10,123 and 1,316 mugshots show that the proposed method achieves promising performance (rank-100 accuracies of 77.2% and 89.4%, respectively) compared to a leading commercial face recognition system (rank-100 accuracies of 22.8% and 52.0%) and densely sampled MLBP on holistic faces (rank-100 accuracies of 27.6% and 10.6%). We believe our prototype system will be of great value to law enforcement agencies in apprehending suspects in a timely fashion.

Memetically Optimized MCWLD for Matching Sketches With Digital Face Images

One of the important cues in solving crimes and apprehending criminals is matching sketches with digital face images. This paper presents an automated algorithm to extract discriminating information from local regions of both sketches and digital face images. Structural information along with minute details present in local facial regions are encoded using multiscale circular Weber's local descriptor. Further, an evolutionary memetic optimization algorithm is proposed to assign optimal weight to every local facial region to boost the identification performance. Since forensic sketches or digital face images can be of poor quality, a preprocessing technique is used to enhance the quality of images and improve the identification performance. Comprehensive experimental evaluation on different sketch databases show that the proposed algorithm yields better identification performance compared to existing face recognition algorithms and two commercial face recognition systems.

Analysis of Facial Marks to Distinguish Between Identical Twins

Identical twin face recognition is a challenging task due to the existence of a high degree of correlation in overall facial appearance. Commercial face recognition systems exhibit poor performance in differentiating between identical twins under practical conditions. In this paper, we study the usability of facial marks as biometric signatures to distinguish between identical twins. We propose a multiscale automatic facial mark detector based on a gradient-based operator known as the fast radial symmetry transform. The transform detects bright or dark regions with high radial symmetry at different scales. Next, the detections are tracked across scales to determine the prominence of facial marks. Extensive experiments are performed both on manually annotated and on automatically detected facial marks to evaluate the usefulness of facial marks as biometric signatures. Experiment results are based on identical twin images acquired at the 2009 Twins Days Festival in Twinsburg, Ohio. The results of our analysis signify the usefulness of the distribution of facial marks as a biometric signature. In addition, our results indicate the existence of some degree of correlation between geometric distribution of facial marks across identical twins.

Matching Forensic Sketches to Mug Shot Photos

The problem of matching a forensic sketch to a gallery of mug shot images is addressed in this paper. Previous research in sketch matching only offered solutions to matching highly accurate sketches that were drawn while looking at the subject (viewed sketches). Forensic sketches differ from viewed sketches in that they are drawn by a police sketch artist using the description of the subject provided by an eyewitness. To identify forensic sketches, we present a framework called local feature-based discriminant analysis (LFDA). In LFDA, we individually represent both sketches and photos using SIFT feature descriptors and multiscale local binary patterns (MLBP). Multiple discriminant projections are then used on partitioned vectors of the feature-based representation for minimum distance matching. We apply this method to match a data set of 159 forensic sketches against a mug shot gallery containing 10,159 images. Compared to a leading commercial face recognition system, LFDA offers substantial improvements in matching forensic sketches to the corresponding face images. We were able to further improve the matching performance using race and gender information to reduce the target gallery size. Additional experiments demonstrate that the proposed framework leads to state-of-the-art accuracys when matching viewed sketches.

Face Recognition: Where We Are and Where To Go From Here

Face recognition has become a very active research field. Despite the existence of commercial face recognition systems, there are still important challenges for further research. The problems of variable lighting, pose, facial expression, aging, and inaccurate alignment continue to cause larger than desired error rates. This paper discusses the current state of the art in face recognition and suggests some promising directions for future research.

From Scores to Face Templates: A Model-Based Approach

Regeneration of templates from match scores has security and privacy implications related to any biometric authentication system. We propose a novel paradigm to reconstruct face templates from match scores using a linear approach. It proceeds by first modeling the behavior of the given face recognition algorithm by an affine transformation. The goal of the modeling is to approximate the distances computed by a face recognition algorithm between two faces by distances between points, representing these faces, in an affine space. Given this space, templates from an independent image set (break-in) are matched only once with the enrolled template of the targeted subject and match scores are recorded. These scores are then used to embed the targeted subject in the approximating affine (non-orthogonal) space. Given the coordinates of the targeted subject in the affine space, the original template of the targeted subject is reconstructed using the inverse of the affine transformation. We demonstrate our ideas using three, fundamentally different, face recognition algorithms: Principal Component Analysis (PCA) with Mahalanobis cosine distance measure, Bayesian intra-extrapersonal classifier (BIC), and a feature-based commercial algorithm. To demonstrate the independence of the break-in set with the gallery set, we select face templates from two different databases: Face Recognition Grand Challenge (FRGC) and Facial Recognition Technology (FERET) Database (FERET). With an operational point set at 1 percent False Acceptance Rate (FAR) and 99 percent True Acceptance Rate (TAR) for 1,196 enrollments (FERET gallery), we show that at most 600 attempts (score computations) are required to achieve a 73 percent chance of breaking in as a randomly chosen target subject for the commercial face recognition system. With similar operational set up, we achieve a 72 percent and 100 percent chance of breaking in for the Bayesian and PCA based face recognition systems, respectively. With three different levels of score quantization, we achieve 69 percent, 68 percent and 49 percent probability of break-in, indicating the robustness of our proposed scheme to score quantization. We also show that the proposed reconstruction scheme has 47 percent more probability of breaking in as a randomly chosen target subject for the commercial system as compared to a hill climbing approach with the same number of attempts. Given that the proposed template reconstruction method uses distinct face templates to reconstruct faces, this work exposes a more severe form of vulnerability than a hill climbing kind of attack where incrementally different versions of the same face are used. Also, the ability of the proposed approach to reconstruct actual face templates of the users increases privacy concerns in biometric systems.