Face Image Quality Research Articles

Assessing the Effect of Data Augmentation on Occluded Frontal Faces Using DWT-PCA/SVD Recognition Algorithm

The drift towards face-based recognition systems can be attributed to recent advances in supportive technology and emerging areas of application including voting systems, access control, human-computer interactions, entertainments, and crime control. Despite the obvious advantages of such systems being less intrusive and requiring minimal cooperation of subjects, the performances of their underlying recognition algorithms are challenged by the quality of face images, usually acquired from uncontrolled environments with poor illuminations, varying head poses, ageing, facial expressions, and occlusions. Although several researchers have leveraged on the property of bilateral symmetry to reconstruct half-occluded face images, their approach becomes deficient in the presence of random occlusions. In this paper, we harnessed the benefits of the multiple imputation by the chained equation technique and image denoising using Discrete Wavelet Transforms (DWTs) to reconstruct degraded face images with random missing pixels. Numerical evaluation of the study algorithm gave a perfect (100%) average recognition rate each for recognition of occluded and augmented face images. The study also revealed that the average recognition rate for the augmented face images (75.5811) was significantly lower than the average recognition rate (430.7153) of the occluded face images. MICE augmentation is recommended as a suitable data enhancement mechanism for imputing missing data/pixel of occluded face images.

DDAT: Dual domain adaptive translation for low-resolution face verification in the wild

Low-resolution (LR) face verification has received much attention because of its wide applicability in real scenarios, especially in long-distance surveillance. However, the poor quality and scarcity of training data make the accuracy far from satisfactory. In this paper, we propose an end-to-end LR face translation and verification framework to improve the generation quality of face images and face verification accuracy simultaneously. We design a dual domain adaptive structure to generate high-quality images. On one hand, the structure can reduce the domain gap between training data and test data. On the other hand, the structure preserves identity consistency and low-level attributes. Meanwhile, in order to make the whole model more robust, we treat the generated images of the target domain as an extension of the training data. We conduct extensive comparative experiments on multiple benchmark data sets. Experimental results verify that our method achieves improved results in high-quality face generation and LR face verification. In particular, our model DDAT reduces FID to 18.63 and 39.55 on the source and the target domain from 254.7 and 206.19 of the up-sampling results, respectively. Our method outperforms competing approaches by more than 10 percentage points in terms of face verification accuracy on multiple surveillance benchmarks.

Hierarchic Clustering-Based Face Enhancement for Images Captured in Dark Fields

A hierarchic clustering-based enhancement is proposed to solve the luminance compensation of face recognition in the dark field. First, the face image is divided into five levels by a clustering method. Second, the results above are mapped into three hierarchies according to the histogram thresholds. A low, a middle, and a high-intensity block are found. Third, two kinds of linear transforms are performed to the high and the low-intensity blocks. Finally, a center wrap function-based enhancement is carried out. Experiment results show our method can improve both the face recognition accuracy and image quality.

Facial Expression Ambiguity and Face Image Quality Affect Differently on Expression Interpretation Bias.

We often show an invariant or comparable recognition performance for perceiving prototypical facial expressions, such as happiness and anger, under different viewing settings. However, it is unclear to what extent the categorisation of ambiguous expressions and associated interpretation bias are invariant in degraded viewing conditions. In this exploratory eye-tracking study, we systematically manipulated both facial expression ambiguity (via morphing happy and angry expressions in different proportions) and face image clarity/quality (via manipulating image resolution) to measure participants' expression categorisation performance, perceived expression intensity, and associated face-viewing gaze distribution. Our analysis revealed that increasing facial expression ambiguity and decreasing face image quality induced the opposite direction of expression interpretation bias (negativity vs. positivity bias, or increased anger vs. increased happiness categorisation), the same direction of deterioration impact on rating expression intensity, and qualitatively different influence on face-viewing gaze allocation (decreased gaze at eyes but increased gaze at mouth vs. stronger central fixation bias). These novel findings suggest that in comparison with prototypical facial expressions, our visual system has less perceptual tolerance in processing ambiguous expressions which are subject to viewing condition-dependent interpretation bias.

IE-WGAN: A model with Identity Extracting for Face Frontal Synthesis

Face pose affects the effect of the frontal face synthesis, we propose a model used for frontal face synthesis based on WGAN-GP. This model includes identity extracting module, which is used to supervise the training of the face generation module. On the one hand, the model improves the quality of synthetic face images, on the other hand, it can accelerate the convergence speed of network training. We conduct verification experiments on CelebA data sets, and the results show that this model improves the graphic quality of frontal synthesis.

Image processing effects on the deep face recognition system.

Face recognition technology has become an important quantitative examination method in the field of forensic identification of human images. However, face image quality affects the recognition performance of face recognition systems. Existing research on the effects of face image denoising and enhancement methods on the face recognition performance are typically based on facial images with manually synthesized noises rather than the noises under natural environmental corruption, and their studied face recognition techniques are limited on the traditional face recognition algorithms rather than state-of-the-art convolutional neural network based face recognition methods. In this work, face image materials from 33 real cases in forensic identification of human images were collected for quantitative analysis of the effects of face image denoising and enhancement methods on the deep face recognition performance of the MXNet system architecture based face recognition system. The results show that face image quality has a significant effect on the recognition performance of the face recognition system, and the image processing techniques can enhance the quality of face images, and then improve the recognition precision of the face recognition system. In addition, the effects of the Gaussian filtering are better than the self-snake model based image enhancement method, which indicates that the image denoising methods are more suitable for performance improvement of the deep face recognition system rather than the image enhancement techniques under the application of the practical cases.

Deep Face Age Progression: A Survey

Face Age Progression (FAP) refers to synthesizing face images while simulating ageing effects, thus enabling predicting the future appearance of an individual. The generation of age-progressed face images brings benefits for various applications, ranging from face recognition systems to forensic investigations and digital entertainment. In particular, the recent success achieved with deep generative networks significantly leveraged the quality of age-synthesized face images in terms of visual fidelity, ageing accuracy and identity preservation. However, the high number of contributions in recent years requires systematically structuring new findings and ideas to identify a common taxonomy, accelerate future research and reduce redundancy. Therefore, we present a comparative analysis of recent deep learning based face age progression methods for both adult and child-based face ageing, broken down into three high-level concepts: translation-based, condition-based, and sequence-based FAP. Further, we offer a comprehensive summary of the most common performance evaluation techniques, cross-age datasets, and open challenges to steer future research in the right direction.

Gabor and Maximum Response Filters with Random Forest Classifier for Face Recognition in the Wild

Research on face recognition has been evolving for decades. There are numerous approaches developed with highly desirable outcomes in constrained environments. In contrast, approaches to face recognition in an unconstrained environment where varied facial posing, occlusion, aging, and image quality still pose vast challenges. Thus, face recognition in the unconstrained environment still an unresolved problem. Many current techniques are not performed well when experimented in unconstrained databases. Additionally, most of the real-world application needs a good face recognition performance in the unconstrained environment. This paper presents a comprehensive process aimed to enhance the performance of face recognition in an unconstrained environment. This paper presents a face recognition system in an unconstrained environment. The fusion between Gabor filters and Maximum Response (MR) filters with Random Forest classifier is implemented in the proposed system. Gabor filters are a hybrid of Gabor magnitude filters and Oriented Gabor Phase Congruency (OGPC) filters. Gabor magnitude filters produce the magnitude response while the OGPC filters produce the phase response of Gabor filters. The MR filters contain the edge- and bar-anisotropic filter responses and isotropic filter responses. In the face features selection process, Monte Carlo Uninformative Variable Elimination Partial Least Squares Regression (MC-UVE-PLSR) is used to select the optimal face features in order to minimize the computational costs without compromising the accuracy of face recognition. Random Forests is used in the classification of the generated feature vectors. The algorithm performance is evaluated using two unconstrained facial image databases: Labelled Faces in the Wild (LFW) and Unconstrained Facial Images (UFI). The proposed technique used produces encouraging results in these evaluated databases in which it recorded face recognition rates that are comparable with other state-of-the-art algorithms.

Learning Spatial Attention for Face Super-Resolution.

General image super-resolution techniques have difficulties in recovering detailed face structures when applying to low resolution face images. Recent deep learning based methods tailored for face images have achieved improved performance by jointly trained with additional task such as face parsing and landmark prediction. However, multi-task learning requires extra manually labeled data. Besides, most of the existing works can only generate relatively low resolution face images (e.g., 128×128 ), and their applications are therefore limited. In this paper, we introduce a novel SPatial Attention Residual Network (SPARNet) built on our newly proposed Face Attention Units (FAUs) for face super-resolution. Specifically, we introduce a spatial attention mechanism to the vanilla residual blocks. This enables the convolutional layers to adaptively bootstrap features related to the key face structures and pay less attention to those less feature-rich regions. This makes the training more effective and efficient as the key face structures only account for a very small portion of the face image. Visualization of the attention maps shows that our spatial attention network can capture the key face structures well even for very low resolution faces (e.g., 16×16 ). Quantitative comparisons on various kinds of metrics (including PSNR, SSIM, identity similarity, and landmark detection) demonstrate the superiority of our method over current state-of-the-arts. We further extend SPARNet with multi-scale discriminators, named as SPARNetHD, to produce high resolution results (i.e., 512×512 ). We show that SPARNetHD trained with synthetic data can not only produce high quality and high resolution outputs for synthetically degraded face images, but also show good generalization ability to real world low quality face images. Codes are available at https://github.com/chaofengc/Face-SPARNet.

Improving the Neural Network Algorithm for Assessing the Quality of Facial Images

The paper considers the task of obtaining a quality assessment of facial images for usage in various video surveillance systems, video analytics, and biometric identification. The accuracy of person recognition and classification depends on the quality of the input images. We consider an approach to obtaining single face image quality assessment using a neural network model, which is trained on pairs of images that are split into two possible classes: the quality of the first image is better or worse than the quality of the second one. Two modifications of the selected baseline algorithm are proposed. A face recognition system is applied to change the loss function and image and face quality attributes are used when training the model. Experimental studies of the proposed modifications show their effectiveness. The accuracy of selecting the best and worst frame is increased by 1.3% and 1.9%, respectively.

Preferred skin tones reproduction of three ethnic groups under different ambient lighting conditions

A large-scale experiment was conducted to investigate facial image quality on mobile phones. There were 8 original facial images from 4 skin tone types, each included a male and a female image. Each image was captured at 6500K and they were rendered to have 5 CCT (correlated colour temperature) and 5 Duv (the shifts away from the Blackbody locus) levels via CAT02 chromatic adaptation transform to simulate the effect of the images captured under 25 different lighting conditions. Each image was assessed under 9 ambient lighting conditions( including one dark condition) by 90 observers from 3 ethnic groups (Caucasian, Chinese and South Asian), each 30 observers. Preferred facial skin tone ellipse was established by maximizing the correlation coefficient between the model predicted probability and the preference percentage from the visual results. Four types of preferred skin tones had small differences in hue angle and chroma, but concentrated into a small colour region, about [24.7, 46.1°] for Cab* and hab values respectively. All ethnic group preferred images taken under illuminants having high CCT (6500-8000 K). It was also found that the chroma of the preferred skin tones will slightly increase as the ambient lighting CCT decrease.

Multi-Frame Labeled Faces Database: Towards Face Super-Resolution from Realistic Video Sequences

Forensically trained facial reviewers are still considered as one of the most accurate approaches for person identification from video records. The human brain can utilize information, not just from a single image, but also from a sequence of images (i.e., videos), and even in the case of low-quality records or a long distance from a camera, it can accurately identify a given person. Unfortunately, in many cases, a single still image is needed. An example of such a case is a police search that is about to be announced in newspapers. This paper introduces a face database obtained from real environment counting in 17,426 sequences of images. The dataset includes persons of various races and ages and also different environments, different lighting conditions or camera device types. This paper also introduces a new multi-frame face super-resolution method and compares this method with the state-of-the-art single-frame and multi-frame super-resolution methods. We prove that the proposed method increases the quality of face images, even in cases of low-resolution low-quality input images, and provides better results than single-frame approaches that are still considered the best in this area. Quality of face images was evaluated using several objective mathematical methods, and also subjective ones, by several volunteers. The source code and the dataset were released and the experiment is fully reproducible.

Attributes guided facial image completion

Recently, facial image completion has witnessed substantial progress due to the increasing power of deep convolutional networks and generative adversarial nets. These methods try to reproduce the pixels in the missing regions of a facial image so that the completion results are indistinguishable from the original one. However, due to the multimodality of image contents, there are often many plausible ways to complete a facial image, which are equally good and meaningful. A user might be specifically interested in obtaining a particular completion result with certain desired attributes. To address this need, we propose a novel deep learning approach to facial image completion with multiple controllable attributes (e.g., gender and smiling). First, an attribute classification loss is proposed to guarantee the completed image should have some desired attributes. Second, our approach introduces a perceptual loss to model high-level semantic structures in a facial image, which greatly reduces structure distortions in the large missing regions. Third, an adversarial loss is employed to ensure the completion results are visually realistic. The three components cooperate well in our approach, forming an effective framework for high quality facial image completion with multiple controllable attributes. Experimental results on the CelebA datasets show our method not only achieves excellent performance on facial completion task, but also allows users to obtain specific completion results with facial attributes of interest.

Off-the-shelf Convolutional Neural Network (CNN) features for Automatic Face Quality Prediction

Estimation of face image quality helps in correctly recognizing faces which in turn helps in many practical applications related to face. This paper presents a face quality prediction approach using Off-the-Shelf CNN features. Here we evaluated three image descriptors-binary patterns (LBP), Histogram of oriented gradients (HOG), Oriented Fast and Rotated Brief (ORB), and deep Convolution Neural Network (CNN) Networks pre-trained on ImageNet-VGG19, ResNet50, and VGG Small (4 layers) for feature extraction to detect face region image quality. Furthermore, to classify extracted features, we have evaluated three classifiers, that are different from each other in their own ways (SVM, DT and MLP) For experimental analysis, we created a face quality dataset by collecting images from web and publicly available face datasets and manually labeled images under seven categories-Good and six bad quality classes (e.g. Expression, Makeup, Pose, Occlusion, Illumination and Blur). The accuracy of face image classification using VGG19 along with MLP as a classifier was the highest (i.e.98.76%) followed by ResNet50 and MLP at 98.69% of accuracy. The lowest accuracy was obtained with LBP and SVM, this shows that deep features gives a better solution.

Full-Reference Facial Image Quality Assessment and Identification by Two Proposed Measures

In this paper, the goal was to identify a person’s face in the acquired image by the proposed measures. We discuss the appearance of two types of noise together in an image. The acquired facial image quality was also assessed by two proposed measures, the histogram similarity measure and the histogram error mean measure. The histogram structural similarity measure is a previously described modified version of the information-theoretic structural similarity measure. It was merged with the structural similarity measure and the error mean measure, derived from the mean squared error, to get the proposed measures. The first proposed histogram similarity measure consists of merging histogram structural similarity with structural similarity measure, and the second proposed histogram error mean measure consists of merging histogram structural similarity with error mean measure. Finally, many algorithms for identification have recently been proposed to measure the similarity between two images. The results showed that the two proposed measures were better than existing methods. Different noises types (such as white Gaussian, speckle, and salt-and-pepper) are used with the proposed methods. Two facial image datasets were used in this paper. The AT&T database included color images of 92 x 112 pixels (px), and the Faculty of Industrial Engineering database included color images of 480 x 640 px. To evaluate performance and quantify the error, the structural similarity measure, histogram structural similarity, and error mean measure were considered. Noise ratios that depended on a peak signal-to-noise ratio were used in this experiment.

Video Content Analysis for Compliance Audit in Finance and Security Industry

The quality and accessibility of modern financial service have been quickly and dramatically improved, which benefits from the fast development of information technology. It has also witnessed the trend for applying artificial intelligence related technology, especially machine learning to the finance and security industry ranging from face recognition to fraud detection. In particular, deep neural networks have proven to be far superior to traditional algorithms in various application scenarios of computer vision. In this paper, we propose a deep learning-based video analysis system for automated compliance audit in stock brokerage, which in general consists of five modules here: 1) Video tampering and integrity detection; 2) Objects of interest localization and association; 3) Analysis of presence and departure of personnel in a video; 4) Face image quality assessment; and 5) Signature action positioning. To the best of our knowledge, this is the first work that introduces remote automated compliance audit system for the dual-recorded video in finance and security industry. The experimental results suggest our system can identify most of the potential non-compliant videos and has greatly improved the working efficiency of the auditors and reduced human labor costs. The collected dataset in our experiment will be released with this paper.

An inspired haxby brain perceptual model for facial images quality assessment

Face is the most important and most popular biometric used in many identification and verification systems. In these systems, for reducing recognition error rate, the quality of input images need to be as high as possible. Face Image Compliancy verification (FICV) is one of the most essential methods for this purpose. In this research, a brain functionality inspired model is presented for FICV using Haxby model, which is a face visual perception consistent model containing three bilateral areas for three different functionalities. As a result, contribution of this work is presenting a new model, based on human brain functionality, improving the compliancy verification of face images in FICV context. Perceptual understanding of an image is the motivation of most of the quality assessment methods, i.e., the human quality perception is considered as a gold standard and a perfect reference for recognition and quality assessment. The model presented in this work aims to make the operational process of face image quality assessment system closer to the performance of a human expert. Three basic modules have been introduced. Face structural information, for initial information encoding, is simulated by an extended Viola-Jones model. Face image quality assessment is presented by International Civil Aviation Organization (ICAO), in ICAO (ISO / IEC19794 -11) requirements’ compliancy assessment document. Like Haxby model, perception is performed through two distinct functional and neurological pathways, using Hierarchical Maximum pooling (HMAX) and Convolutional Deep Belief Networks (CDBN). Information storing and fetching for training are similar to their corresponding modules in brain. For simulating the brain decision making, the final results of two separate paths are integrated by weighting sum operator. Nine ISO / ICAO requirements were used for testing the model. The simulation results, using AR and PUT databases, shows improvements in six requirements using the proposed method, in comparison with the FICV benchmark.

Low Resolution Face Recognition System Performance for Basic Feature Space Methods with Various Illumination Normalization Techniques

The Face recognition research identified with the field of automated surveillance systems in real life application has pulled in more consideration today with extensive use of vision cameras in biometrics and surveillance applications, globally. The small size images or poor quality images are generally Low Resolution images. Authenticating faces, with variation in pose, illumination, disguise and more, from such Low-resolution images, is the main purpose of Low Resolution Face Recognition (LR FR) system. We have assessed the LR FR system for various basic feature space techniques like PCA, LDA and Fisherface. Different illumination normalization techniques are applied on the cropped Yale face database prior to feature extraction and identification. In our work the low-resolution images, of size 32x32, are the down-sampled versions high-resolution facial images from cropped Yale face database. Our experiment demonstrates the encouraging performance, with recognition accuracy as 97.43%, on Low Resolution, Low quality face images.

Novel Face Recognition Framework for Plastic Surgery and Unconstrained Facial Datasets

Since from the last decade, the importance of efficient and robust face recognition becomes interesting research problem due to its growing demand in user authentication process across different applications. Several face recognition method designed by considering the challenges of constrained and unconstrained face acquisition. The performance of face recognition systems becomes worst while working with unconstrained and plastic surgery datasets. The unconstrained dataset may contain the significant variations in illumination, expressions, and face image quality. This paper proposed the novel face recognition framework which is built on top of hybrid face descriptor and hybrid features extraction technique. The purpose of hybrid face descriptor is to achieve the bridge the gap between spatial information and histogram representations to address the challenges of unconstrained face conditions efficiently. The hybrid features extraction includes the histogram features (single and multi-level) and invariant features. Since the moment invariant features become effective shape descriptor to address the challenges of unconstrained face images, we proposed 11 different invariant moments from the hybrid face descriptor. To normalize and reduce the features, we applied lightweight features selection technique. The experimental result shows that proposed face recognition framework improves the overall accuracy different datasets.

Recognition oriented facial image quality assessment via deep convolutional neural network

Quality of facial images significantly impacts the performance of face recognition algorithms. Being able to predict “which facial image is good for recognition” is of great importance for real application scenarios, where a sequence of facial images are always presented and one should select the image frame with “best quality” for the subsequent matching and recognition task. To this end, we introduce a novel facial image quality automatic assessment framework directly targeting at “selecting better face image for better face recognition”. For such as purpose, a deep convolutional neural network (DCNN) is trained to output a general facial quality metric which comprehensively considers various quality factors including brightness, contrast, blurriness, occlusion, and pose etc. Based on this trained facial quality metric network, we are able to sort the input face images accordingly and “select” good face images for recognition. Our method is comprehensively evaluated on Color FERET and KinectFace face datasets. Results show that the proposed facial image quality metric network works end-to-end and it well distinguishes “good” images from “bad” ones, which is highly correlated with the final recognition performance.