Facial Feature Representation Research Articles

Weighted Patch-based Manifold Regularization Dictionary Pair Learning model for facial expression recognition using Iterative Optimization Classification Strategy

This paper presents a novel method for solving Facial Expression Recognition (FER) task which integrates Manifold Regularization Dictionary Pair Learning and Intra-class Variation Reduced Feature (MRDPL-IVRF) model and Iterative Optimization Classification Strategy (IOCS). In the proposed framework, we firstly divide the facial image into overlapping patches. Weighted Patch-based Local Binary Patterns (WPLBP) is proposed for feature extraction by cascading the weighted LBP features from each patch, which highlights the informative facial regions. MRDPL-IVRF model is then adopted for robust sparse representation of facial features, where IVRFs are generated from the difference between query images and their corresponding approximation of each expression class, which can reduce the intra-class variations resulted by illumination and identity. Finally, IOCS is presented for expression classification which can explore the most unlikely class according to the maximum reconstruction residual. Different from traditional sparse coding classification scheme, our IOCS method iteratively removes irrelevant samples to obtain a more precise representation model, thus can further improve the recognition performance. The feasibility of our framework has been successfully tested on CK, CK+, SEFW, CMU-PIE and Multi-PIE databases. Furthermore, the experiments combining deep features shows that our model can be integrated with existing deep models and obtain better FER results.

Attended Relation Feature Representation of Facial Dynamics for Facial Authentication

In psychology, it is known that facial dynamics benefit the perception of identity. This paper proposes a novel deep network framework to capture identity information from facial dynamics and their relations. In the proposed method, facial dynamics occurred from a smile expression are analyzed and utilized for facial authentication. Detailed changes in the local regions of a face such as wrinkles and dimples are encoded in the facial dynamic feature representation. The latent relationships of the facial dynamic features are learned by the facial dynamic relational network. In the facial dynamic relational network, the relation features of the facial dynamic are encoded and the relational importance is encoded based on the relation features. As a result, the proposed method has more attention on the important relation features in facial authentication. Through comprehensive and comparative experiments, the effectiveness of the proposed method has been verified in facial authentication.

Near Real-Time Three Axis Head Pose Estimation Without Training

Head pose estimation methods evaluate the amount of head rotation according to two or three axes, aiming at optimizing the face acquisition process, or extracting neutral-pose frames from a video sequence. Most approaches to pose estimation exploits machine-learning techniques requiring a training phase on a large number of positive and negative examples. In this paper, a novel pose estimation method that exploits a quad-tree-based representation of facial features is described. The locations of a set of landmarks detected over the face image guide its subdivision into smaller and smaller quadrants based on the presence or lack of landmarks within each quadrant. The proposed pose descriptor is both effective and efficient, providing accurate yaw, pitch and roll axis estimates almost in real-time, without need for any training or previous knowledge about the subject. The experiments conducted on both the BIWI Kinect Head Pose Database and the challenging automated facial landmarks in the wild dataset, highlight a pose estimate precision exceeding the state-of-the-art with regard to methods not involving training and machine learning approaches.

Unsupervised Feature Learning with Single Layer ICANet for Face Recognition

Compared to supervised learning, unsupervised learning allows systems to learn consistent patterns from cheap and abundant unlabeled data, without the need for manual annotation. In this paper, we present a novel unsupervised feature learning method with single layer (SL) network based independent component analysis (ICA) filters called SL-ICANet, with the goal of achieving compact and robust facial feature representation. Our contributions are twofold: (i) We developed a single-layer convolutional network for unsupervised learning wherein the trainable kernels are replaced by ICA filters. (ii) We extended our SL-ICANet to use multi-scale information for better feature learning. Extensive experiments on two popular face recognition benchmarks, namely, labeled faces in the wild and facial recognition technology show that the proposed method might serve as a simple but highly competitive baseline for face recognition.

Face recognition algorithm using extended vector quantization histogram features.

In this paper, we propose a face recognition algorithm based on a combination of vector quantization (VQ) and Markov stationary features (MSF). The VQ algorithm has been shown to be an effective method for generating features; it extracts a codevector histogram as a facial feature representation for face recognition. Still, the VQ histogram features are unable to convey spatial structural information, which to some extent limits their usefulness in discrimination. To alleviate this limitation of VQ histograms, we utilize Markov stationary features (MSF) to extend the VQ histogram-based features so as to add spatial structural information. We demonstrate the effectiveness of our proposed algorithm by achieving recognition results superior to those of several state-of-the-art methods on publicly available face databases.

AGE ESTIMATION USING NEURAL NETWORKS BASED ON FACE IMAGES WITH STUDY OF DIFFERENT FEATURE EXTRACTION METHODS

Facial age estimation recently becomes active research topic in pattern recognition. As there are vast potential application in age specific human computer interaction security control and surveillance monitoring. Insufficient and incomplete training data, uncontrollable environment, facial expression are the most prominent challenges in facial age estimation. Degree of accuracy for age estimation is obtained by forming appropriate feature vector of a facial image. Feature vectors are constructed from facial features. Therefore comparative study of feature extraction from facial image by bio inspired feature (BIF), histogram of gradient (HOG), Gabor filter, wavelet transform and scattering transform is done. The propose approach exploits scattering transform gives more information about features of the facial images. Well organized system consist scattering transform that disperse gabber coefficients pulling with smooth gaussian process in number of layers which isused to calculate for facial feature representation. These extracted features are classified using support vector machine and artificial neural network.

Joint facial expression recognition and intensity estimation based on weighted votes of image sequences

Facial behavior consists of dynamically changing properties of facial features as a result of muscle activation. Facial behavior analysis is a challenging problem due to complexity of emotions and variability of the facial expressions associated with the emotions. Most facial expression recognition systems attempt to recognize facial expressions without taking into account the intensity of the expressions. In this paper, a novel framework for facial expression recognition and intensity estimation with low computational complexity requirement is proposed. The algorithm constructs a representation of facial features based on a weighted voting scheme and employs Hidden Markov Models to classify an input video into one of the six basic expressions, namely anger, disgust, fear, happiness, sadness, and surprise. The temporal segments, neutral, onset, and apex, of an expression are then obtained by means of a change-point detector. Evaluations on subject-independent analysis was conducted using Cohn-Kanade dataset and Beihang University facial expression datasets. The proposed approach has demonstrated a superior performance in recognizing facial expressions and estimating expression intensities.

Experimental comparison of approaches for feature extraction of facial attributes

In this paper, we compare the effectiveness of widely used approaches for representation of facial features in face images. Feature extraction is performed on face images for representation of four facial attributes, namely gender, age, race, and expression, by using discrete wavelet transform (DWT), Gabor wavelet, scale-invariant feature transform, local binary pattern (LBP), and Eigenfaces. After feature extraction and dimension reduction, demographic and expression classification is performed to identify the most discriminating techniques for representation of facial features. Extensive experiments are performed using publicly available face databases, namely Yale, Face95 Essex, and Cohn-Kanade (CK+) databases. Experimental results show that DWT, LBP, and Gabor wavelet methods are robust to variations of illumination, facial expression, and geometric transformations. Experimental results also show that race and expression are more difficult to predict than gender and age.

Distinctive feature representations and classifiers for age invariant face recognition - a survey

Face recognition across ages is very challenging area for research. It is highly affected with many uncontrolled factors like expressions, lightning, head position and aging. Though many researchers contributed their studies to solve this problem still it is unsolved. Facial feature representations and classifiers play an important role in age invariant face recognition. Thus, in this paper, we have presented a complete survey on distinctive feature representations along with classifiers useful in this area. The study includes features like LBP and its various modifications, Gabor wavelets, gradient orientation pyramid (GOP), α-shape and PC-based features. As classifiers play an important role to recognise face images across ages, some of the classifiers like support vector machines, neural networks, convolutional neural networks, etc. are also discussed. The comparative analysis of distinctive age invariant face recognition models is also presented in this paper. Two very widely and publically available datasets in this area, FGNET and MORPH are also discussed in details and compared.

Recognizing faces with normalized local Gabor features and Spiking Neuron Patterns

Gabor Wavelets (GW) have been extensively used for facial feature representation due to its inherent multi-resolution and multi-orientation characteristics. In this work we extend the work on Local Gabor Feature Vector (LGFV) and propose a new face recognition method called LGFV//LN//SNP, which employs local normalization filter in pre-processing stage. We propose a novel Spiking Neuron Patterns (SNP) as a dimensionality reduction method to reduce the dimensions of local Gabor features. SNP is acquired from projection of LGFV//LN features using Spike Response Model (SRM), a neuron model describing the spike behavior of a biological neuron. Results on AR, FERET, Yale B and FRGC 2.0 face datasets showed that SNP implementation delivered significant improvement in accuracy. Comparisons with several previously published results also suggested that LGFV//LN//SNP achieved better results in some tests. Additionally, LGFV//LN//SNP requires relatively smaller number of GW than LGFV//LN to produce optimal results.

A study on the discriminating characteristics of Gabor phase-face and an improved method for face recognition

In this paper, characteristics of different Gabor phase based facial feature representations are studied for the face recognition problem. Experiments show that filter responses acquired from larger scales show higher discriminating ability. Based on this study, following methods are proposed: (1) a faster face recognition method that is forty times faster than the conventional Gabor face representation that can discriminate faces with 91.4 % accuracy, (2) a five times faster method than the conventional representation that showed 95.7 % accuracy, (3) a novel weighted vote of Gabor filters-based face recognition method which showed 99 % accuracy, and (4) an online face authentication system using global and person specific threshold. The tests are performed using a 1200 subject dataset collected from a combination of three databases: FERET, Indian and in-house.

Task-dependent multi-task multiple kernel learning for facial action unit detection

Facial action unit (AU) detection from images and videos is a challenging research topic and has attracted great attention in the past few years. This paper presents a novel method, task-dependent multi-task multiple kernel learning (TD-MTMKL), to jointly detect the absence and presence of multiple AUs. TD-MTMKL attempts to learn an optimal kernel combination from a given set of basis kernels for each involved AU and obtain a finer depiction of AU relations through kernel combination weights. In other words, AU detection is solved as a multi-task multiple kernel learning problem, where AU relations are encoded via their SVM discriminative hyperplanes and kernel combination weights. The kernel learning increases the discriminant power of the classifier by fusing different types of facial feature representations with multiple kernels. Specifically, based on the TD-MTMKL method proposed in this paper, co-occurrence AUs share the same kernel weights while AUs with weak co-occurrence relations may employ distinct sets of kernels. Such “task-dependent” kernel learning framework seeks a trade-off between capturing commonalities and adapting to variations in modeling AU relations. Our experiments on the CK+ and DISFA databases show that our method achieved encouraging detection results of both post and spontaneous AUs compared to the state-of-the-art methods.

Facial expression recognition using $${l}_{p}$$ l p -norm MKL multiclass-SVM

Automatic recognition of facial expressions is an interesting and challenging research topic in the field of pattern recognition due to applications such as human---machine interface design and developmental psychology. Designing classifiers for facial expression recognition with high reliability is a vital step in this research. This paper presents a novel framework for person-independent expression recognition by combining multiple types of facial features via multiple kernel learning (MKL) in multiclass support vector machines (SVM). Existing MKL-based approaches jointly learn the same kernel weights with $$l_{1}$$l1-norm constraint for all binary classifiers, whereas our framework learns one kernel weight vector per binary classifier in the multiclass-SVM with $$l_{p}$$lp-norm constraints $$(p \ge 1)$$(p?1), which considers both sparse and non-sparse kernel combinations within MKL. We studied the effect of $$l_{p}$$lp-norm MKL algorithm for learning the kernel weights and empirically evaluated the recognition results of six basic facial expressions and neutral faces with respect to the value of "$$p$$p". In our experiments, we combined two popular facial feature representations, histogram of oriented gradient and local binary pattern histogram, with two kernel functions, the heavy-tailed radial basis function and the polynomial function. Our experimental results on the CK$$+$$+, MMI and GEMEP-FERA face databases as well as our theoretical justification show that this framework outperforms the state-of-the-art methods and the SimpleMKL-based multiclass-SVM for facial expression recognition.

Facial Expression Recognition via Non-Negative Least-Squares Sparse Coding

Sparse coding is an active research subject in signal processing, computer vision, and pattern recognition. A novel method of facial expression recognition via non-negative least squares (NNLS) sparse coding is presented in this paper. The NNLS sparse coding is used to form a facial expression classifier. To testify the performance of the presented method, local binary patterns (LBP) and the raw pixels are extracted for facial feature representation. Facial expression recognition experiments are conducted on the Japanese Female Facial Expression (JAFFE) database. Compared with other widely used methods such as linear support vector machines (SVM), sparse representation-based classifier (SRC), nearest subspace classifier (NSC), K-nearest neighbor (KNN) and radial basis function neural networks (RBFNN), the experiment results indicate that the presented NNLS method performs better than other used methods on facial expression recognition tasks.

Facial Expression Recognition Based on Monogenic Binary Coding

A new method based on Monogenic Binary Coding (MBC) is proposed for facial expression feature extraction and representation. Firstly, monogenic signal analysis is used to extract multi-scale magnitude, orientation and phase components. Secondly, Monogenic Binary Coding (MBC) is used to encode the monogenic local variation and intensity in local regions of each extracted component in each scale and local histograms are built. Then Blocked Fisher Linear Discrimination (BFLD) is used to reduce the dimensionality of histogram features and to enhance discrimination. Finally the three complementary components are fused for more effective facial expression recognition (FER). Experiment results on Japanese female expression database (JAFFE) show that the performance of the fusion method is even better than state-of-the-art local feature based FER methods such as Local Binary Pattern (LBP)+Sparse Representation (SRC), Local Phase Quantization (LPQ)+SRC ,etc.

Qualitative and quantitative analysis for facial complexion in traditional Chinese medicine.

Facial diagnosis is an important and very intuitive diagnostic method in Traditional Chinese Medicine (TCM). However, due to its qualitative and experience-based subjective property, traditional facial diagnosis has a certain limitation in clinical medicine. The computerized inspection method provides classification models to recognize facial complexion (including color and gloss). However, the previous works only study the classification problems of facial complexion, which is considered as qualitative analysis in our perspective. For quantitative analysis expectation, the severity or degree of facial complexion has not been reported yet. This paper aims to make both qualitative and quantitative analysis for facial complexion. We propose a novel feature representation of facial complexion from the whole face of patients. The features are established with four chromaticity bases splitting up by luminance distribution on CIELAB color space. Chromaticity bases are constructed from facial dominant color using two-level clustering; the optimal luminance distribution is simply implemented with experimental comparisons. The features are proved to be more distinctive than the previous facial complexion feature representation. Complexion recognition proceeds by training an SVM classifier with the optimal model parameters. In addition, further improved features are more developed by the weighted fusion of five local regions. Extensive experimental results show that the proposed features achieve highest facial color recognition performance with a total accuracy of 86.89%. And, furthermore, the proposed recognition framework could analyze both color and gloss degrees of facial complexion by learning a ranking function.

Does My Face FIT?: A Face Image Task Reveals Structure and Distortions of Facial Feature Representation

Despite extensive research on face perception, few studies have investigated individuals’ knowledge about the physical features of their own face. In this study, 50 participants indicated the location of key features of their own face, relative to an anchor point corresponding to the tip of the nose, and the results were compared to the true location of the same individual’s features from a standardised photograph. Horizontal and vertical errors were analysed separately. An overall bias to underestimate vertical distances revealed a distorted face representation, with reduced face height. Factor analyses were used to identify separable subconfigurations of facial features with correlated localisation errors. Independent representations of upper and lower facial features emerged from the data pattern. The major source of variation across individuals was in representation of face shape, with a spectrum from tall/thin to short/wide representation. Visual identification of one’s own face is excellent, and facial features are routinely used for establishing personal identity. However, our results show that spatial knowledge of one’s own face is remarkably poor, suggesting that face representation may not contribute strongly to self-awareness.

Global Feature Extraction and Recognition from Matrices of Gabor Feature Faces

This paper presents a method for facial feature representation and recognition from the Covariance Matrices of the Gabor-filtered images. Gabor filters are a very powerful tool for processing images that respond to different local orientations and wave numbers around points of interest, especially on the local features on the face. This is a very unique attribute needed to extract special features around the facial components like eyebrows, eyes, mouth and nose. The Covariance matrices computed on Gabor filtered faces are adopted as the feature representation for face recognition. Geodesic distance measure is used as a matching measure and is preferred for its global consistency over other methods. Geodesic measure takes into consideration the position of the data points in addition to the geometric structure of given face images. The proposed method is invariant and robust under rotation, pose, or boundary distortion. Tests run on random images and also on publicly available JAFFE and FRAV3D face recognition databases provide impressively high percentage of recognition.

External facial features modify the representation of internal facial features in the fusiform face area

Most studies of face identity have excluded external facial features by either removing them or covering them with a hat. However, external facial features may modify the representation of internal facial features. Here we assessed whether the representation of face identity in the fusiform face area (FFA), which has been primarily studied for internal facial features, is modified by differences in external facial features. We presented faces in which external and internal facial features were manipulated independently. Our findings show that the FFA was sensitive to differences in external facial features, but this effect was significantly larger when the external and internal features were aligned than misaligned. We conclude that the FFA generates a holistic representation in which the internal and the external facial features are integrated. These results indicate that to better understand real-life face recognition both external and internal features should be included.

Face recognition based on eigen features of multi scaled face components and an artificial neural network

Abstract Face recognition has been a very active research area in the past two decades. Many attempts have been made to understand the process of how human beings recognize human faces. It is widely accepted that face recognition may depend on both componential information (such as eyes, mouth and nose) and non-componential/holistic information (the spatial relations between these features), though how these cues should be optimally integrated remains unclear. In the present study, a different observer’s approach is proposed using eigen/fisher features of multi-scaled face components and artificial neural networks. The basic idea of the proposed method is to construct facial feature vector by down-sampling face components such as eyes, nose, mouth and whole face with different resolutions based on significance of face component, and then subspace Principal Component Analysis (PCA) or Linear Discriminant Analysis (LDA) method is employed for further dimensionality reduction and to acquire a good representation of facial features. Each face in the database to be recognized is projected onto eigenspace or fisherface to find its weight vector. The weight vectors of face images to be trained become the input to a neural network classifier, which uses back propagation/radial basis functions to recognize faces with variation in facial expression, and with/without spectacles. The proposed algorithm has been tested on 400 faces of 10 subjects of the ORL database and results are encouraging compared to the existing methods in literature.