Facial Feature Space Research Articles

Learning a Deep Regression Forest for Head Pose Estimation from a Single Depth Image

Robust head pose estimation significantly improves the performance of applications related to face analysis in Cyber-Physical Systems (CPS) such as driving assistance and expression recognition. However, there exist two main challenges in this issue, i.e., the large pose variations and the property of inhomogeneous facial feature space. Head pose in large variations makes the distinguished facial features, such as nose or lips, invisible, especially in extreme cases. Additionally, features extracted from a head do not change in a stationary manner with respect to the head pose, which results in an inhomogeneous feature space. To deal with the above problems, we propose an end-to-end framework to estimate the head pose from a single depth image. To be specific, the PointNet network is adopted to automatically select distinguished facial feature points from visible surface of a head and to extract discriminative features. The Deep Regression Forest is utilized to handle the nonstationary property of the facial feature space and to learn the head pose distributions. Experimental results show that our proposed method achieves the state-of-the-art performance on the Biwi Kinect Head Pose Dataset, the Pandora Dataset and the ICT-3DHP Dataset.

Deep Differentiable Random Forests for Age Estimation.

Age estimation from facial images is typically cast as a label distribution learning or regression problem, since aging is a gradual progress. Its main challenge is the facial feature space w.r.t. ages is inhomogeneous, due to the large variation in facial appearance across different persons of the same age and the non-stationary property of aging. In this paper, we propose two Deep Differentiable Random Forests methods, Deep Label Distribution Learning Forest (DLDLF) and Deep Regression Forest (DRF), for age estimation. Both of them connect split nodes to the top layer of convolutional neural networks (CNNs) and deal with inhomogeneous data by jointly learning input-dependent data partitions at the split nodes and age distributions at the leaf nodes. This joint learning follows an alternating strategy: (1) Fixing the leaf nodes and optimizing the split nodes and the CNN parameters by Back-propagation; (2) Fixing the split nodes and optimizing the leaf nodes by Variational Bounding. Two Deterministic Annealing processes are introduced into the learning of the split and leaf nodes, respectively, to avoid poor local optima and obtain better estimates of tree parameters free of initial values. Experimental results show that DLDLF and DRF achieve state-of-the-art performance on three age estimation datasets.

Face recognition method with feature fusion and error feedback

A face recognition method with feature fusion and error feedback is proposed to imitate human cognition process. The intelligent recognition mode with multi-level feedback from global to local and from coarse to fine is implemented. Firstly, based on the idea of feedback control, a multi-level feedback model of face recognition with performance index constraints and supervised learning under finite domain is established. Secondly, based on entropy theory and Stochastic Configuration Networks (SCN), a facial feature space and face image classification network model with multi-feature fusion and sufficient separability under certain feature performance conditions are established from the perspective of information theory. Thirdly, based on entropy theory and error theory, the face recognition error representation and calculation model under certain feature performance conditions is established. Finally, the adjustment index of facial feature performance based on recognition error is constructed, and the face recognition algorithm with feature fusion and error feedback is given out. Test and comparison experiments show that our method is effective and feasible.

Hybrid BFO and PSO Swarm Intelligence Approach for Biometric Feature Optimization

Nature-inspired novel swarm intelligence algorithms have gained more proliferation due to a variety of applications and uses in optimization of complex problems and selection of discriminatory sets of features to classify huge datasets during the past few decades. Feature selection is an efficient and useful pre-processing technique for solving classification problems in computer vision, data mining and pattern recognition. The major challenges of solving the feature selection problems lay in swarm intelligence algorithms which are capable of handling the vast number of feature sets from involved databases. In biometric based recognition systems, face recognition is a non-intrusive approach to identify individuals based on their discriminatory sets of facial feature vectors. In this paper, the authors tend to propose a unique novel hybrid based on Bacterial Foraging Optimization (BFO) and Particle swarm optimization (PSO) approach for the selection of best facial feature vectors that enhance the identification accuracy of the individual recognition because concerned facial info will contain useless and redundant face expression. The proposed hybrid approach mitigates irrelevant facial features in the feature space and selects the relevant set of features from the facial feature space. The proposed feature selection approach presents promising experimental results with respect to the number of facial feature subsets. The identification accuracies are superior to other approaches from the literature.

The evolution of face processing in primates.

The ability to recognize faces is an important socio-cognitive skill that is associated with a number of cognitive specializations in humans. While numerous studies have examined the presence of these specializations in non-human primates, species where face recognition would confer distinct advantages in social situations, results have been mixed. The majority of studies in chimpanzees support homologous face-processing mechanisms with humans, but results from monkey studies appear largely dependent on the type of testing methods used. Studies that employ passive viewing paradigms, like the visual paired comparison task, report evidence of similarities between monkeys and humans, but tasks that use more stringent, operant response tasks, like the matching-to-sample task, often report species differences. Moreover, the data suggest that monkeys may be less sensitive than chimpanzees and humans to the precise spacing of facial features, in addition to the surface-based cues reflected in those features, information that is critical for the representation of individual identity. The aim of this paper is to provide a comprehensive review of the available data from face-processing tasks in non-human primates with the goal of understanding the evolution of this complex cognitive skill.

Development of sensitivity to spacing versus feature changes in pictures of houses: Evidence for slow development of a general spacing detection mechanism?

Adults are expert at recognizing faces, in part because of exquisite sensitivity to the spacing of facial features. Children are poorer than adults at recognizing facial identity and less sensitive to spacing differences. Here we examined the specificity of the immaturity by comparing the ability of 8-year-olds, 14-year-olds, and adults to discriminate houses differing in the spacing between features versus those differing in the shape of the features themselves. By 8 years of age, children were more accurate for discriminations involving the feature set compared with the spacing set, and the difference in accuracy compared with adults was greater for the spacing set than for the feature set. Importantly, when sets were matched in difficulty for adults, this greater immaturity on the spacing set than on the feature set remained. The results suggest that, at least by age 8, immaturities in sensitivity to the spacing of features may be related to immaturities in general perceptual mechanisms rather than face-specific mechanisms.

Quantitative analysis of human facial beauty using geometric features

Perception of human facial beauty is an important aspect of human intelligence and has attracted interests of researchers from diverse fields such as psychology and computer science. Previous studies, however, have the following limitations. First, they did not well quantify the facial feature space. Second, they seldom consider the transformation occurring to faces or the physical sizes of faces. Third, most of them require intensive manual work, e.g. marking landmarks. To overcome these limitations, this paper maps faces onto a human face shape space, and then quantitatively analyses the effect of facial geometric features on human facial beauty by using a similarity transformation invariant shape distance measurement and advanced automatic image processing techniques. With the proposed methodology, we experiment on tens of thousands of female and male faces, revealing that human face shapes lie in a very compact region of the geometric feature space and that female and male average face shapes are very similar. Further, we demonstrate that a face can become more beautiful by making its geometric feature getting obviously closer to the average face shape, but if its distance to the average face shape is already relatively small, deforming it further toward the average face shape cannot effectively improve its attractiveness.

The Function and Specificity of Sensitivity to Cues to Facial Identity: An Individual-Differences Approach

The expertise of adults in recognising the identity of individual faces has been attributed to their exquisite sensitivity to differences among faces in the spacing of features (second-order relations). However, the reliability of individual differences and the extent to which this sensitivity predicts individuals' ability to recognise faces has not been tested directly. We administered two sets of tasks to adult females (n = 31); the tests were separated by 2 to 11 days. Individual differences in sensitivity to the spacing of facial features were reliable across days and correlated with individual differences in sensitivity to the spacing of features (doors and windows) in houses, but did not predict accuracy when participants matched facial identity across changes in point of view. Individual differences in sensitivity to featural cues to facial identity were not reliable, likely because of ceiling effects. The function and specificity of sensitivity to the spacing of features is discussed.

Limitations in 4‐Year‐Old Children’s Sensitivity to the Spacing Among Facial Features

Four-year-olds' sensitivity to differences among faces in the spacing of features was tested under 4 task conditions: judging distinctiveness when the external contour was visible and when it was occluded, simultaneous match-to-sample, and recognizing the face of a friend. In each task, the foil differed only in the spacing of features, and spacing alterations were within normal limits. Children performed at chance levels in all but 1 task-match-to-sample, and in that task, only 10 of 18 children were correct on more than 50% of the trials. Sensitivity to the spacing of facial features in identity and distinctiveness tasks is very weak at 4 years of age-at least when the changes do not exceed +/-2.5 SD of normal variability.

The utility of surface reflectance for the recognition of upright and inverted faces

The variation among faces can be partitioned into two sources: (a) shape and (b) surface reflectance. To compare the utility of shape and reflectance for face recognition, we created two sets of faces, with individual exemplars differing only by shape in one set and only by reflectance in the other set. Grayscale and full color versions of the stimuli were used in separate experiments; the physical variation between exemplars was equated across the two sets with the grayscale but not the full color stimuli. Subjects performed a matching task in which both the target and distractor were drawn from the same set, so that only shape or only reflectance information could be used to perform the task. With the grayscale stimuli, performance was better in the shape condition, but with the color stimuli, performance was better in the reflectance condition. Inversion of the faces disrupted performance with the shape and reflectance sets about equally, suggesting that the inversion effect is not caused specifically by the spacing of facial features, or even by shape information more generally. These results provide evidence that facial identity is a function of reflectance as well as shape, and place important constraints on explanations of why inversion impairs face recognition.

Becoming a Face Expert

Expertise in recognizing facial identity, and, in particular, sensitivity to subtle differences in the spacing among facial features, improves into adolescence. To assess the influence of experience, we tested adults and 8-year-olds with faces differing only in the spacing of facial features. Stimuli were human adult, human 8-year-old, and monkey faces. We show that adults' expertise is shaped by experience: They were 9% more accurate in seeing differences in the spacing of features in upright human faces than in upright monkey faces. Eight-year-olds were 14% less accurate than adults for both human and monkey faces (Experiment 1), and their accuracy for human faces was not higher for children's faces than for adults' faces (Experiment 2). The results indicate that improvements in face recognition after age 8 are not related to experience with human faces and may be related to general improvements in memory or in perception (e.g., hyperacuity and spatial integration).

Why 8-year-olds cannot tell the difference between Steve Martin and Paul Newman:Factors contributing to the slow development of sensitivity to the spacing of facial features

Children are nearly as sensitive as adults to some cues to facial identity (e.g., differences in the shape of internal features and the external contour), but children are much less sensitive to small differences in the spacing of facial features. To identify factors that contribute to this pattern, we compared 8-year-olds’ sensitivity to spacing cues with that of adults under a variety of conditions. In the first two experiments, participants made same/different judgments about faces differing only in the spacing of facial features, with the variations being kept within natural limits. To measure the effect of attention, we reduced the salience of featural information by blurring faces and occluding features (Experiment 1). To measure the role of encoding speed and memory limitations, we presented pairs of faces simultaneously and for an unlimited time (Experiment 2). To determine whether participants’ sensitivity would increase when spacing distortions were so extreme as to make the faces grotesque, we manipulated the spacing of features beyond normal limits and asked participants to rate each face on a “bizarreness” scale (Experiment 3). The results from the three experiments indicate that low salience, poor encoding efficiency, and limited memory can partially account for 8-year-olds’ poor performance on face processing tasks that require sensitivity to the spacing of features, a kind of configural processing that underlies adults’ expertise. However, even when the task is modified to compensate for these problems, children remain less sensitive than adults to the spacing of features.

Why 8-year-olds cannot tell the difference between Steve Martin and Paul Newman:: Factors contributing to the slow development of sensitivity to the spacing of facial features

Children are nearly as sensitive as adults to some cues to facial identity (e.g., differences in the shape of internal features and the external contour), but children are much less sensitive to small differences in the spacing of facial features. To identify factors that contribute to this pattern, we compared 8-year-olds’ sensitivity to spacing cues with that of adults under a variety of conditions. In the first two experiments, participants made same/different judgments about faces differing only in the spacing of facial features, with the variations being kept within natural limits. To measure the effect of attention, we reduced the salience of featural information by blurring faces and occluding features (Experiment 1). To measure the role of encoding speed and memory limitations, we presented pairs of faces simultaneously and for an unlimited time (Experiment 2). To determine whether participants’ sensitivity would increase when spacing distortions were so extreme as to make the faces grotesque, we manipulated the spacing of features beyond normal limits and asked participants to rate each face on a “bizarreness” scale (Experiment 3). The results from the three experiments indicate that low salience, poor encoding efficiency, and limited memory can partially account for 8-year-olds’ poor performance on face processing tasks that require sensitivity to the spacing of features, a kind of configural processing that underlies adults’ expertise. However, even when the task is modified to compensate for these problems, children remain less sensitive than adults to the spacing of features.

Neuroperception. Early visual experience and face processing.

Adult-like expertise in processing face information takes years to develop1 and is mediated in part by specialized cortical mechanisms2 sensitive to the spacing of facial features (configural processing)3. Here we show that deprivation of patterned visual input from birth until 2–6 months of age results in permanent deficits in configural face processing. Even after more than nine years' recovery, patients treated for bilateral congenital cataracts were severely impaired at differentiating faces that differed only in the spacing of their features, but were normal in distinguishing those varying only in the shape of individual features. These findings indicate that early visual input is necessary for normal development of the neural architecture that will later specialize for configural processing of faces.