Frontal View Face Research Articles

2-2 実写映像と距離情報による正面顔映像のリアルタイム生成に関する検討(第2部門 情報ディスプレイ,マルチメディアストレージ,コンシューマエレクトロニクス,情報センシング)

2-2 実写映像と距離情報による正面顔映像のリアルタイム生成に関する検討(第2部門 情報ディスプレイ,マルチメディアストレージ,コンシューマエレクトロニクス,情報センシング)

Facial expression recognition of a speaker using front-view face judgment, vowel judgment, and thermal image processing

For facial expression recognition, we selected three images: (i) just before speaking, (ii) speaking the first vowel, and (iii) speaking the last vowel in an utterance. In this study, as a pre-processing module, we added a judgment function to distinguish a front-view face for facial expression recognition. A frame of the front-view face in a dynamic image is selected by estimating the face direction. The judgment function measures four feature parameters using thermal image processing, and selects the thermal images that have all the values of the feature parameters within limited ranges which were decided on the basis of training thermal images of front-view faces. As an initial investigation, we adopted the utterance of the Japanese name "Taro," which is semantically neutral. The mean judgment accuracy of the front-view face was 99.5% for six subjects who changed their face direction freely. Using the proposed method, the facial expressions of six subjects were distinguishable with 84.0% accuracy when they exhibited one of the intentional facial expressions of "angry," "happy," "neutral," "sad," and "surprised." We expect the proposed method to be applicable for recognizing facial expressions in daily conversation.

Depth Estimation of Face Images Based on the Constrained ICA Model

In this paper, we propose a novel and efficient algorithm to reconstruct the 3-D structure of a human face from one or a number of its 2-D images with different poses. In our proposed algorithm, the rotation and translation process from a frontal-view face image to a nonfrontal-view face image is at first formulated as a constrained independent component analysis (cICA) model. Then, the overcomplete ICA problem is converted into a normal ICA problem by incorporating a prior from the CANDIDE 3-D face model. Furthermore, the CANDIDE model is employed to construct a reference signal that is used in both the initialization and the objective function of the cICA model. Moreover, a model-integration method is proposed to improve the depth-estimation accuracy when multiple nonfrontal-view face images are available. An important advantage of the proposed algorithm is that no frontal-view face image is required for the estimation of the corresponding 3-D face structure. Experimental results on a real 3-D face image database demonstrate the feasibility and efficiency of the proposed method.

The application of 3D representations in face recognition

Most current psychological theories of face recognition suggest that faces are stored as multiple 2D views. This research aims to explore the application of 3D face representations by means of a new paradigm. Participants were required to match frontal views of faces to silhouettes of the same faces. The formats of the face stimuli were modified in different experiments to make 3D representations accessible (Experiments 1 and 2) or inaccessible (Experiment 3). Multiple 2D view-based algorithms were not applicable due to the singularity of the frontal-view faces. The results disclosed the application and adaptability of 3D face representations. Participants can readily solve the tasks when the face images retain the information essential for the formation of a 3D face representations. However, the performance substantially declined when the 3D information in faces was eliminated (Experiment 3). Performance also varied between different face orientations and different participant groups.

Local binary patterns for multi-view facial expression recognition

Research into facial expression recognition has predominantly been applied to face images at frontal view only. Some attempts have been made to produce pose invariant facial expression classifiers. However, most of these attempts have only considered yaw variations of up to 45°, where all of the face is visible. Little work has been carried out to investigate the intrinsic potential of different poses for facial expression recognition. This is largely due to the databases available, which typically capture frontal view face images only. Recent databases, BU3DFE and multi-pie, allows empirical investigation of facial expression recognition for different viewing angles. A sequential 2 stage approach is taken for pose classification and view dependent facial expression classification to investigate the effects of yaw variations from frontal to profile views. Local binary patterns ( LBPs) and variations of LBPs as texture descriptors are investigated. Such features allow investigation of the influence of orientation and multi-resolution analysis for multi-view facial expression recognition. The influence of pose on different facial expressions is investigated. Others factors are investigated including resolution and construction of global and local feature vectors. An appearance based approach is adopted by dividing images into sub-blocks coarsely aligned over the face. Feature vectors contain concatenated feature histograms built from each sub-block. Multi-class support vector machines are adopted to learn pose and pose dependent facial expression classifiers.

Component Localization in Face Alignment

Face alignment is a significant problem in the processing of face image, and Active Shape Model (ASM) is a popular technology for this problem. However, the initiation of the alignment strongly affects the performance of ASM. If the initiation of alignment is bad, the iteration of ASM optimization will be stuck in a local minima, and the alignment will fail. In this paper, we propose a novel approach to improve ASM by building the classifiers of the face components. We design the SVM classifiers for eyes, mouth and nose, and we use Speeded Up Robust Features(SURF) and Local Binary Pattern(LBP) feature to describe the components which are discriminative for the components than Haar-like features. The face components are firstly located by the classifiers and they indicate the initiation of the alignment. Our approach can make the iterations of ASM optimization converge fast and with the less errors. We evaluate our approach on the frontal views of upright faces of IMM dataset. The experimental results have shown that our approach outperforms the original ASM in terms of efficiency and accuracy.

Visual Object Categorization in the Brain: What Can We Really Learn from ERP Peaks?

In three experiments, Dering et al. (2011) measured the amplitude and latency of the mean P1 and N170 in response to faces, cars, and butterflies, cropped or morphed. The N170 was sensitive to cropping but did not differentiate frontal views of faces and cars. Throughout, P1 amplitude was larger for faces than objects. The authors concluded that P1, not N170, is a reliable face-sensitive event. Although Dering et al. (2011) and Thierry et al. (2007) correctly identified problems with P1 and N170 ERP interpretations, these same problems are evident in their own research, and, consequently, their results are equally inconclusive. We identify four shortcomings in Dering et al.’s approach that also apply to the literature they contest (e.g., Eimer, 1998; Allison et al., 1999; Carmel and Bentin, 2002; Itier and Taylor, 2004; Rousselet et al., 2004; Rossion and Jacques, 2008): categorical design, uncontrolled task demands, group statistics, and peak measurements. ERPs are worthwhile measurements of visual processing, but only when the shortcomings reviewed below are addressed.

Fusing multiple features for Fourier Mellin-based face recognition with single example image per person

At present there are many methods that could deal well with frontal view face recognition. However, most of them cannot work well when there is only single example image per person. In order to deal with this problem of single example image per person is stored in the system in the real-world application. In this paper, a comparative study of the AFMT, Fourier-AFMT and Taylor-AFMT are identified for face recognition. And then, we present hybrid Fourier-AFMT framework based face recognition for feature extraction. Firstly, both directionality of edges and intensity facial features are extracted and secondly fuse two kinds of features and classify with correlation coefficient method (CCM). Experiments are implemented on YALE and ORL face databases to demonstrate the efficient of proposed methods. The experimental results show that the average recognition accuracy rates of our proposed fuse multiple feature domains much higher than that of single feature domain.

Face recognition using combined multiple feature extraction based on Fourier-Mellin approach for single example image per person

At present there are many methods that could deal well with frontal view face recognition. However, most of them cannot work well when there is only single example image per person. In order to deal with this problem of single example image per person is stored in the system in the real-world application. In this paper, we present a combined multiple features extraction based on Fourier-Mellin approach for face recognition with single example per person. The performance of Fourier-AFMT approach extracted frequency invariant features and OFMM approach extracted moment invariant features is applied individually, and these two kinds of features are combined and classified with correlation coefficient method (CCM). Experiments are implemented on YALE and ORL face databases to demonstrate the efficient of proposed methods. The experimental results show that the average recognition accuracy rate of our proposed methods higher than that of state-of-the-art methods.

Anchoring gaze when categorizing faces’ sex: Evidence from eye-tracking data

Previous research has shown that during recognition of frontal views of faces, the preferred landing positions of eye fixations are either on the nose or the eye region. Can these findings generalize to other facial views and a simpler perceptual task? An eye-tracking experiment investigated categorization of the sex of faces seen in four views. The results revealed a strategy, preferred in all views, which consisted of focusing gaze within an ‘infraorbital region’ of the face. This region was fixated more in the first than in subsequent fixations. Males anchored gaze lower and more centrally than females.

Correlates of facial expressions in the primary visual cortex

Face detection and recognition should be complemented by recognition of facial expression, for example for social robots which must react to human emotions. Our framework is based on two multi-scale representations in cortical area V1: keypoints at eyes, nose and mouth are grouped for face detection [1]; lines and edges provide information for face recognition [2]. We assume that keypoints play a key role in the where system, lines and edges being exploited in the what system. This dichotomy, together with coarse-to-fine-scale processing, yields translation and rotation invariance, refining object categorisations until recognition, assuming that objects are represented by normalised templates in memory. Faces are processed the following way: (1) Keypoints at coarse scales are used to translate and rotate the entire input face, using a generic face template with neutral expression. (2) At medium scales, cells with dendritic fields at corners of mouth and eyebrows of the generic template collect evidence for expressions using the line and edge information of the (globally normalised) input face at those scales. Big structures, including mouth and eyebrows, are further normalised using keypoints and first categorizations (gender, race) are obtained using lines and edges. (3) The latter process continues until the finest scale, with normalisation of the expression to neutral for final face recognition. The advantage of this framework is that only one frontal view of a person's face with neutral expression must be stored in memory. This procedure resulted from an analysis of the multi-scale line/edge representation of normalised faces with seven expressions: neutral, anger, disgust, fear, happy, sad and surprise. Following [3], where Action Units (AUs) are related to facial muscles, we analysed the line/edge representation in all AUs. We found that positions of lines and edges at one medium scale, and only at AUs covering the mouth and eyebrows, relative to positions in the neutral face at the same scale, suffice to extract the right expression. Moreover, by implementing AUs by means of six groups of vertically aligned summation cells with a dendritic field size related to that scale (sizes of simple and complex cells), covering a range of positions above and below the corners of mouth and eyebrows in the neutral face, the summation cell with maximum response of each of the six cell groups can be detected, and it is possible to estimate the degree of the expression, from mild to extreme. This work is in progress, since the method must still be tested using big databases with many faces and their natural variations. Perhaps some expressions detected at one medium scale must be validated at one or more finer scales. Nevertheless, in this framework detection of expression occurs before face recognition, which may be an advantage in the development of social robots.

Explicit versus implicit gaze processing assessed by ERPs

Gaze processing was investigated using event-related potentials in two different tasks in which front-view and 3/4-view faces were presented, with eyes gazing straight ahead or averted. Task alternated between an explicit gaze direction judgment and a judgment on head orientation where gaze was irrelevant. Accuracy and reaction times were affected by the congruency of gaze and head directions in both tasks suggesting gaze was processed implicitly in the head orientation task. In both tasks, larger P1 and N170 were found for 3/4-view faces compared to front-view faces that were not due to the luminance or contrast of the pictures. The N170 was also larger for averted than straight gaze but for front-view faces only. In contrast, larger amplitudes for straight than averted gaze were reliably measured around 400–600 ms regardless of head orientation or task demands, and likely reflected the outcome of gaze processing. The results suggest that head orientation and gaze direction discrimination occur regardless of task demands and interact at the decision making level. Neural markers of head orientation occur before those for gaze direction and the early structural encoding stages of face processing are view-dependant.

3D model-based pose invariant face recognition from multiple views

A 3D model-based pose invariant face recognition method that can recognise a human face from its multiple views is proposed. First, pose estimation and 3D face model adaptation are achieved by means of a three-layer linear iterative process. Frontal view face images are synthesised using the estimated 3D models and poses. Then the discriminant ‘waveletfaces’ are extracted from these synthesised frontal view images. Finally, corresponding nearest feature space classifier is implemented. Experimental results show that the proposed method can recognise faces under variable poses with good accuracy.

Multiple primary tumors associated with chromosome 9p deletion

Multiple primary tumors associated with chromosome 9p deletion

Face Configuration Processing in the Human Brain: The Role of Symmetry

Symmetry is an important cue in face perception. We manipulated symmetry and other configurational variables to study their role in face processing in the human brain. We employed 2 types of symmetry: image symmetry (where one part of the image is defined as the mirrored transform of the other part about an axis) and object symmetry (where the spatial relationships among the image components are interpreted as parts of a symmetric 3-dimensional object). We compared blood oxygenation level dependent responses in healthy human observers for upright front-view faces with responses to different symmetry-controlled images. The cortical areas activated by the face images, relative to Fourier-matched scrambled images, were the fusiform (FFA) and occipital (OFA) face areas, the middle occipital gyri (MOG), and areas around the superior temporal and intraoccipital sulci (IOS). Contrasting faces and their image-symmetric scrambled versions showed a similar activation pattern except in the right OFA, suggesting an involvement in facial symmetry processing. The upright versus inverted faces (with the same image symmetry but unfamiliar object identity) showed robust differential activation in the FFA, OFA, MOG, IOS, and precuneus. The response to frontal-view versus 3/4-view faces (having the same object symmetry but disrupted image symmetry) showed little differential activation in the FFA or the OFA but strong responses in the MOG and IOS, suggesting that face processing in the FFA and the OFA is holistic and viewpoint invariant.

Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences

Automatic analysis of human facial expression is a challenging problem with many applications. Most of the existing automated systems for facial expression analysis attempt to recognize a few prototypic emotional expressions, such as anger and happiness. Instead of representing another approach to machine analysis of prototypic facial expressions of emotion, the method presented in this paper attempts to handle a large range of human facial behavior by recognizing facial muscle actions that produce expressions. Virtually all of the existing vision systems for facial muscle action detection deal only with frontal-view face images and cannot handle temporal dynamics of facial actions. In this paper, we present a system for automatic recognition of facial action units (AUs) and their temporal models from long, profile-view face image sequences. We exploit particle filtering to track 15 facial points in an input face-profile sequence, and we introduce facial-action-dynamics recognition from continuous video input using temporal rules. The algorithm performs both automatic segmentation of an input video into facial expressions pictured and recognition of temporal segments (i.e., onset, apex, offset) of 27 AUs occurring alone or in a combination in the input face-profile video. A recognition rate of 87% is achieved.

Development of a view-invariant representation of the human head

Do infants perceive visual cues as diverse as frontal-view faces, profiles or bodies as being different aspects of the same object, a fellow human? If that is the case, visual exposure to one such cue should facilitate the subsequent processing of the others. To verify this hypothesis, we recorded event-related responses in 4-month-old infants and in adults. Pictures of eyes were interleaved amongst images belonging to three human contexts (frontal-view faces, profiles or bodies) or non-human contexts (houses, cars or pliers). In adults, both profile and frontal-face contexts elicited suppression of the N170 response to eye pictures, indicating an access to a view-invariant representation of faces. In infants, a response suppression of the N290 component was recorded only in the context of frontal faces, while profile context induces a different effect (i.e., a P400 enhancement) on eye processing. This dissociation suggests that the view-invariant representation of faces is learned, as it is for other 3-D objects and needs more than 4 months of exposure to be established. In a follow-up study, where infants were exposed to a short movie showing people rotating their heads, the profile-induced P400 effect was speeded up, indicating that exposure to successive views of the same object is probably a way to build up adult-like face representations.

Generating frontal view face image for pose invariant face recognition

Recognizing human faces is one of the most important areas of research in biometrics. However, drastic change of facial poses is a big challenge for its practical application. This paper proposes generating frontal view face image using linear transformation in feature space for face recognition. We extract features from a posed face image using the kernel PCA. Then, we transform the posed face image into its corresponding frontal face image using the transformation matrix predetermined by learning. Then, the generated frontal face image is identified by three different discrimination methods such as LDA, NDA, or GDA. Experimental results show that the recognition rate with the pose transformation outperforms that without pose transformation greatly.

A search advantage for faces learned in motion

Recently there has been growing interest in the role that motion might play in the perception and representation of facial identity. Most studies have considered old/new recognition as a task. However, especially for non-rigid motion, these studies have often produced contradictory results. Here, we used a delayed visual search paradigm to explore how learning is affected by non-rigid facial motion. In the current studies we trained observers on two frontal view faces, one moving non-rigidly, the other a static picture. After a delay, observers were asked to identify the targets in static search arrays containing 2, 4 or 6 faces. On a given trial target and distractor faces could be shown in one of five viewpoints, frontal, 22 degrees or 45 degrees to the left or right. We found that familiarizing observers with dynamic faces led to a constant reaction time advantage across all setsizes and viewpoints compared to static familiarization. This suggests that non-rigid motion affects identity decisions even across extended periods of time and changes in viewpoint. Furthermore, it seems as if such effects may be difficult to observe using more traditional old/new recognition tasks.

Composite Support Vector Machines with Extended Discriminative Features for Accurate Face Detection

This paper describes a pattern classifier for detecting frontal-view faces via learning a decision boundary. The proposed classifier consists of two major parts for improving classification accuracy: the implicit modeling of both the face and the near-face classes resulting in an extended discriminative feature set, and the subsequent composite Support Vector Machines (SVMs) for speeding up the classification. For the extended discriminative feature set, Principal Component Analysis (PCA) or Independent Component Analysis (ICA) is performed for the face and near-face classes separately. The projections and distances to the two different subspaces are complementary, which significantly enhances classification accuracy of SVM. Multiple nonlinear SVMs are trained for the local facial feature spaces considering the general multi-modal characteristic of the face space. Each component SVM has a simpler boundary than that of a single SVM for the whole face space. The most appropriate component SVM is selected by a gating mechanism based on clustering. The classification by utilizing one of the multiple SVMs guarantees good generalization performance and speeds up face detection. The proposed classifier is finally implemented to work in real-time by cascading a boosting based face detector.