Facial Appearance Model Research Articles

Eyewitnesses’ Visual Recollection in Suspect Identification by using Facial Appearance Model

Facial recognition has been an active field of imaging science. With the recent progresses in computer vision development, it is extensively applied in various areas, especially in law enforcement and security. Human face is a viable biometric that could be effectively used in both identification and verification. Thus far, regardless of a facial model and relevant metrics employed, its main shortcoming is that it requires a facial image, against which comparison is made. Therefore, closed circuit televisions and a facial database are always needed in an operational system. For the last few decades, unfortunately, we have experienced an emergence of asymmetric warfare, where acts of terrorism are often committed in secluded area with no camera installed and possibly by persons whose photos have never been kept in any official database prior to the event. During subsequent investigations, the authorities thus had to rely on traumatized and frustrated witnesses, whose testimonial accounts regarding suspect’s appearance are dubious and often misleading. To address this issue, this paper presents an application of a statistical appearance model of human face in assisting suspect identification based on witness’s visual recollection. An online prototype system was implemented to demonstrate its core functionalities. Both visual and numerical assessments reported herein evidentially indicated potential benefits of the system for the intended purpose.

Practical dynamic facial appearance modeling and acquisition

We present a method to acquire dynamic properties of facial skin appearance, including dynamic diffuse albedo encoding blood flow, dynamic specular intensity, and per-frame high resolution normal maps for a facial performance sequence. The method reconstructs these maps from a purely passive multi-camera setup, without the need for polarization or requiring temporally multiplexed illumination. Hence, it is very well suited for integration with existing passive systems for facial performance capture. To solve this seemingly underconstrained problem, we demonstrate that albedo dynamics during a facial performance can be modeled as a combination of: (1) a static, high-resolution base albedo map, modeling full skin pigmentation; and (2) a dynamic, one-dimensional component in the CIE L*a*b* color space, which explains changes in hemoglobin concentration due to blood flow. We leverage this albedo subspace and additional constraints on appearance and surface geometry to also estimate specular reflection parameters and resolve high-resolution normal maps with unprecedented detail in a passive capture system. These constraints are built into an inverse rendering framework that minimizes the difference of the rendered face to the captured images, incorporating constraints from multiple views for every texel on the face. The presented method is the first system capable of capturing high-quality dynamic appearance maps at full resolution and video framerates, providing a major step forward in the area of facial appearance acquisition.

Measuring social variables in real and artificial faces

Human observers rapidly make social judgments about faces, including estimates of trustworthiness, aggression, and many other personality states and traits. Are social variables like these estimated effectively from artificial faces? If not, this could easily compromise interactions between humans and social agents. We examined how a set of personality variables were estimated from real face images and artificial face images synthesized from the original photographs using a morphable model of facial appearance. This stimulus set provided us with an identity-matched set of real and artificial agents, allowing for a close comparison of how social variables are estimated as a function of animacy. We found that observers’ ratings of social variables in both cases were well-described using a two-factor model of social “face space” based on valence and dominance, but real vs. artificial scores were only correlated along the valence axis. Further, we found that correlations between real and synthetic face social variables depended critically on sex categories, suggesting that social evaluation in artificial agents depends on male/female appearance.

Hybrid Ageing Patterns for face age estimation

Wrinkles can be embedded in several image-based applications as a descriptor for human skin. However, wrinkle-based age estimation research has not been widely addressed. In this paper, we introduce a Multi-scale Wrinkle Patterns (MWP) representation, investigate the effect of wrinkles on face age estimation and propose Hybrid Ageing Patterns (HAP) for face age estimation. To define the wrinkle regions more precisely, a template consisting of 10 regions constructed relatively to a set of automatically located facial landmarks is used. We extract the multi-scale wrinkles in each region and encode them into MWP. We use Support Vector Regression to estimate age from the combination of such patterns. The performance of the algorithms is assessed by using Mean Absolute Error (MAE) on three state-of-the-art datasets - FG-NET, FERET and MORPH. We observe that MWP produces a comparable MAE of 4.16 on FERET to the state of the art. Finally we propose HAP, which combines the features from MWP and the facial appearance model (FAM), and demonstrate improved performance on FERET and MORPH with MAE of 3.02 (±2.92) and 3.68 (±2.98), respectively. Therefore, we conclude that MWP is an important complementary feature for face age estimation.

Query-Driven Approach to Face Clustering and Tagging.

In the era of big data, a traditional offline setting to processing image data is simply not tenable. We simply do not have the computational power to process every image with every possible tag; moreover, we will not have the manpower to clean up the potentially noisy results. In this paper, we introduce a query-driven approach to visual tagging, focusing on the application of face tagging and clustering. We integrate active learning with query-driven probabilistic databases. Rather than asking a user to provide manual labels so as to minimize the uncertainty of labels (face tags) across the entire data set, we ask the user to provide labels that minimize the uncertainty of his/her query result (e.g., "How many times did Bob and Jim appear together?"). We use a data-driven Gaussian process model of facial appearance to write the probabilistic estimates of facial identity into a probabilistic database, which can then support inference through query answering. Importantly, the database is augmented with contextual constraints (faces in the same image cannot be the same identity, while faces in the same track must be identical). Experiments on the real-world photo collections demonstrate the effectiveness of the proposed method.

Reshaping 3D facial scans for facial appearance modeling and 3D facial expression analysis

3D face scans have been widely used for face modeling and analysis. Due to the fact that face scans provide variable point clouds across frames, they may not capture complete facial data or miss point-to-point correspondences across various facial scans, thus causing difficulties to use such data for analysis. This paper presents an efficient approach to representing facial shapes from face scans through the reconstruction of face models based on regional information and a generic model. A new approach for 3D feature detection and a hybrid approach using two vertex mapping algorithms, displacement mapping and point-to-surface mapping, and a regional blending algorithm are proposed to reconstruct the facial surface detail. The resulting models can represent individual facial shapes consistently and adaptively, establishing facial point correspondences across individual models. The accuracy of the generated models is evaluated quantitatively. The applicability of the models is validated through the application of 3D facial expression recognition using the static 3DFE and dynamic 4DFE databases. A comparison with the state of the art has also been reported.

A comparative study of facial appearance modeling methods for active appearance models

Active appearance models (AAMs) have been widely used in many face modeling and facial feature extraction methods. One of the problems of AAMs is that it is difficult to model a sufficiently wide range of human facial appearances, the pattern of intensities across a face image patch. Previous researches have used principal component analysis (PCA) for facial appearance modeling, but there has been little analysis and comparison between PCA and many other facial appearance modeling methods such as non-negative matrix factorization (NMF), local NMF (LNMF), and non-smooth NMF (ns-NMF). The main contribution of this paper is to find a suitable facial appearance modeling method for AAMs by a comparative study. In the experiments, PCA, NMF, LNMF, and ns-NMF were used to produce the appearance model of the AAMs and the root mean square (RMS) errors of the detected feature points were analyzed using the AR and BERC face databases. Experimental results showed that (1) if the appearance variations of testing face images were relatively non-sparser than those of training face images, the non-sparse methods (PCA, NMF) based AAMs outperformed the sparse methods (nsNMF, LNMF) based AAMs. (2) If the appearance variations of testing face images are relatively sparser than those of training face images, the sparse methods (nsNMF) based AAMs outperformed the non-sparse methods (PCA, NMF) based AAMs.

Predict Your Child: a System to Suggest the Facial Appearance of Children

We have developed a novel software program called 'Predict Your Child' that, given photographs of potential parent faces, generates plausible looking children. The parent photographs are imported into a PCA-based model of facial appearance to give a set of face parameters that can be mixed together to produce offspring faces. The program is intended for entertainment and has been used commercially, with customers emailing photographs of parents to be bred. In this paper, we describe the system, developed from an evolutionary facial composite system called EvoFIT, outline some of the problems encountered and present some performance data.

On Appearance Based Face and Facial Action Tracking

In this work, we address the problem of tracking faces and facial actions in a single video sequence. The main contributions of the paper are as follows. First, we develop a particle filter based framework for tracking the global 3-D motion of a face using a statistical facial appearance model. Second, we propose a framework for tracking the 3-D face pose as well as the local motion of inner features of the face due for instance to spontaneous facial actions, using an adaptive appearance model. We allow the statistics of the facial appearance as well as the dynamics to be adaptively updated during tracking. Third, we propose a variant of the second framework based on a heuristic search. Tracking real video sequences demonstrated the effectiveness of the developed methods. Accurate tracking was obtained even in the presence of perturbing factors including significant head pose and facial expression variations, occlusions, and illumination changes

Nonlinear, near photo-realistic caricatures using a parametric facial appearance model

A mathematical model previously developed for use in computer vision applications is presented as an empirical model for face space. The term appearance space is used to distinguish this from previous models. Appearance space is a linear vector space that is dimensionally optimal, enables us to model and describe any human facial appearance, and possesses characteristics that are plausible for the representation of psychological face space. Randomly sampling from a multivariate distribution for a location in appearance space produces entirely plausible faces, and manipulation of a small set of defining parameters enables the automatic generation of photo-realistic caricatures. The appearance space model leads us to the new concept of nonlinear caricatures, and we show that the accepted linear method for caricature is only a special case of a more general paradigm. Nonlinear methods are also viable, and we present examples of photographic quality caricatures, using a number of different transformation functions. Results of a simple experiment are presented that suggest that nonlinear transformations can accurately capture key aspects of the caricature effect. Finally, we discuss the relationship between appearance space, caricature, and facial distinctiveness. On the basis of our new theoretical framework, we suggest an experimental approach that can yield new evidence for the plausibility of face space and its ability to explain processes of recognition.

Reactive memories: an interactive talking-head

We demonstrate a novel method for producing a synthetic talking-head. The method is based on earlier work in which the behaviour of a synthetic individual is generated by reference to a probabilistic model of interactive behaviour within the visual domain—such models are learnt automatically from typical interactions. We extend this work into a combined visual and auditory domain and employ a state-of-the-art facial appearance model. The result is a real-time synthetic talking-head that responds appropriately and with realistic timing to simple forms of greeting.

Human facial beauty: current theories and methodologies.

This article examines current theories of beauty and describes recent progress in the ability to generate photorealistic faces using a computer. First, we describe a novel experimental tool, FacePrints, that allows a user to "evolve" an attractive face using a computer. We discuss the use of this program for research on human beauty and review the main experimental studies that have led to our current theoretical perspective: beauty is a product of sexual selection. Second, we outline major improvements to the FacePrints program and demonstrate the near photographic quality of facial composites that can be obtained by combining the FacePrints algorithm with a principal components analysis-based facial appearance model. The technical basis for a possible computer-planning system that could help the patient and surgeon define reasonable and desirable surgical outcomes is also outlined. Finally, we summarize the current state of the art and examine the issues that need to be addressed for developing the current program into a practical experimental and/or clinical tool.

Automatic interpretation and coding of face images using flexible models

Face images are difficult to interpret because they are highly variable. Sources of variability include individual appearance, 3D pose, facial expression, and lighting. We describe a compact parametrized model of facial appearance which takes into account all these sources of variability. The model represents both shape and gray-level appearance, and is created by performing a statistical analysis over a training set of face images. A robust multiresolution search algorithm is used to fit the model to faces in new images. This allows the main facial features to be located, and a set of shape, and gray-level appearance parameters to be recovered. A good approximation to a given face can be reconstructed using less than 100 of these parameters. This representation can be used for tasks such as image coding, person identification, 3D pose recovery, gender recognition, and expression recognition. Experimental results are presented for a database of 690 face images obtained under widely varying conditions of 3D pose, lighting, and facial expression. The system performs well on all the tasks listed above.