Dynamic Face Research Articles

3D Visual passcode: Speech-driven 3D facial dynamics for behaviometrics

Face biometrics have achieved remarkable performance over the past decades, but unexpected spoofing of the static faces poses a threat to information security. There is an increasing demand for stable and discriminative biological modalities which are hard to be mimicked and deceived. Speech-driven 3D facial motion is a distinctive and measurable behavior-signature that is promising for biometrics. In this paper, we propose a novel 3D behaviometrics framework based on a “3D visual passcode” derived from speech-driven 3D facial dynamics. The 3D facial dynamics are jointly represented by 3D-keypoint-based measurements and 3D shape patch features, extracted from both static and speech-driven dynamic regions. An ensemble of subject-specific classifiers are then trained over selected discriminative features, which allows for a discriminant speech-driven 3D facial dynamics representation. We construct the first publicly available Speech-driven 3D Facial Motion dataset (S3DFM) that includes 2D-3D face video plus audio samples from 77 participants. The experimental results on the S3DFM show that the proposed pipeline achieves a face identification rate of 96.1%. Detailed discussions are presented, concerning anti-spoofing, head pose variation, video frame rate, and applicability cases. We also give comparison with other baselines on “deep” and “shallow” 2D face features.

Anatomical study of the zygomatic and buccal branches of the facial nerve: Application to facial reanimation procedures.

The facial nerve is responsible for any facial expression channeling human emotions. Facial paralysis causes asymmetry, lagophthalmus, oral incontinence, and social limitations. Facial dynamics may be re-established with cross-face-nerve-grafts (CFNG). Our aim was to reappraise the zygomaticobuccal branch system relevant for facial reanimation surgery with respect to anastomoses and crossings. Dissection was performed on 106 facial halves of 53 fresh frozen cadavers. Study endpoints were quantity and relative thickness of branches, correlation to "Zuker's point", interconnection patterns and crossings. Level I and level II branches were classified as relevant for CFNG. Anastomoses and fusion patterns were assessed in both levels. The zygomatic branch showed 2.98 ± 0.86 (range 2-5) twigs at level II and the buccal branch 3.45 ± 0.96 (range 2-5), respectively. In the zygomatic system a single dominant branch was present in 50%, two co-dominant branches in 9% and three in 1%. In 66% of cases a single dominant buccal twig, two co-dominant in 12.6%, and three in 1% of cases were detected. The most inferior zygomatic branch was the most dominant branch (P = 0.003). Using Zuker's point, a facial nerve branch was found within 5 mm in all facial halves. Fusions were detected in 80% of specimens. Two different types of fusion patterns could be identified. Undercrossing of branches was found in 24% at levels I and II. Our study describes facial nerve branch systems relevant for facial reanimation surgery in a three-dimensional relationship of branches to each other. Clin. Anat. 32:480-488, 2019. © 2019 Wiley Periodicals, Inc.

Addressing facial dynamics using k-medoids cohort selection algorithm for face recognition

Face recognition is itself a very challenging task and it becomes more challenging when the input images have intra class variations and inter class similarities in a large scale. Yet the recognition accuracy can be improved in some extent by supporting the system with non-matched templates. Therefore a set of cohort images is used in this regard. But all the cohort templates of the initial cohort pool may not be relevant for each and every enrolled subject. So the main focus of this work is to select a subject specific and meaningful cohort subset. This paper proposes a cohort selection method called K-medoids Cohort Selection (KMCS) to select a reference set of non-matched templates which are almost appropriate to the respective subjects. Basically, all cohort scores of a subject are clustered first using K-medoids clustering. Afterward the cluster having more scattered members/scores from its medoid is selected as a cohort subset because this cluster is constituted with the cohorts carrying more discriminative features compared to others. The SIFT points and SURF points are extracted as facial feature. The experiments are conducted on FEI, ORL and Look-alike databases of face images. The matching scores between probe and query images are normalized using T-norm, Max-Min and Aggarwal (Max rule) cohort score normalization techniques before taking the final decision of acceptance or rejection. The results obtained from the experiments show the domination of the proposed system over the non-cohort face recognition system as well as random and Top 10 cohort selection methods. There is another comparative study between k-means and K-medoids clustering for cohort selection.

Rheology of hyaluronic acid and dynamic facial rejuvenation: Topographical specificities.

Sound knowledge of the rheology of hyaluronic acids is fundamental for facial rejuvenation procedures. Hyaluronic acid implants are subjected to complex mechanical forces within the facial tissues that vary depending on the anatomical region. Analysis of these forces is important to establish the rheological specifications of hyaluronic acid implants in each specific zone. This approach will optimize the correction of age-related loss of volume, since the filler will be chosen as a function of the mechanics of the anatomical zone, with the objective to respect facial dynamics.

Subjective facial dynamics with the use of hyaluronic acid dermal fillers formulated for facial movement and expression

Subjective facial dynamics with the use of hyaluronic acid dermal fillers formulated for facial movement and expression

The contribution of facial dynamics to subtle expression recognition in typical viewers and developmental visual agnosia

Facial expressions are inherently dynamic cues that develop and change over time, unfolding their affective signal. Although facial dynamics are assumed important for emotion recognition, testing often involves intense and stereotypical expressions and little is known about the role of temporal information in the recognition of subtle, non-stereotypical expressions. In Experiment 1 we demonstrate that facial dynamics are critical for recognizing subtle and non-stereotypical facial expressions, but not for recognizing intense and stereotypical displays of emotion. In Experiment 2 we further examined whether the facilitative effect of motion can lead to improved emotion recognition in LG, an individual with developmental visual agnosia and prosopagnosia, who has poor emotion recognition when tested with static facial expressions. LG's emotion recognition improved when subtle, non-stereotypical faces were dynamic rather than static. However, compared to controls, his relative gain from temporal information was diminished. Furthermore, LG's eye-tracking data demonstrated atypical visual scanning of the dynamic faces, consisting of longer fixations and lower fixation rates for the dynamic-subtle facial expressions, comparing to the dynamic-intense facial expressions. We suggest that deciphering subtle dynamic expressions strongly relies on integrating broad facial regions across time, rather than focusing on local emotional cues, skills which are impaired in developmental visual agnosia.

Abstract: Subjective and Objective Facial Dynamics Using Dermal Fillers Formulated for Facial Movement Adaptation

INTRODUCTION: The fear of looking unnatural is a well-known concern for patients contemplating dermal aesthetic procedures. Facial dynamics is an increasing area of clinical focus extending beyond 3-D volume restoration, as naturalness of clinical outcomes at rest and with animation may vary. We evaluated the dynamic face using subjective and objective methods following treatment with dermal fillers formulated with physicochemical properties for facial movement adaptation. METHODS: Thirty Caucasian females (40–65 years) with moderate to severe, bilateral wrinkles in the lower face were treated with HA fillers (20mg/mL with XpresHAn TechnologyTM) and followed 4 weeks post-optimal correction. Subjective, dynamic assessments evaluated pre- versus post- expressions in motion (2D videos), using a series of standardized expressions. Facial dynamics were objectively evaluated and quantified using 3-D stereophotogrammetry (Canfield Scientific, Inc), including a younger, untreated Caucasian female cohort (N=20; 25–35 years). Satisfaction of treated subjects was assessed using a 5-point Likert scale. RESULTS: Subjective facial dynamics revealed naturalness of the lower face in motion to be at least maintained in 100% of subjects (naturalness maintained or enhanced). Collectively, 83.3% of subjects were rated with enhanced attractiveness and looked younger, without compromise in naturalness. Rater agreement was high for individual assessments of attractiveness, youthfulness, and naturalness (70.0% – 83.3%). Subject satisfaction ratings were consistent with treating investigator assessment, with post-treatment improvement across all items assessed based on proportions of subject agreement (strongly agree or agree). Highest levels of subject satisfaction (>80%) observed post-treatment pertained to overall facial appearance is pleasing (90.0%); overall facial appearance looking natural (100%); face looking natural when relaxed (96.7%) and when smiling (93.3%), and looking younger than actual age (83.3%). For specific anatomic areas (marionette lines), global dynamic assessment using 3-D stereophotogrammetry showed significantly higher levels of stretch in older (20.1%, pre-treatment) versus younger subjects (17.7%; p<=.05), with stretch levels significantly reduced post-treatment (17.9%; p<=.05) such that older subjects post-treatment resembled younger subjects. CONCLUSION: Dermal fillers formulated with XpresHAn TechnologyTM resulted in subjective dynamic assessments characterized by improvements in attractiveness and youthfulness, without compromising naturalness. Objective facial dynamics provided quantitative evidence of stretch levels resembling a younger phenotype, in areas specifically prone to dynamic volumetric effects of facial aging. This work underscores the importance of objective dynamic assessment as the fourth dimension of facial aesthetics.

Contributions of individual face features to face discrimination

Faces are highly complex stimuli that contain a host of information. Such complexity poses the following questions: (a) do observers exhibit preferences for specific information? (b) how does sensitivity to individual face parts compare? These questions were addressed by quantifying sensitivity to different face features. Discrimination thresholds were determined for synthetic faces under the following conditions: (i) ‘full face’: all face features visible; (ii) ‘isolated feature’: single feature presented in isolation; (iii) ‘embedded feature’: all features visible, but only one feature modified. Mean threshold elevations for isolated features, relative to full-faces, were 0.84x, 1.08, 2.12, 3.34, 4.07 and 4.47 for head-shape, hairline, nose, mouth, eyes and eyebrows respectively. Hence, when two full faces can be discriminated at threshold, the difference between the eyes is about four times less than what is required when discriminating between isolated eyes. In all cases, sensitivity was higher when features were presented in isolation than when they were embedded within a face context (threshold elevations of 0.94x, 1.74, 2.67, 2.90, 5.94 and 9.94). This reveals a specific pattern of sensitivity to face information. Observers are between two and four times more sensitive to external than internal features. The pattern for internal features (higher sensitivity for the nose, compared to mouth, eyes and eyebrows) is consistent with lower sensitivity for those parts affected by facial dynamics (e.g. facial expressions). That isolated features are easier to discriminate than embedded features supports a holistic face processing mechanism which impedes extraction of information about individual features from full faces.

Spontaneous versus posed smile recognition via region-specific texture descriptor and geometric facial dynamics

As a typical biometric cue with great diversities, smile is a fairly influential signal in social interaction, which reveals the emotional feeling and inner state of a person. Spontaneous and posed smiles initiated by different brain systems have differences in both morphology and dynamics. Distinguishing the two types of smiles remains challenging as discriminative subtle changes need to be captured, which are also uneasily observed by human eyes. Most previous related works about spontaneous versus posed smile recognition concentrate on extracting geometric features while appearance features are not fully used, leading to the loss of texture information. In this paper, we propose a region-specific texture descriptor to represent local pattern changes of different facial regions and compensate for limitations of geometric features. The temporal phase of each facial region is divided by calculating the intensity of the corresponding facial region rather than the intensity of only the mouth region. A mid-level fusion strategy of support vector machine is employed to combine the two feature types. Experimental results show that both our proposed appearance representation and its combination with geometry-based facial dynamics achieve favorable performances on four baseline databases: BBC, SPOS, MMI, and UvA-NEMO.

Sparsity in Dynamics of Spontaneous Subtle Emotions: Analysis and Application

Subtle emotions are present in diverse real-life situations: in hostile environments, enemies and/or spies maliciouslyconceal their emotions as part of their deception; in life-threatening situations, victims under duress have no choice but to withhold theirreal feelings; in the medical scene, patients with psychological conditions such as depression could either be intentionally orsubconsciously suppressing their anguish from loved ones. Under such circumstances, it is often crucial that these subtle emotions arerecognized before it is too late. These spontaneous subtle emotions are typically expressed through micro-expressions, which are tiny,sudden and short-lived dynamics of facial muscles; thus, such micro-expressions pose a great challenge for visual recognition. Theabrupt but significant dynamics for the recognition task are temporally sparse while the rest, i.e. irrelevant dynamics, are temporallyredundant. In this work, we analyze and enforce sparsity constraints to learn significant temporal and spectral structures whileeliminating irrelevant facial dynamics of micro-expressions, which would ease the challenge in the visual recognition of spontaneoussubtle emotions. The hypothesis is confirmed through experimental results of automatic spontaneous subtle emotion recognition withseveral sparsity levels on CASME II and SMIC, the two well-established and publicly available spontaneous subtle emotion databases.The overall performances of the automatic subtle emotion recognition are boosted when only significant dynamics of the originalsequences are preserved.

Event-related potentials to changes in facial expression in two-phase transitions.

The purpose of the study was to compare event-related potentials (ERPs) to different transitions between emotional and neutral facial expressions. The stimuli contained a single transition between two different images of the same face, giving a strong impression of changing expression though apparent motion whilst eliminating change in irrelevant stimulus variables such as image contrast or identity. Stimuli were calibrated for intensity, valence and perceived emotion category and only trials where the target emotion was correctly identified were included. In the first experiment, a magnification change (zoom) was a control condition. Transitions from neutral to angry expressions produced a more negative N1 with longer peak latency, and more positive P2 than did an increase in magnification. Critically, response to neutral following angry, relative to neutral following magnified, showed a generally more negative ERP with a delayed N1 peak and reduced P2 amplitude. In the second experiment, comparison of neutral-happy and neutral-frightened transitions showed significantly different ERPs to emotional expression change. Responses to the reversed direction of a transition (happy-neutral and frightened-neutral) were much reduced. Unlike the comparison of angry-neutral with magnified-neutral, there were minimal differences in the responses to neutral following happy and neutral following frightened. The results demonstrate in a young adult sample the directionality of responses to facial expression dynamics, and suggest a separation of neural mechanisms for detecting expression changes and magnification changes.

Microexpression Identification and Categorization Using a Facial Dynamics Map

Unlike conventional facial expressions, microexpressions are instantaneous and involuntary reflections of human emotion. Because microexpressions are fleeting, lasting only a few frames within a video sequence, they are difficult to perceive and interpret correctly, and they are highly challenging to identify and categorize automatically. Existing recognition methods are often ineffective at handling subtle face displacements, which can be prevalent in typical microexpression applications due to the constant movements of the individuals being observed. To address this problem, a novel method called the Facial Dynamics Map is proposed to characterize the movements of a microexpression in different granularity. Specifically, an algorithm based on optical flow estimation is used to perform pixel-level alignment for microexpression sequences. Each expression sequence is then divided into spatiotemporal cuboids in the chosen granularity. We also present an iterative optimal strategy to calculate the principal optical flow direction of each cuboid for better representation of the local facial dynamics. With these principal directions, the resulting Facial Dynamics Map can characterize a microexpression sequence. Finally, a classifier is developed to identify the presence of microexpressions and to categorize different types. Experimental results on four benchmark datasets demonstrate higher recognition performance and improved interpretability.

Extraction of texture and geometrical features from informative facial regions for sign language recognition

In general, the most common form of gestures is made up of movements of the hand and/or arm associated with facial expressions. In this, the hand is used to make different message signs, while facial movements are used to reflect the mood and emotion of the person. In this paper, some sign language gestures are recognized only with the help of associated facial expressions. Existing facial expression based sign language recognition (SLR) methods only used facial geometric features to recognize sign language gestures. However, the performance of geometric feature-based SLR methods depends on the accuracy of tracking algorithms and the number of facial landmark points. Additionally, facial textures are more informative as compared to the geometric features of a face. Inspiring from these facts, we propose to recognize sign language gestures with the help of spatio-temporal characteristics of facial texture patterns. For this, a new face model is proposed by extracting texture features only from the informative regions of a face. The proposed face model can also be employed to extract the geometrical features of a face. The features extracted from the informative regions of a face are significantly discriminative, and so the proposed face model can track/encode the facial dynamics of the associated facial expressions of a sign. Finally, a 3-state hidden conditional random field is employed to model the texture variations of facial gestures. Experimental results on RWTH-BOSTON data-set show that proposed method can achieve upto 80.06% recognition rate.

Late fusion of facial dynamics for automatic expression recognition

Installment of a facial expression is associated with contractions and extensions of specific facial muscles. Noting that expression is about changes, we present a model for expression classification based on facial landmarks dynamics. Our model isolates the trajectory of facial fiducial points by wrapping them up in relevant features and discriminating among various alternatives with a machine learning classification system. The used features are geometric and temporal-based and the classification system is represented by a late fusion framework that combines several neural networks with binary responses. The proposed method is robust, being able to handle complex expression classes.

4D Analysis of Facial Ageing Using Dynamic Features

Facial ageing analysis based on 4D data (3D plus time) is much more robust to pose changes and illumination variations than using 2D image and video. The purpose of this investigation was to measure the effects of age and gender related facial changes using dynamic 3D facial scans. Experiments were carried out on the subjects, who were divided into two groups by age (15-30 years and 31-60 years). Each group was further subdivided by gender. 3D scans of the subjects were processed to extract facial features which were tracked through the duration of the data capture. Subsequently, a set of dynamic features were computed from these facial features, as well as static features for comparison. Two-way multivariate analysis of variance (MANOVA) of these features demonstrated that statistically significant age and gender related differences could be detected. We show that 3D facial dynamics provide more useful information than static features for the characterisation of smiles.

Gender Estimation Based on Smile-Dynamics

Automated gender estimation has numerous applications, including video surveillance, human-computer interaction, anonymous customized advertisement, and image retrieval. Most commonly, the underlying algorithms analyze the facial appearance for clues of gender. In this paper, we propose a novel method for gender estimation, which exploits dynamic features gleaned from smiles and we proceed to show that: a) facial dynamics incorporate clues for gender dimorphism and b) while for adult individuals appearance features are more accurate than dynamic features, for subjects under 18 years facial dynamics can outperform appearance features. In addition, we fuse proposed dynamics-based approach with state-of-the-art appearance-based algorithms, predominantly improving performance of the latter. Results show that smile-dynamics include pertinent and complementary to appearance gender information.

Partial Matching of Facial Expression Sequence Using Over-Complete Transition Dictionary for Emotion Recognition

Facial dynamics contain useful information for facial expression recognition (FER). However, exploiting dynamics in FER is challenging. This is mainly due to a variety of expression transitions. For example, video sequences belonging to a same emotion class may have different characteristics in transition duration and/or transition type (e.g., onset versus offset). The temporal mismatches between query and training video sequences could degrade the FER. This paper proposes a new partial matching framework that aims to overcome the temporal mismatch of expression transition. During the training stage, we construct an over-complete transition dictionary where many possible partial expression transitions are contained. During the test stage, we extract a number of partial expression transitions from a query video sequence. Each partial expression transition is analyzed individually. This increases the possibility of matching a partial expression transition in the query video sequence against the partial expression transitions in the over-complete transition dictionary. To make a partial matching subject-independent and robust to the temporal mismatch, each partial expression transition is defined as facial shape displacement between a pair of face clusters. Experimental results show that the proposed method is robust to variations of transition duration and transition type in subject-independent recognition.

A Computational Study of Expressive Facial Dynamics in Children with Autism

Several studies have established that facial expressions of children with autism are often perceived as atypical, awkward or less engaging by typical adult observers. Despite this clear deficit in the quality of facial expression production, very little is understood about its underlying mechanisms and characteristics. This paper takes a computational approach to studying details of facial expressions of children with high functioning autism (HFA). The objective is to uncover those characteristics of facial expressions, notably distinct from those in typically developing children, and which are otherwise difficult to detect by visual inspection. We use motion capture data obtained from subjects with HFA and typically developing subjects while they produced various facial expressions. This data is analyzed to investigate how the overall and local facial dynamics of children with HFA differ from their typically developing peers. Our major observations include reduced complexity in the dynamic facial behavior of the HFA group arising primarily from the eye region.

BlendForces: A Dynamic Framework for Facial Animation

AbstractIn this paper we present a new paradigm for the generation and retargeting of facial animation. Like a vast majority of the approaches that have adressed these topics, our formalism is built on blendshapes. However, where prior works have generally encoded facial geometry using a low dimensional basis of these blendshapes, we propose to encode facial dynamics by looking at blendshapes as a basis of forces rather than a basis of shapes. We develop this idea into a dynamic model that naturally combines the blendshapes paradigm with physics‐based techniques for the simulation of deforming meshes. Because it escapes the linear span of the shape basis through time‐integration and physics‐inspired simulation, this approach has a wider expressive range than previous blendshape‐based methods. Its inherent physically‐based formulation also enables the simulation of more advanced physical interactions, such as collision responses on lip contacts.

Amélioration de l’exploration visuelle d’un visage par des enfants avec autisme grâce au ralentissement de la dynamique faciale : une étude préliminaire en oculométrie

ObjectiveFace and gaze avoidance are among the most characteristic and salient symptoms of autism spectrum disorders (ASD). Studies using eye tracking highlighted early and lifelong ASD-specific abnormalities in attention to face such as decreased attention to internal facial features. These specificities could be partly explained by disorders in the perception and integration of rapid and complex information such as that conveyed by facial movements and more broadly by biological and physical environment. Therefore, we wish to test whether slowing down facial dynamics may improve the way children with ASD attend to a face. MethodsWe used an eye tracking method to examine gaze patterns of children with ASD aged 3 to 8 (n=23) and TD controls (n=29) while viewing the face of a speaker telling a story. The story was divided into 6 sequences that were randomly displayed at 3 different speeds, i.e. a real-time speed (RT), a slow speed (S70=70% of RT speed), a very slow speed (S50=50% of RT speed). S70 and S50 were displayed thanks to software called Logiral™, aimed at slowing down visual and auditory stimuli simultaneously and without tone distortion. The visual scene was divided into four regions of interest (ROI): eyes region; mouth region; whole face region; outside the face region. The total time, number and mean duration of visual fixations on the whole visual scene and the four ROI were measured between and within the two groups. ResultsCompared to TD children, children with ASD spent significantly less time attending to the visual scenes and, when they looked at the scene, they spent less time scanning the speaker's face in general and her mouth in particular, and more time looking outside facial area. Within the ASD group mean duration of fixation increased on the whole scene and particularly on the mouth area, in R50 compared to RT. Children with mild autism spent more time looking at the face than the two other groups of ASD children, and spent more time attending to the face and mouth as well as longer mean duration of visual fixation on mouth and eyes, at slow speeds (S50 and/or S70) than at RT one. ConclusionsSlowing down facial dynamics enhances looking time on face, and particularly on mouth and/or eyes, in a group of 23 children with ASD and particularly in a small subgroup with mild autism. Given the crucial role of reading the eyes for emotional processing and that of lip-reading for language processing, our present result and other converging ones could pave the way for novel socio-emotional and verbal rehabilitation methods for autistic population. Further studies should investigate whether increased attention to face and particularly eyes and mouth is correlated to emotional/social and/or verbal/language improvements.