Dynamic Face Research Articles

Automated video-based assessment of facial bradykinesia in de-novo Parkinson’s disease

Even though hypomimia is a hallmark of Parkinson’s disease (PD), objective and easily interpretable tools to capture the disruption of spontaneous and deliberate facial movements are lacking. This study aimed to develop a fully automatic video-based hypomimia assessment tool and estimate the prevalence and characteristics of hypomimia in de-novo PD patients with relation to clinical and dopamine transporter imaging markers. For this cross-sectional study, video samples of spontaneous speech were collected from 91 de-novo, drug-naïve PD participants and 75 age and sex-matched healthy controls. Twelve facial markers covering areas of forehead, nose root, eyebrows, eyes, lateral canthal areas, cheeks, mouth, and jaw were used to quantitatively describe facial dynamics. All patients were evaluated using Movement Disorder Society-Unified PD Rating Scale and Dopamine Transporter Single-Photon Emission Computed Tomography. Newly developed automated facial analysis tool enabled high-accuracy discrimination between PD and controls with area under the curve of 0.87. The prevalence of hypomimia in de-novo PD cohort was 57%, mainly associated with dysfunction of mouth and jaw movements, and decreased variability in forehead and nose root wrinkles (p < 0.001). Strongest correlation was found between reduction of lower lip movements and nigro-putaminal dopaminergic loss (r = 0.32, p = 0.002) as well as limb bradykinesia/rigidity scores (r = −0.37 p < 0.001). Hypomimia represents a frequent, early marker of motor impairment in PD that can be robustly assessed via automatic video-based analysis. Our results support an association between striatal dopaminergic deficit and hypomimia in PD.

Midface Multilayering Filler Injection Technique: Understanding of the Dynamic Facial Anatomy Through a "Smiling Cadavers" Anatomical Study.

Understanding age-related changes in compartmentalized facial fat and their role in facial dynamics and aesthetics is essential to target filler injections for midface rejuvenation. A novel anatomical approach ("smiling cadavers") was used to identify the main midface fat compartments involved during muscular contraction when smiling and their motion and behavior with and without filler injections. Based on these insights and the literature, a multilayering filler injection approach was developed to optimize midface rejuvenation by restoring fat volumes using rheologically different products injected into different fat compartments. Twenty-four hemifacial dissections confirmed the presence of two fat compartment layers, separated by the orbicularis oculi muscle in the horizontal plane and by the septa in the vertical plane, and revealed the anatomical effects of facial movement. The midface is composed of deep static fat compartments and a superficial dynamic adipose layer that follows the facial movements, creating a natural dynamic appearance. A proof-of-concept study involved 130 White patients (36 to 56 years; 91 percent women). After the procedure, 95 percent of patients and 98 percent of practitioners rated facial appearance as "improved" or "much/very much improved." No major complications were reported. The smiling cadavers method enhances understanding of dynamic facial anatomy by showing the superficial and deep fat compartments of the midface at rest and their motion during a procedure to represent a smile. The multilayered injection technique takes into account these anatomical findings to rejuvenate the midface, achieving a natural appearance at rest and during motion.

Live speech portraits

To the best of our knowledge, we first present a live system that generates personalized photorealistic talking-head animation only driven by audio signals at over 30 fps. Our system contains three stages. The first stage is a deep neural network that extracts deep audio features along with a manifold projection to project the features to the target person's speech space. In the second stage, we learn facial dynamics and motions from the projected audio features. The predicted motions include head poses and upper body motions, where the former is generated by an autoregressive probabilistic model which models the head pose distribution of the target person. Upper body motions are deduced from head poses. In the final stage, we generate conditional feature maps from previous predictions and send them with a candidate image set to an image-to-image translation network to synthesize photorealistic renderings. Our method generalizes well to wild audio and successfully synthesizes high-fidelity personalized facial details, e.g., wrinkles, teeth. Our method also allows explicit control of head poses. Extensive qualitative and quantitative evaluations, along with user studies, demonstrate the superiority of our method over state-of-the-art techniques.

Shape-invariant encoding of dynamic primate facial expressions in human perception.

Dynamic facial expressions are crucial for communication in primates. Due to the difficulty to control shape and dynamics of facial expressions across species, it is unknown how species-specific facial expressions are perceptually encoded and interact with the representation of facial shape. While popular neural network models predict a joint encoding of facial shape and dynamics, the neuromuscular control of faces evolved more slowly than facial shape, suggesting a separate encoding. To investigate these alternative hypotheses, we developed photo-realistic human and monkey heads that were animated with motion capture data from monkeys and humans. Exact control of expression dynamics was accomplished by a Bayesian machine-learning technique. Consistent with our hypothesis, we found that human observers learned cross-species expressions very quickly, where face dynamics was represented largely independently of facial shape. This result supports the co-evolution of the visual processing and motor control of facial expressions, while it challenges appearance-based neural network theories of dynamic expression recognition.

A Spatio-Temporal Approach for Apathy Classification

Apathy is characterized by symptoms such as reduced emotional response, lack of motivation, and limited social interaction. Current methods for apathy diagnosis require the patient’s presence in a clinic and time consuming clinical interviews, which are costly and inconvenient for both, patients and clinical staff, hindering among other large-scale diagnostics. In this work, we propose a novel spatio-temporal framework for apathy classification, which is streamlined to analyze facial dynamics and emotion in videos. Specifically, we divide the videos into smaller clips, and proceed to extract associated facial dynamics and emotion-based features. Statistical representations/descriptors based on each feature and clip serve as input of the proposed Gated Recurrent Unit (GRU)-architecture. Temporal representations of individual features at the lower level of the proposed architecture are combined at deeper layers of the proposed GRU architecture, in order to obtain the final feature-set for apathy classification. Based on extensive experiments, we show that fusion of characteristics such as emotion and facial dynamics in proposed deep-bi-directional GRU obtains an accuracy of 95.34% in apathy classification.

Self-Supervised Learning of Person-Specific Facial Dynamics for Automatic Personality Recognition

This article aims to solve two important issues that frequently occur in existing automatic personality analysis systems: 1. Attempting to use very short video segments or even single frames, rather than long-term behaviour, to infer personality traits; 2. Lack of methods to encode person-specific facial dynamics for personality recognition. To deal with these issues, this paper first proposes a novel Rank Loss which utilizes the natural temporal evolution of facial actions, rather than personality labels, for self-supervised learning of facial dynamics. Our approach first trains a generic U-net style model that can infer general facial dynamics learned from a set of unlabelled face videos. Then, the generic model is frozen, and a set of intermediate filters are incorporated into this architecture. The self-supervised learning is then resumed with only person-specific videos. This way, the learned filters’ weights are person-specific, making them a valuable source for modeling person-specific facial dynamics. We then propose to concatenate the weights of the learned filters as a person-specific representation, which can be directly used to predict the personality traits without needing other parts of the network. We evaluate the proposed approach on both self-reported personality and apparent personality datasets. In addition to achieving promising results in the estimation of personality trait scores from videos, we show that the tasks conducted by the subject in the video matters, that fusion of a combination of tasks reaches highest accuracy, and that multi-scale dynamics are more informative than single-scale dynamics.

Efficacy and Safety of Flexible Hyaluronic Acid Fillers in Lip and Perioral Enhancement

Background: Since lips have a significant role in facial aesthetic perception, lip augmentation is an increasingly popular aesthetic procedure. Objective: To evaluate aesthetic improvement and facial dynamics with hyaluronic acid (HA) fillers in the lips and perioral region at 8 weeks after the last treatment compared to pre-treatment. Methods and Materials: In this open-label study, all subjects received HARK in the lips, and an additional group also received HARR and/or HARD in nasolabial folds (NLFs) and marionette lines (MLs). Assessments included aesthetic improvement, naturalness of facial expressions, perception of age, and lip texture. Results: Nineteen subjects received HARK only; 40 received HARK and HARR and/or HARD. The primary objective was met. All subjects experienced aesthetic improvement in lip fullness at week 8. The investigators also reported aesthetic improvement in all subjects. For the majority of subjects, aesthetic improvement was associated with maintenance of natural and youthful facial expressions, and improved lip texture. Most treatment-emergent adverse events were mild; none were serious. Conclusion: HARK is a well-tolerated and effective treatment for enhancing lip fullness, maintaining naturalness and youthfulness of facial expressions, and smoothing lip texture, whether used alone or in combination with HARR/ HARD in the NLFs and/or MLs. J Drugs Dermatol. 20(4): 402-408. doi:10.36849/JDD.2021.5525More on lip augmentation.

Advanced Concepts in Rheology for the Evaluation of Hyaluronic Acid-Based Soft Tissue Fillers.

Crosslinked hyaluronic acid (HA)-based soft tissue fillers possess unique viscoelastic properties intended to match specific product indications. Manufacturing has an impact on HA chain integrity and on filler properties. This study introduces 2 new rheological parameters to evaluate the macroscopic characteristics of fillers. A library of reference commercialized HA fillers was selected to cover the full spectrum of product indications. Gels were assessed in terms of size of released HA fragments as a readout of gel integrity, degree of modification, cohesivity, and rheological properties. The elastic modulus G' often used to characterize fillers was shown not to follow macroscopic mechanical properties. To improve the mechanical characterization of fillers, Strength and Stretch scores were developed and tested. The Strength score defined the ability of a filler to sustain constant viscoelasticity over a wide range of constraints and represented the filler mechanical resilience. The Stretch score measured the propensity of a filler to deform in view to improve implant adaptation to facial animation for natural-looking results. Strength and Stretch scores sorted rheological parameters to macroscopic cohesivity assays more accurately than G' and may thus help predict the gel behavior once implanted and submitted to facial dynamics.

Face colorings and chiral face colorings of icosahedral giant fullerenes: C80 to C240

Combinatorial techniques based on Sheehan’s modification of Pόlya’s theorem and Mȍbius inversion technique together with character cycle indices are applied to face colorings of giant fullerenes. These techniques are applied to icosahedral fullerenes, C80 with a chamfered dodecahedron structure, a chiral fullerene C140, icosahedral C180 and C240 with a chamfered truncated icosahedron geometry. The techniques are shown to provide both achiral and chiral face colorings of giant fullerenes for (a) coloring only pentagons with all hexagons painted white, (b) coloring hexagonal faces with all pentagons kept white, (c) both pentagons and hexagons painted with variable colors from a single set, and (d) both pentagons and hexagons painted with variable colors but chosen from two different sets. We have shown that in order to produce chiral colorings for the Ih fullerenes at least 2 black colors and remaining being white are needed when both pentagons and hexagons are varied. Results provide new insights into facial labeling and facial dynamics of fullerenes, while therefore being fully prepared to be inscribed in the new so called neutrosophical science (of carbon) nano-structural physical-chemistry.

Evaluation of Subjective and Objective Levels of Drowsiness, it’s Rating and its Relation with Facial Dynamics

Objective: To determine the subjective and observed levels of lethargy and changes in facial dynamics.&#x0D; Materials and Methods: This study was conducted at Department of community medicine, Services Institute of Medical Sciences Lahore for one-year duration from January 2017 to December 2018. The changes in facial dynamics, such as changes in eyes, lips and eyebrows, were tested in KKS (Karolinska Sleepiness Scale) in twenty-five drivers. And ORD (drowsiness observer rating). Repeated ANOVA measurements and repeated MANOVA measurements were used to analyze the data. In addition, a neural network and Viola-Jones were used to detect facial features. PERCLOS (percentage of eye closure), blink frequency and blink time were examined to see eye parameters. The size of the open mouth during sleep was examined for oral parameters. When examining the eyebrows, the number 50 indicates that the eyebrow is in the normal position. For eyebrows above the normal position, a range of 50 to 55 was specified; In addition, 45-50 was found to be a defined range for normal eyebrows.&#x0D; Results: Descriptive statistics of dynamic changes in the mouth and eyes showed increased drowsiness while driving, as well as changes in the eyes and mouth. However, statistical findings made while driving showed that dynamic eyebrow changes had a clear expression with a continuous trend. Similar studies on data obtained from CSR and ORD showed that both parameters increased at the same time and lethargy level. There was also a significant relationship between facial expressions and lethargy.&#x0D; Conclusion: This study will be an effective and efficient tool for alerting and detecting sleep in a timely and accurate manner.

Reducing facial dynamics' speed during speech enhances attention to mouth in children with autism spectrum disorder: An eye-tracking study.

Facial movements of others during verbal and social interaction are often too rapid to be faced and/or processed in time by numerous children and adults with autism spectrum disorder (ASD), which could contribute to their face-to-face interaction peculiarities. We wish here to measure the effect of reducing the speed of one's facial dynamics on the visual exploration of the face by children with ASD. Twenty-three children with ASD and 29 typically-developing control children matched for chronological age passively viewed a video of a speaker telling a story at various velocities, i.e., a real-time speed and two slowed-down speeds. The visual scene was divided into four areas of interest (AOI): face, mouth, eyes, and outside the face. With an eye-tracking system, we measured the percentage of total fixation duration per AOI and the number and mean duration of the visual fixations made on each AOI. In children with ASD, the mean duration of visual fixations on the mouth region, which correlated with their verbal level, increased at slowed-down velocity compared with the real-time one, a finding which parallels a result also found in the control children. These findings strengthen the therapeutic potential of slowness for enhancing verbal and language abilities in children with ASD.

Trustworthy Method for Person Identification in IIoT Environments by Means of Facial Dynamics

In industrial Internet of Things (IIoT) environments, dependability of a complex manufacturing process in which human operators play a key role can be improved by identity recognition/authentication of whoever is involved in various stages of a production process, according to where and when he/she is supposed to be. To this aim, we propose an approach that exploits the dynamic appearance and the time-dependent local features characterizing the face of an individual during speech utterance with regard to their spatial and temporal components. The proposed method models these dynamic facial patterns captured from edge Internet of Things devices by means of the Local Binary Pattern on Three Orthogonal Planes descriptor, which effectively extract both face's local features and movement at the fog level of the architecture. A deep feedforward network available in the cloud is trained and optimized to match the extracted features to a reference database. The achieved results highlight state-of-the-art performances of the proposed method with regard to robustness and trustworthiness of identification, especially for challenging IIoT scenarios.

Face space representations of movement

The challenging computational problem of perceiving dynamic faces “in the wild” goes unresolved because most research focuses on easier questions about static photograph perception. This literature conceptualizes face representation as a dissimilarity-based “face space”, with axes that describe the dimensions of static images. Some versions express positions in face space relative to a central tendency (norm). Are facial movements represented like this? We tested for representations that accord with an a priori hypothesized motion-based face space by experimentally manipulating faces’ motion-based dissimilarity. Because we caricatured movements, we could test for representations of dissimilarity from a motion-based norm. Behaviorally, participants perceived these caricatured expressions as convincing and recognizable. Moreover, as expected, caricature enhanced perceived dissimilarity between facial expressions. Functional magnetic resonance imaging showed that occipitotemporal brain responses, including face-selective and motion-sensitive areas, reflect this face space. This evidence converged across methods including analysis of univariate mean responses (which additionally exhibited norm-based responses), repetition suppression and representational similarity analysis. This accumulated evidence for “representational geometry” shows how perception and visual brain responses to facial dynamics reflect representations of movement-based dissimilarity spaces, including explicit computation of distance from a norm movement.

Emerging bimodal biometrics authentication for non-venue-based assessments in open distance e-learning (OdeL) environments

Authentication plays a significant role in security during the non-venue-based examination, as it verifies the identity of online students. Existing authentication techniques have not yet provided an optimal cheating-free, non-venue-based assessment. This research proposes an emerging biometrics technology model of facial recognition and keystroke dynamics (FRAKD) to minimise the problems of malpractice during examinations significantly. Detailed experiments conducted on the Moodle learning management system (LMS), using a client-server platform of 100 diverse students who were captured as a positive data set, produced an average recognition rate (RR) of 85%, while for 20 imposter students, an average rejection rate of 90% was realised. When benchmarked with related solutions, this result shows that using the FRAKD biometrics model in non-venue-based assessment demonstrates reliable security and minimises impersonation more than state-of-the-art practices such as portfolio-based distance assessment without supervision. This improved solution will obviously have great value for ODeL institutions and examination bodies.

Facial Expressions of Comprehension (FEC)

While the relationship between facial expressions and emotion has been a productive area of inquiry, research is only recently exploring whether a link exists between facial expressions and cognitive processes. Using findings from psychology and neuroscience to guide predictions of affectation during a cognitive task, this article aimed to study facial dynamics as a mean to understand comprehension. We present a new multimodal facial expression database, named Facial Expressions of Comprehension (FEC), consisting of the videos recorded during a computer-mediated task in which each trial consisted of reading, answering, and feedback to general knowledge true and false statements. To identify the level of engagement with the corresponding stimuli, we present a new methodology using animation units (AnUs) from the Kinect v2 device to explore the changes in facial configuration caused by an event: Event-Related Intensities (ERIs). To identify dynamic facial configurations, we used ERIs in statistical analyses with generalized additive models. To identify differential facial dynamics linked to knowing vs. guessing and true vs. false responses, we employed an SVM classifier with facial appearance information extracted using LPQ-TOP. Results of ERIs in sentence comprehension show that facial dynamics are promising to help understand affective and cognitive states of the mind.

Attended End-to-end Architecture for Age Estimation from Facial Expression Videos.

The main challenges of age estimation from facial expression videos lie not only in the modeling of the static facial appearance, but also in the capturing of the temporal facial dynamics. Traditional techniques to this problem focus on constructing handcrafted features to explore the discriminative information contained in facial appearance and dynamics separately. This relies on sophisticated feature-refinement and framework-design. In this paper, we present an end-toend architecture for age estimation, called Spatially-Indexed Attention Model (SIAM), which is able to simultaneously learn both the appearance and dynamics of age from raw videos of facial expressions. Specifically, we employ convolutional neural networks to extract effective latent appearance representations and feed them into recurrent networks to model the temporal dynamics. More importantly, we propose to leverage attention models for salience detection in both the spatial domain for each single image and the temporal domain for the whole video as well. We design a specific spatially-indexed attention mechanism among the convolutional layers to extract the salient facial regions in each individual image, and a temporal attention layer to assign attention weights to each frame. This two-pronged approach not only improves the performance by allowing the model to focus on informative frames and facial areas, but it also offers an interpretable correspondence between the spatial facial regions as well as temporal frames, and the task of age estimation. We demonstrate the strong performance of our model in experiments on a large, gender-balanced database with 400 subjects with ages spanning from 8 to 76 years. Experiments reveal that our model exhibits significant superiority over the state-of-the-art methods given sufficient training data.

UIN: STUDI ISLAM DAN ARAH BARU ISLAM INDONESIA

Islam as a religion has actually become the target of study not only by Muslims but also by non-Muslims for diverse purposes and interests with various approaches. Indonesian Islamic Studies conducted by Dutch scholars and Orientalists are mostly intended for political and policy purposes related to the fate of Muslim-majority male citizens. Islamic dynamics revealed more progressive dynamics in the 1960s and 70s when institutionally the Ministry of Religion of the Republic of Indonesia released the State Islamic Religious College (PTKIN) with the form of an Institute that developed religious scientific disciplines in the form of various faculties. From this, a variant of State Islamic University (UIN) developed with the development of study areas outside the religious. The fundamental problem that is the focus of this research relates to PTKIN's Islamic facial dynamics in the context of changes to the State Islamic University (UIN), especially Islamic studies and the new direction of Indonesian Islam which is becoming a new force at the State Islamic University (UIN). Specifically, this study aims to understand the trends and challenges of Islam in PTKIN in the future, and the Islamic thinking map of PTKIN in the new face of UIN which includes UIN SGD Bandung, UIN SUKA Yogyakarta and UIN Ar-Raniry Aceh.

The Expression of Ego Depletion

Abstract. Is depleted self-control capacity detectable from thin slices of behavior? In four video-conditions (early-depleted; early-non-depleted; late-depleted; late-non-depleted) untrained observers rated target persons’ available self-control capacity in Experiments 1 and 2 (without sound) and in terms of fatigue and negative affect (Experiment 3). Videos were analyzed with Noldus FaceReader software (Experiment 4). Ego depletion could reliably be detected from thin slices of behavior. Nonverbal expressions coinciding with ego depletion are associated with cues signaling momentary levels of self-control capacity, fatigue, and some negative affective states. FaceReader analyses indicated that facial expressions coinciding with ego depletion are subtle and are extractable from facial dynamics rather than expression intensities. Results indicate that self-control depletion might not only have intrapersonal effects, but also interpersonal effects.

Attended Relation Feature Representation of Facial Dynamics for Facial Authentication

In psychology, it is known that facial dynamics benefit the perception of identity. This paper proposes a novel deep network framework to capture identity information from facial dynamics and their relations. In the proposed method, facial dynamics occurred from a smile expression are analyzed and utilized for facial authentication. Detailed changes in the local regions of a face such as wrinkles and dimples are encoded in the facial dynamic feature representation. The latent relationships of the facial dynamic features are learned by the facial dynamic relational network. In the facial dynamic relational network, the relation features of the facial dynamic are encoded and the relational importance is encoded based on the relation features. As a result, the proposed method has more attention on the important relation features in facial authentication. Through comprehensive and comparative experiments, the effectiveness of the proposed method has been verified in facial authentication.

Emotion in motion: Facial dynamics affect infants' neural processing of emotions.

Research investigating the early development of emotional processing has focused mainly on infants' perception of static facial emotional expressions, likely restricting the amount and type of information available to infants. In particular, the question of whether dynamic information in emotional facial expressions modulates infants' neural responses has been rarely investigated. The present study aimed to fill this gap by recording 7-month-olds' event-related potentials to static (Study 1) and dynamic (Study 2) happy, angry, and neutral faces. In Study 1, happy faces evoked a faster right-lateralized negative central (Nc) component compared to angry faces. In Study 2, both happy and angry faces elicited a larger right-lateralized Nc compared to neutral faces. Irrespective of stimulus dynamicity, a larger P400 to angry faces was associated with higher scores on the Negative Affect temperamental dimension. Overall, results suggest that 7-month-olds are sensitive to facial dynamics, which might play a role in shaping the neural processing of facial emotional expressions. Results also suggest that the amount of attentional resources infants allocate to angry expressions is associated to their temperamental traits. These findings represent a promising avenue for future studies exploring the neurobiological processes involved in perceiving emotional expressions using dynamic stimuli.