Imitation Of Facial Expressions Research Articles

A Two-Stage Facial Kinematic Control Strategy for Humanoid Robots Based on Keyframe Detection and Keypoint Cubic Spline Interpolation

A plentiful number of facial expressions is the basis of natural human–robot interaction for high-fidelity humanoid robots. The facial expression imitation of humanoid robots involves the transmission of human facial expression data to servos situated within the robot’s head. These data drive the servos to manipulate the skin, thereby enabling the robot to exhibit various facial expressions. However, since the mechanical transmission rate cannot keep up with the data processing rate, humanoid robots often suffer from jitters in the imitation. We conducted a thorough analysis of the transmitted facial expression sequence data and discovered that they are extremely redundant. Therefore, we designed a two-stage strategy for humanoid robots based on facial keyframe detection and facial keypoint detection to achieve more natural and smooth expression imitation. We first built a facial keyframe detection model based on ResNet-50, combined with optical flow estimation, which can identify key expression frames in the sequence. Then, a facial keypoint detection model is used on the keyframes to obtain the facial keypoint coordinates. Based on the coordinates, the cubic spline interpolation method is used to obtain the motion trajectory parameters of the servos, thus realizing the robust control of the humanoid robot’s facial expression. Experiments show that, unlike before where the robot’s imitation would stutter at frame rates above 25 fps, our strategy allows the robot to maintain good facial expression imitation similarity (cosine similarity of 0.7226), even at higher frame rates.

Voluntary imitation of dynamic facial expressions in attention deficit hyperactivity disorder: a facial-behavior analysis

ABSTRACT Objective The difficulties involved in social interaction among children with attention deficit hyperactivity disorder (ADHD) have been shown in many studies. Based on the knowledge that the imitation of facial expressions is a key factor in social interaction and functionality, the focus of prior studies has been on the evaluation of facial expressions in individuals with ADHD. However, little is known about voluntary facial mimicry in individuals with ADHD. In this context, we aimed to evaluate the voluntary-facial-imitation intensity of dynamic facial expressions in children with ADHD. Method Forty-one children with ADHD and 53 typically developing children were included in the study. Participants were presented with a video of six basic emotions and neutral facial expressions selected from the EU-Emotion Stimulus Set via a screen. After each emotion, the instruction “now imitate it” was given. While the children watched the video, their faces were recorded with a webcam. The intensity of the children’s voluntary facial imitations was examined with a computer vision program (Openface) that performs facial analysis on recorded videos. Results There was no significant difference between the groups in terms of facial emotion recognition accuracy. In group comparisons of voluntary facial mimicry, children with ADHD showed a significantly higher imitation intensity after emotional expressions of sadness, surprise and fear. There was no difference between the groups after the emotions of happiness, anger and disgust. Conclusion This non-obtrusive, noninvasive, and cost-effective method allowed us to measure the quantitative differences in facial mimicry between children with ADHD and typically developing children. Our results contribute new information to the literature by indicating which emotions can be used in the evaluation of social communication skills, as well as intervention targets for these skills, in children with ADHD.

Differential responses in the mirror neuron system during imitation of individual emotional facial expressions and association with autistic traits

The mirror neuron system (MNS), including the inferior frontal gyrus (IFG), inferior parietal lobule (IPL) and superior temporal sulcus (STS) plays an important role in action representation and imitation and may be dysfunctional in autism spectrum disorder (ASD). However, it's not clear how these three regions respond and interact during the imitation of different basic facial expressions and whether the pattern of responses is influenced by autistic traits. Thus, we conducted a natural facial expression (happiness, angry, sadness and fear) imitation task in 100 healthy male subjects where expression intensity was measured using facial emotion recognition software (FaceReader) and MNS responses were recorded using functional near-infrared spectroscopy (fNIRS). Autistic traits were measured using the Autism Spectrum Quotient questionnaire. Results showed that imitation of happy expressions produced the highest expression intensity but a small deactivation in MNS responses, suggesting a lower processing requirement compared to other expressions. A cosine similarity analysis indicated a distinct pattern of MNS responses during imitation of each facial expression with functional intra-hemispheric connectivity between the left IPL and left STS being significantly higher during happy compared to other expressions, while inter-hemispheric connectivity between the left and right IPL differed between imitation of fearful and sad expressions. Furthermore, functional connectivity changes during imitation of each different expression could reliably predict autistic trait scores. Overall, the results provide evidence for distinct patterns of functional connectivity changes between MNS regions during imitation of different emotions which are also associated with autistic traits.

SERIOUS GAMES TO TEACH EMOTION RECOGNITION TO CHILDREN WITH AUTISM SPECTRUM DISORDERS (ASD)

The serious game "JeStiMulE" (Educational Game for Multisensory Stimulation of Children with developmental disorders), developed by the Autism Resources Center of Nice, was created to teach social cognition including emotion recognition for children and adolescents with autism. The purpose of our study is to investigate the effectiveness of the serious game JESTIMULE in remediating recognition' deficits of emotional facial expressions (EFE) in autistic children. Thirty-two Moroccan children and adolescents were recruited for this study. All participants received a diagnosis of autism spectrum disorder (ASD) based on the Diagnostic and Statistical Manual of Mental Diseases, Fourth Edition (DSM-IV-R) criteria for ASD, as well as the Rimland Checklist E2. IQ level has been assessed by using Raven's Progressive Matrices as an IQ testing scale. The participants received two one-hour JeStiMulE sessions per week over four weeks. Game data were collected for each participant. The Faces test was used to quantify the progression of the ability to recognize emotions in our subjects. The results of the descriptive analyses showed suitable adaptability, effectiveness and efficiency of JeStiMulE. In Faces test, a significant difference between scores of the Pre-intervention and Post-intervention (Z= -3.58, p<0.001), in favour of the Post-intervention (M=23.22, SD=2.96 versus M=27.27, SD=2.77). That indicate that participants were more accurate at recognizing emotions after JeStiMulE. In addition, a main effect of type of autism was found for the facial scale (H = 6.673, ddl = 2, p = 0.036). High-functioning autism were significantly better than Low-functioning autism at recognizing emotions from faces in both Pre-intervention and Post-intervention. With such non-verbal tool, training could start early. Thus, early management, the more the progression in the recognition and imitation of facial expressions is important, the more we gain autonomy and social integration just in time to begin school.

Dynamic Causal Modeling for fMRI With Wilson-Cowan-Based Neuronal Equations.

Dynamic causal modeling (DCM) is an analysis technique that has been successfully used to infer about directed connectivity between brain regions based on imaging data such as functional magnetic resonance imaging (fMRI). Most variants of DCM for fMRI rely on a simple bilinear differential equation for neural activation, making it difficult to interpret the results in terms of local neural dynamics. In this work, we introduce a modification to DCM for fMRI by replacing the bilinear equation with a non-linear Wilson-Cowan based equation and use Bayesian Model Comparison (BMC) to show that this modification improves the model evidences. Improved model evidence of the non-linear model is shown for our empirical data (imitation of facial expressions) and validated by synthetic data as well as an empirical test dataset (attention to visual motion) used in previous foundational papers. For our empirical data, we conduct the analysis for a group of 42 healthy participants who performed an imitation task, activating regions putatively containing the human mirror neuron system (MNS). In this regard, we build 540 models as one family for comparing the standard bilinear with the modified Wilson-Cowan models on the family-level. Using this modification, we can interpret the sigmoid transfer function as an averaged f-I curve of many neurons in a single region with a sigmoidal format. In this way, we can make a direct inference from the macroscopic model to detailed microscopic models. The new DCM variant shows superior model evidence on all tested data sets.

Facial Expression Imitation Method for Humanoid Robot Based on Smooth-Constraint Reversed Mechanical Model (SRMM)

To improve the space-time similarity and motion smoothness of facial expression imitation (FEI), a real-time FEI method for a humanoid robot is proposed based on smooth-constraint reversed mechanical model (SRMM) by combining a sequence-to-sequence deep learning model and a motion-smoothing constraint. First, on the basis of facial data from a Kinect capture device, a facial feature vector is characterized based on 3 head postures, 17 facial animation units, and facial geometric deformation cascaded by Laplace coordinates. Second, a reversed mechanical model is constructed via a multilayer long short-term memory neural network to accomplish direct mapping from facial feature sequences to motor position sequences. Additionally, to overcome the motor chattering phenomenon during real-time FEI, a high-order polynomial is constructed to fit the position sequence of motors, and an SRMM is proposed and designed based on the deviation of position, velocity, and acceleration. Finally, aiming to imitate the real-time facial feature sequences of a performer captured from Kinect, the optimal position sequences generated based on the SRMM is sent to the hardware system to keep the space-time characteristics consistent with those of the performer. The experimental results demonstrate that the motor position deviation of the SRMM is less than 8%. The space-time similarity between the robot and the performer is greater than 85%, and the motion smoothness of the online FEI exceeded 90%. Compared with other related methods, the proposed method achieves a remarkable improvement in motor position deviation, space-time similarity, and motion smoothness.

Effect of facial expression imitation training on children’s empathy

Effect of facial expression imitation training on children’s empathy

An Exploratory Study on the Psychophysiological Relationship Between Active Facial Imitation and Emotion Perception

An Exploratory Study on the Psychophysiological Relationship Between Active Facial Imitation and Emotion Perception

Mothers with depressive symptoms display differential brain activations when empathizing with infant faces

Maternal care during the first year of life provides the foundation for the infant’s emotional and cognitive development. Depressive symptoms in mothers can undermine their early dyadic interaction, which may lead to various psychopathological disorders with long-term consequences. During this period, the mother-child interaction is exclusively preverbal and is based on the mother’s ability to understand her infant’s needs and feelings (i.e., empathy) and on reciprocal imitation of facial expressions that promote a social dialog that influences the development of the infant self. To study the effects of maternal depressive symptoms on neural circuits underlying these processes, we studied 16 healthy mothers (H) and 14 mothers with depressive symptoms (D), as assessed by the Center for Epidemiologic Studies Depression Scale. Subjects underwent functional magnetic resonance during observation/empathizing (OE) and imitation (IM) of the faces of both their own child and of that of an unknown child aged between 6 and 12 months. During OE, D deactivated the orbital and medial prefrontal cortex to a greater extent (compared with H), thus pointing to an increased internally focused cognitive style during rest. Moreover, D, in respect to H, displayed a greater reactivity of the right amygdala, which may be an expression of emotional dysregulation.

Beamformer-based spatiotemporal imaging of linearly-related source components using electromagnetic neural signals

Functional connectivity calculated using multiple channels of electromagnetic brain signals is often over- or underestimated due to volume conduction or field spread. Considering connectivity measures, coherence is suitable for the detection of rhythmic synchronization, whereas temporal correlation is appropriate for transient synchronization. This paper presents a beamformer-based imaging method, called spatiotemporal imaging of linearly-related source component (SILSC), which is capable of estimating connectivity at the cortical level by extracting the source component with the maximum temporal correlation between the activity of each targeted region and a reference signal. The spatiotemporal correlation dynamics can be obtained by applying SILSC at every brain region and with various time latencies. The results of six simulation studies demonstrated that SILSC is sensitive to detect the source activity correlated to the specified reference signal and is accurate and robust to noise in terms of source localization. In a facial expression imitation experiment, the correlation dynamics estimated by SILSC revealed the regions with mirror properties and the regions involved in motor control network when performing the imitation and execution tasks, respectively, with the left inferior frontal gyrus specified as the reference region.

Effects of context and facial expression on imitation tasks in preschool children with autism

The present study explored the effect of the context in which an imitation act occurs (elicited/spontaneous) and the experimenter's facial expression (neutral or smiling) during the imitation task with young children with autism and typically developing children. The participants were 10 typically developing children and 10 children with autism (mean chronological age: 72 months). They were tested in imitation of tasks and facial expressions posed by the researcher. The results showed that, compared with typically developing children, children with autism: (a) engaged less in imitation of action with objects, (b) had more difficulties with the imitation of facial expressions in the elicited condition, and (c) performed less accurately both at imitating the experimenter's facial expression and on tasks involving the simultaneous imitation of action with objects and facial expression, and (d) the type of the experimenter's facial expression did not influence the imitative performance of either group in either the elicited or the spontaneous condition. The present study attempts to advance investigation of imitative ability and impairment in autism.

Positive facial affect - an fMRI study on the involvement of insula and amygdala.

Imitation of facial expressions engages the putative human mirror neuron system as well as the insula and the amygdala as part of the limbic system. The specific function of the latter two regions during emotional actions is still under debate. The current study investigated brain responses during imitation of positive in comparison to non-emotional facial expressions. Differences in brain activation of the amygdala and insula were additionally examined during observation and execution of facial expressions. Participants imitated, executed and observed happy and non-emotional facial expressions, as well as neutral faces. During imitation, higher right hemispheric activation emerged in the happy compared to the non-emotional condition in the right anterior insula and the right amygdala, in addition to the pre-supplementary motor area, middle temporal gyrus and the inferior frontal gyrus. Region-of-interest analyses revealed that the right insula was more strongly recruited by (i) imitation and execution than by observation of facial expressions, that (ii) the insula was significantly stronger activated by happy than by non-emotional facial expressions during observation and imitation and that (iii) the activation differences in the right amygdala between happy and non-emotional facial expressions were increased during imitation and execution, in comparison to sole observation. We suggest that the insula and the amygdala contribute specifically to the happy emotional connotation of the facial expressions depending on the task. The pattern of the insula activity might reflect increased bodily awareness during active execution compared to passive observation and during visual processing of the happy compared to non-emotional facial expressions. The activation specific for the happy facial expression of the amygdala during motor tasks, but not in the observation condition, might reflect increased autonomic activity or feedback from facial muscles to the amygdala.

In play we trust. Rapid facial mimicry predicts the duration of playful interactions in geladas.

The primate play-face is homologous to the human facial display accompanying laughter. Through facial mimicry, the play-face evokes in the perceiver a similar positive emotional state. This sensorimotor and emotional sharing can be adaptive, as it allows individuals to fine-tune their own motor sequences accordingly thus increasing cooperation in play. It has been recently demonstrated that, not only humans and apes, but also geladas are able to mimic others' facial expressions. Here, we describe two forms of facial mimicry in Theropithecus gelada: rapid (RFM, within 1.0 s) and delayed (DFM, within 5.0 s). Play interactions characterized by the presence of RFM were longer than those with DFM thus suggesting that RFM is a good indicator of the quality of communicative exchanges and behavioral coordination. These findings agree with the proposal of a mirror mechanism operating during perception and imitation of facial expressions. In an evolutionary perspective, our findings suggest that RFM not only was already present in the common ancestor of cercopitecoids and hominoids, but also that there is a relationship between RFM and length and quality of playful interactions.

The Link between Facial Feedback and Neural Activity within Central Circuitries of Emotion—New Insights from Botulinum Toxin–Induced Denervation of Frown Muscles

Afferent feedback from muscles and skin has been suggested to influence our emotions during the control of facial expressions. Recent imaging studies have shown that imitation of facial expressions is associated with activation in limbic regions such as the amygdala. Yet, the physiological interaction between this limbic activation and facial feedback remains unclear. To study if facial feedback effects on limbic brain responses during intentional imitation of facial expressions, we applied botulinum toxin (BTX)-induced denervation of frown muscles in combination with functional magnetic resonance imaging as a reversible lesion model to minimize the occurrence of afferent muscular and cutaneous input. We show that, during imitation of angry facial expressions, reduced feedback due to BTX treatment attenuates activation of the left amygdala and its functional coupling with brain stem regions implicated in autonomic manifestations of emotional states. These findings demonstrate that facial feedback modulates neural activity within central circuitries of emotion during intentional imitation of facial expressions. Given that people tend to mimic the emotional expressions of others, this could provide a potential physiological basis for the social transfer of emotion.

Imitation of in-group versus out-group members’ facial expressions of anger: A test with a time perception task

This research investigated the automatic imitation of facial expressions of anger by in-group and out-group members, using a temporal estimation task. Individuals typically overestimate duration represented by angry faces, probably due to increases in arousal (Droit-Volet, Brunot, & Niedenthal, 2004). Overestimation is not observed when imitation of the facial expressions is inhibited, suggesting that embodied simulation mediates the changes in arousal (Effron, Niedenthal, Gil, & Droit-Volet, 2006). This method thus provides an implicit measure of imitation and was used to test the hypothesis that individuals imitate in-group, but not out-group members’ facial expressions of emotion. In separate studies Chinese and French Caucasian participants were presented with short (400 ms) and long (1600 ms) standard durations in a temporal bisection task. They then categorized standard and intermediate durations, represented by angry and neutral faces, in terms of similarity to the short and long standard durations. Half of the face stimuli were Chinese, and half Caucasian. Results revealed a bias in the temporal perception of emotion for the Caucasian participants when they were presented with Caucasian facial expressions and not Chinese ones. In contrast, this bias in time perception was observed when Chinese individuals imitated faces of both in- and out-group members. The results of the Chinese participants are interpreted in terms of familiarity with and motivations to understand the emotional expressions of members of a host culture.

Imitation of facial expressions in schizophrenia

Diminished facial expressivity is a common feature of schizophrenia that interferes with effective interpersonal communication. This study was designed to determine if real-time visual feedback improved the ability of patients with schizophrenia to imitate and produce modeled facial expressions. Twenty patients with schizophrenia and 10 controls viewed static images of facial expressions and were asked to imitate them. Half of the images were imitated with the use of a mirror and half were imitated without the use of a mirror. In addition, we examined whether practice in imitating and producing expressions improved the ability of participants to generate facial expressions on their own, without the aid of a model or mirror. Participants' facial expressions were photographed with a digital camera and each was rated for accuracy in producing characteristic facial expressions. Patients with schizophrenia were less accurate in imitating and producing facial expressions than controls, and real-time visual feedback did not improve accuracy in either group. Preliminary findings suggest that exposure to model expressions and practice in generating these expressions can improve the accuracy of certain posed expressions in schizophrenia.

Amélioration de la reconnaissance et de l’imitation d’expressions faciales chez des enfants autistes grâce à une présentation visuelle et sonore ralentie

An increasing amount of studies have demonstrated the existence of visual, auditive and motion integration deficit in individuals with autism, especially when movement is rapid. Since visual motion is intrinsically involved in social interactions through movements of the body and face, visual motion processing deficit could therefore account for their verbal and non-verbal comprehension and social interaction disorders. Besides, other studies have revealed the existence of speech integration deficit in this population which could also account for their verbal comprehension deficits. The aim of our study is to test whether slowing down sound flow and facial movement enhance facial recognition and imitation by autistic children compared to normal control children. Results show that a slow dynamic presentation enhances recognition and imitation of facial expressions by autistic children. Our perspective is to elaborate software that could simultaneously slow down facial movements and speech flow, and to assess its impact on the comprehension of words and instructions by autistic children.

Development of social cognition in infant chimpanzees (Pan troglodytes): Face recognition, smiling, gaze, and the lack of triadic interactions1

Abstract: In this paper, we summarize a series of studies on the developmental changes in social cognition in mother‐raised infant chimpanzees from birth to around 2 years old. The infants preferentially tracked a photograph of their mother's face at 1 month but showed indifferent preferences to faces at 2 months old. This change in facial recognition was correlated with a decrease in neonatal spontaneous smiling, increase in social smiling and a decline in neonatal imitation of facial expressions. Also at around 2 months, the infants began to show preferences for directed‐gaze faces over averted gazes, and the amount of mutual gaze time between mother and infant chimpanzees increased. Thus, by 2 months of age, abilities required for dyadic interactions are already developed in chimpanzees as is the case in humans. The development of triadic interactions, however, is rather different between these two species. The infant chimpanzee can follow another's pointing or gaze at around 1 year, but even by 2 years old, does not “share” attention with the others.

The Faclal Feedback Hypothesis in Human Interaction

In theory and research on human emotional response, the facial feedback hypothesis (FFH) has held a special place because its claims are both counterintuitive and controversial. In this article, evidence for the hypothesis is reviewed in light of certain of the critical assessments of the hypothesis. It is concluded that the empirical evidence and the theoretical speculation are of sufficient strength to warrant confidence in the reliability of the FFH. The FFH is applied to the interpersonal context by noting the evidence for the imitation of facial expressions in both adults and infants. The `interpersonal facial feedback hypothesis' holds that in face-to-face contexts the hedonic tone of facial displays achieved through imitation should influence the underlying affect experienced by the partners. The link between facial imitation and affect can account in part for their attraction to the partner and situation. A reanalysis of previous data provides evidence for the imitation of smiles in conversation among various types of dyads and for an independent effect of a person's own smiles on his or her own attraction to the partner (after the effects of other predictors of attraction have been removed).

Neonatal discrimination and imitation of facial expression: A failure to replicate

Neonatal discrimination and imitation of facial expression: A failure to replicate