Perception Of Facial Expressions Research Articles

Approach‐avoidance responses and categorical perception of ambiguous facial expressions

Emotion perception of facial expressions involves two processes: quick approach-avoidance responses and subsequent sorting into emotional categories (i.e., happiness, anger), considering the context. Sorting of morphed ambiguous facial expressions is known to occur categorically, but the occurrence of approach-avoidance responses for morphed facial expressions is yet to be investigated. The present study used morphed angry and fearful facial expressions and measured approach-avoidance responses among Japanese university students (Experiment 1, n= 29). Similar experiments with linguistic load (Experiment 2, n= 28) and visual load (Experiment 3, n= 29) were conducted. The results indicated categorical perception in the sorting of facial expressions but no approach-avoidance response for morphed expressions. Furthermore, linguistic load affected the categorisation of facial expressions, but neither linguistic load nor visual load affected the approach-avoidance response. These results support the idea that the non-linguistic approach-avoidance response and the linguistic categorisation of facial expressions are two different processes. The nature of the emotional perception process is also discussed.

Model for predicting perception of facial action unit activation using virtual humans

Blendshape facial rigs are used extensively in the industry for facial animation of virtual humans. However, storing and manipulating large numbers of facial meshes (blendshapes) is costly in terms of memory and computation for gaming applications. Blendshape rigs are comprised of sets of semantically-meaningful expressions, which govern how expressive the character will be, often based on Action Units from the Facial Action Coding System (FACS). However, the relative perceptual importance of blendshapes has not yet been investigated. Research in Psychology and Neuroscience has shown that our brains process faces differently than other objects so we postulate that the perception of facial expressions will be feature-dependent rather than based purely on the amount of movement required to make the expression. Therefore, we believe that perception of blendshape visibility will not be reliably predicted by numerical calculations of the difference between the expression and the neutral mesh. In this paper, we explore the noticeability of blendshapes under different activation levels, and present new perceptually-based models to predict perceptual importance of blendshapes. The models predict visibility based on commonly-used geometry and image-based metrics.

Recognition Characteristics of Facial and Bodily Expressions: Evidence From ERPs.

In the natural environment, facial and bodily expressions influence each other. Previous research has shown that bodily expressions significantly influence the perception of facial expressions. However, little is known about the cognitive processing of facial and bodily emotional expressions and its temporal characteristics. Therefore, this study presented facial and bodily expressions, both separately and together, to examine the electrophysiological mechanism of emotional recognition using event-related potential (ERP). Participants assessed the emotions of facial and bodily expressions that varied by valence (positive/negative) and consistency (matching/non-matching emotions). The results showed that bodily expressions induced a more positive P1 component and a shortened latency, whereas facial expressions triggered a more negative N170 and prolonged latency. Among N2 and P3, N2 was more sensitive to inconsistent emotional information and P3 was more sensitive to consistent emotional information. The cognitive processing of facial and bodily expressions had distinctive integrating features, with the interaction occurring in the early stage (N170). The results of the study highlight the importance of facial and bodily expressions in the cognitive processing of emotion recognition.

Cross-Cultural and Cultural-Specific Production and Perception of Facial Expressions of Emotion in the Wild

Automatic recognition of emotion from facial expressions is an intense area of research, with a potentially long list of important application. Yet, the study of emotion requires knowing which facial expressions are used within and across cultures in the wild, not in controlled lab conditions; but such studies do not exist. Which and how many cross-cultural and cultural-specific facial expressions do people commonly use? And, what affect variables does each expression communicate to observers? If we are to design technology that understands the emotion of users, we need answers to these two fundamental questions. In this paper, we present the first large-scale study of the production and visual perception of facial expressions of emotion in the wild. We find that of the 16,384 possible facial configurations that people can theoretically produce, only 35 are successfully used to transmit emotive information across cultures, and only 8 within a smaller number of cultures. Crucially, we find that visual analysis of cross-cultural expressions yields consistent perception of emotion categories and valence, but not arousal. In contrast, visual analysis of cultural-specific expressions yields consistent perception of valence and arousal, but not of emotion categories. Additionally, we find that the number of expressions used to communicate each emotion is also different, e.g., 17 expressions transmit happiness, but only 1 is used to convey disgust.

Facial expression stereotypes of rich and poor adults and children.

Facial expression stereotypes can affect the perception of other people's facial expressions. This study examined facial expression stereotypes of poor and rich adults and children. Experiment 1 found that the adult participants associated rich adults with positive emotions (i.e., happiness) and poor adults with negative emotions (i.e., sadness). In Experiments 2-4, adult participants still thought that rich 4-, 6-, and 10-year-old children would show positive facial expressions (happiness) but did not think that poor 4-, 6-, and 10-year-old children would show negative emotions (sadness). These finding have implications concerning how adults communicate with poor and rich adults and children.

Effects of posed smiling on memory for happy and sad facial expressions

The perception and storage of facial emotional expressions constitutes an important human skill that is essential for our daily social interactions. While previous research revealed that facial feedback can influence the perception of facial emotional expressions, it is unclear whether facial feedback also plays a role in memory processes of facial emotional expressions. In the present study we investigated the impact of facial feedback on the performance in emotional visual working memory (WM). For this purpose, 37 participants underwent a classical facial feedback manipulation (FFM) (holding a pen with the teeth—inducing a smiling expression vs. holding a pen with the non-dominant hand—as a control condition) while they performed a WM task on varying intensities of happy or sad facial expressions. Results show that the smiling manipulation improved memory performance selectively for happy faces, especially for highly ambiguous facial expressions. Furthermore, we found that in addition to an overall negative bias specifically for happy faces (i.e. happy faces are remembered as more negative than they initially were), FFM induced a positivity bias when memorizing emotional facial information (i.e. faces were remembered as being more positive than they actually were). Finally, our data demonstrate that men were affected more by FFM: during induced smiling men showed a larger positive bias than women did. These data demonstrate that facial feedback not only influences our perception but also systematically alters our memory of facial emotional expressions.

Blocking facial mimicry during binocular rivalry modulates visual awareness of faces with a neutral expression

Several previous studies have interfered with the observer’s facial mimicry during a variety of facial expression recognition tasks providing evidence in favor of the role of facial mimicry and sensorimotor activity in emotion processing. In this theoretical context, a particularly intriguing facet has been neglected, namely whether blocking facial mimicry modulates conscious perception of facial expressions of emotions. To address this issue, we used a binocular rivalry paradigm, in which two dissimilar stimuli presented to the two eyes alternatingly dominate conscious perception. On each trial, female participants (N = 32) were exposed to a rivalrous pair of a neutral and a happy expression of the same individual through anaglyph glasses in two conditions: in one, they could freely use their facial mimicry, in the other they had to keep a chopstick between their lips, constraining the mobility of the zygomatic muscle and producing ‘noise’ for sensorimotor simulation. We found that blocking facial mimicry affected the perceptual dominance in terms of cumulative time favoring neutral faces, but it did not change the time before the first dominance was established. Taken together, our results open a door to future investigation of the intersection between sensorimotor simulation models and conscious perception of emotional facial expressions.

Children's perception of facial expressions.

This study investigated the developing ability of children to identify emotional facial expressions in terms of the contexts in which they generally occur. We presented Dutch 6- to 9-year-old primary school children (N = 164, 98 girls) prototypical contexts for different emotion categories and asked them whether different kinds of facial expressions belonged to those contexts or not, using a 2-alternative forced-choice task. Correct and incorrect responses were quantified into a single index using signal detection theory, representing children's sensitivity to perceive each facial expression as categorically different from each of the others in terms of their prototypical contexts. Results show age-related improvements in identifying facial expressions as belonging to their prototypical contexts. In addition, we found that older children not only made less misidentifications but also misidentified less kinds of facial expressions to the prototypical contexts. Furthermore, the kinds of misidentifications children made suggest that they do not identify facial expressions based on their conceptual emotional valence. Results were discussed from a perceptual learning account. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Facial expressions alter the fundamental sound properties of speech

Literature from across academic disciplines has demonstrated significant links between emotional valence and language. For example, Whissell’s Dictionary of Affect in Language defines three dimensions upon which the emotionality of words is describable, and Ekman’s Theories of Emotion include the perception and internalization of facial expressions. The present study seeks to expand upon these works by exploring whether holding facial expressions alters the fundamental speech properties of spoken language. Nineteen (19) participants were seated in a soundproof chamber and were asked to speak a series of pseudowords containing target phonemes. The participants spoke the pseudowords either holding no facial expression, smiling, or frowning, and the utterances recorded using a high-definition microphone and phonologically analysed using PRAAT analysis software. Analyses revealed a pervasive gender differences in frequency variables, where males showed lower fundamental but higher formant frequencies compared to females. Significant main effects were found within the fundamental and formant frequencies, but no effects were discerned for the intensity variable. While intricate, these results are indicative of an interaction between the activity of facial musculature when reflecting emotional valence and the sound properties of speech uttered simultaneously.

A Review of the Effects of Valenced Odors on Face Perception and Evaluation.

How do valenced odors affect the perception and evaluation of facial expressions? We reviewed 25 studies published from 1989 to 2020 on cross-modal behavioral effects of odors on the perception of faces. The results indicate that odors may influence facial evaluations and classifications in several ways. Faces are rated as more arousing during simultaneous odor exposure, and the rated valence of faces is affected in the direction of the odor valence. For facial classification tasks, in general, valenced odors, whether pleasant or unpleasant, decrease facial emotion classification speed. The evidence for valence congruency effects was inconsistent. Some studies found that exposure to a valenced odor facilitates the processing of a similarly valenced facial expression. The results for facial evaluation were mirrored in classical conditioning studies, as faces conditioned with valenced odors were rated in the direction of the odor valence. However, the evidence of odor effects was inconsistent when the task was to classify faces. Furthermore, using a z-curve analysis, we found clear evidence for publication bias. Our recommendations for future research include greater consideration of individual differences in sensation and cognition, individual differences (e.g., differences in odor sensitivity related to age, gender, or culture), establishing standardized experimental assessments and stimuli, larger study samples, and embracing open research practices.

Investigating the effects of perinatal status and gender on adults' responses to infant and adult facial emotion.

The perception of emotion in infant faces is a key parental skill, thought to be impacted by caregiving experience. It is widely assumed that women, and in particular mothers in the postnatal period, are more attuned to infant facial expressions than men. However, empirical evidence for this is lacking, and it is not yet clear whether potential differences in emotion processing between adults during pregnancy and postnatally are specific to infant expressions or extend to faces of all ages. In this cross-sectional study using a subsample from a Finnish birth cohort (N = 610), we examine adult and infant facial expression perception in pre- and postnatal men and women. Women rated the happy infant faces more positively on the valence (pleasure) dimension than men, but men rated the faces higher on the arousal (excited) dimension. There were no significant differences between adults responding during pregnancy or postnatally, but first-time mothers rated the faces as higher in arousal overall than multiparous mothers. The ability to identify specific emotions (e.g., sadness) in adult faces correlated with judgments of emotion in similarly valenced infant faces. We conclude that adults differ in their sensitivity to positive or negative emotions, independent of whether they are expressed in infant or adult faces. We did not find that perinatal status (pre- or postnatal) was associated with differences in sensitivity to emotion in infant or adult faces. Men and women were differentially sensitive to the valence and arousal in infant faces, independent of the timing of their responses. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

How Weeping Influences the Perception of Facial Expressions: The Signal Value of Tears

Emotional tears have been proposed to serve as a signal of distress, appeasement, and helplessness, which promotes prosocial responses in observers. They may also facilitate the perception of sadness. A still unanswered question is what information tears convey about emotional states when they are combined with different muscular facial expressions. The current study evaluated three hypotheses: Tears facilitate inferences about (a) emotion intensity in general (b) sadness in particular, or (c) helplessness-related appraisal and behavioral intentions. In the first experiment, participants viewed pictures of (non)tearful real and artificial faces displaying anger, disgust, fear, happiness, sadness, surprise, and neutral state. They had to report which of the seven expressions they recognized, and to rate its intensity, sincerity, and felt empathy. Tears appeared to facilitate the perception of sadness, but also of anger and fear, while they decreased the perception of disgust and surprise. The ratings of the intensity, the perceived sincerity, and the experienced empathy followed a similar pattern. In the second experiment, participants had to indicate if briefly (50 ms) presented (non)tearful faces showed a particular expression, and we measured their accuracy and reaction times. The results of the first experiment were not corroborated. Overall, the findings lend most support to the appraisal/behavioral intentions hypothesis and less support for the intensity and the sadness enhancement hypotheses.

Supplemental Material for Children’s Perception of Facial Expressions

Supplemental Material for Children’s Perception of Facial Expressions

Neural Computations Underlying Contextual Processing in Humans

Context shapes our perception of facial expressions during everyday social interactions. We interpret a person's face in a hostile situation negatively, while judging the same face in a pleasant circumstance positively. Critical to our adaptive fitness, context provides situation-specific framing to resolve ambiguity and guide our interpersonal behavior. This context-specific modulation of facial expression is thought to engage the amygdala, the hippocampus, and the orbitofrontal cortex; however, the underlying neural computations remain unknown. Here, we use human intracranial EEG directly recorded from these regions and report bidirectional theta-gamma interactions within the amygdala-hippocampal network, facilitating contextual processing. Critically, contextual information is subsequently represented in the orbitofrontal cortex, where theta phase precession binds context and face associations within theta cycles, endowing faces with contextual meanings at behavioral timescales. Our results identify theta phase precession as mediating associations between context and face processing, supporting flexible social behavior.

Detached empathic experience of others’ pain in remitted states of depression – An fMRI study

BackgroundMajor depressive disorder is strongly associated with impairments and difficulties in social interactions. Deficits in empathy, a vital skill for social interactions, have been identified as a risk factor for relapse. However, research on empathy in remitted states of depression is scarce. We chose a social neuroscience approach to investigate potentially altered neural processes involved in sub-components of empathy in remitted states of depression. We expected aberrations in cognitive components of empathy, based on previous reports regarding their role as risk factors for relapse. MethodsEmploying functional magnetic resonance imaging and a pain empathy task (video clips of painful medical treatments), we compared behavioral and neural empathic responses of unmedicated remitted depressive patients (N = 32) to those of untreated acutely depressed patients (N = 29) and healthy controls (N = 35). Self-report ratings of pain evaluation and affect-sharing were obtained. ResultsCompared to controls and acutely depressed patients, remitted depressive patients reported higher pain evaluation and showed increased activity in the right temporo-parietal junction. This region, which is central to self-other distinction and which has been linked to adopting a detached perspective, also exhibited reduced connectivity to the anterior insula. Furthermore, we observed reduced activity in regions involved in emotion processing (amygdala) and perception of affective facial expressions (fusiform face area, posterior superior temporal sulcus). ConclusionsRemitted states of depression are associated with a detached empathic style in response to others’ pain, characterized by increased self-other distinction, lowered affective processing, and reduced connectivity between empathy-related brain regions. Although this may prevent emotional harm in specific situations, it may reduce opportunities for positive experiences in social interactions in the long run.

Peculiarities of information transfer within functional cortical network during emotional face perception

Aim: The research aimed to study and model the emotion-related activity of functional networks within the human brain cortex using power spectrum density and detrended phase transfer entropy methods. Attention was focused on revealing alterations in cognitive mechanisms, caused by presentation of neutral human faces as rare stimuli among faces with either negative or positive expression. Methods: EEG-data was recorded during the perception and processing of neutral human facial expressions, presented among positive and negative faces in two series of images, alongside with resting state with open and closed eyes, which was further analyzed using power spectrum density and detrended phase transfer entropy methods. Results: Specific EEG-bands (θ and β) were chosen for the analysis based on their prominent role in memory- and emotion-related mechanisms. The topography of the spectral power density corresponded to the generally accepted ideas describing perception and visual stimuli processing mechanisms. The phase transfer entropy method was not sufficient to analyze resting state data. The results of the analysis performed using the phase transfer entropy method revealed the problems of neutral faces differentiation when presented in a positive emotional context. Simultaneously, enhanced processes of motivational coding and self-reflection were observed during the presentation of neutral faces in a negative emotional context. These results corresponded with the data obtained in our previous ERP-based study. Conclusions: Phase transfer entropy and spectral power density have demonstrated their effectiveness in analyzing the mechanisms of emotional visual stimuli processing mediated in different cortical areas.

Trustworthiness and electrocortical processing of emotionally ambiguous faces in student police officers

Perceptions of emotional facial expressions and trustworthiness of others guides behavior and has considerable implications for individuals who work in fields that require rapid decision making, such as law enforcement. This is particularly complicated for more ambiguous expressions, such as ‘neutral’ faces. We examined behavioral and electrocortical responses to facial expressions in 22 student police officers (18 males; 23.2 ± 3.63 years). Participants completed an emotional face appraisal task that involved viewing three expressions (fearful, neutral, happy) and were asked to identify the emotion and rate the trustworthiness of each face. The late positive potential (LPP), an event-related potential that tracks emotional intensity and/or salience of a stimulus, was measured during the task. Overall, participants rated neutral faces similarly to fearful faces and responded fastest to these expressions. Neutral faces also elicited a robust late LPP response that did not differ from LPP to fearful or happy faces, and there was substantial individual variation in trustworthiness ratings for neutral faces. Together, ‘neutral’ facial expressions elicited similar trustworthiness ratings to negatively-valenced stimuli. Brain and behavioral responses to neutral faces also varied across student officers; thus, encounters with ambiguous faces in the field may promote increased perceived threat in some officers, which may have real-world consequences (e.g., decision to shoot, risk of psychopathology).

From structure to concepts: The two stages of facial expression recognition

Facial expressions are the dominant way of human emotional and social communications. However, it remains unclear that how can perceivers extract emotion from the face. In the present study, we adopted a repetition-priming paradigm in combine with event-related potentials (ERP) to examine neurocognitive processing stages of facial expression perception. Results showed that emotional words were recognized faster than emotional faces, when both of which were primed by emotional faces, indicating the involvement of concepts processes in facial expression recognition. ERP results showed that all emotional faces could evoke responses of N170, while there was no significant difference between positive and negative emotional faces, suggesting that geometrical configurations of faces rather than emotional concepts of faces are processed at the stage of N170. In contrast, both emotional words and faces showed larger P2 in response to anger than happiness, which suggests that emotional concepts are extracted from faces at the stage of P2. To examine the underlying dynamic causal connectivity between facial structure and emotional conception, we conducted information flow analysis, which showed significant decreases of information flow from the fusiform gyrus to dorsal anterior cingulate cortex/dorsal medial prefrontal cortex and increases of information flow from the fusiform gyrus to posterior insula. These results revealed neural mechanisms underlying processes from physical structure to emotional concepts. Our findings suggest that facial expression recognition consists of two stages from geometrical structure of faces to emotional concepts of facial expressions, which provides evidence for facial expression processing and has important implications in the diagnosis and treatment of emotion recognition related disorders.

Does body context affect facial emotion perception and eliminate emotional ambiguity without visual awareness?

ABSTRACT In this study, we hypothesized that the emotional cues of conveyer’s body posture play a role in eliminating ambiguity of facial expressions. The participants were presented with faces with emotionally congruent and incongruent body posture or without a posture (no-context) for a brief time (33 ms) in Experiment 1. The results showed that congruent pairs led to higher accuracy of facial emotion perception compared with incongruent pairs and no-context condition. On the other hand, angry, sad and disgusted emotional facial expressions were blended to create emotional ambiguity in Experiment 2, and the results partially supported the hypotheses where the context effect emerged in ambiguous anger–disgust morphs but not in unambiguous facial expressions. Thus, incongruent emotional cues in body posture have an interfering effect on the perception of emotional facial expression without awareness.

Emotional Facial Expression Perception and Processing in AUD Subgroups

Deficits in emotion processing among individuals with AUD are well accepted, however the potential impact of polysubstance use in this population remains uninvestigated. The current work begins to fill this gap by analyzing affective perception and processing in community controls (CCs) and two AUD subgroups differentiated by presence (Alc-Drug) or absence (Alc-Only) of polysubstance use. Behavioral task performance and electroencephalographic (EEG) indices (N170, P3) were measured for an emotion judgement task where participants classified emotional facial expressions (EFEs) morphed to 65 or 95 percent intensity. Mixed model analyses detected deficits in emotion classification accuracy among Alc-Drug relative to other groups. Although there was a main effect of emotion (greater accuracy for positive vs. negative emotions), there was no group by emotion interaction. N170 amplitude analyses found only a main effect for emotion (greater amplitude for negative vs. positive emotions). P3 amplitude analyses detected differences between control and AUD individuals, but no difference between AUD subgroups. No correlation was found between accuracy and ERP amplitudes. These findings contribute to the developing literature on emotional processing deficits in AUD, including highlighting the importance of considering polysubstance use in characterizing these deficits.