Perception Of Facial Expressions Research Articles

Neural computations underlying contextual processing in humans.

SUMMARYContext shapes our perception of facial expressions during everyday social interactions. We interpret a person’s face in a hostile situation negatively and judge the same face under pleasant circumstances positively. Critical to our adaptive fitness, context provides situation-specific framing to resolve ambiguity and guide our interpersonal behavior. This context-specific modulation of facial expression is thought to engage the amygdala, hippocampus, and orbitofrontal cortex; however, the underlying neural computations remain unknown. Here we use human intracranial electroencephalograms (EEGs) directly recorded from these regions and report bidirectional theta-gamma interactions within the amygdala-hippocampal network, facilitating contextual processing. Contextual information is subsequently represented in the orbitofrontal cortex, where a theta phase shift binds context and face associations within theta cycles, endowing faces with contextual meanings at behavioral timescales. Our results identify theta phase shifts as mediating associations between context and face processing, supporting flexible social behavior.

Emotion concept in perception of facial expressions: Effects of emotion-label words and emotion-laden words

Emotion perception, the ability to detect and categorize the emotions of others, is an important component of social competence. The basic emotion theory and constructed emotion theory remain controversial over the role of emotional semantic information in the processing of facial expressions. In the present study, in order to explore the mechanism of emotional semantic information in different processing stages of facial expressions, we used the ERPs to investigate the effects of emotion-label words (e.g., happy, disgusted) and emotion-laden words (e.g., flower, maggot) on the perception of happy and disgusted faces. The results showed that disgusted faces were more susceptible to emotion words, and there were differences between word types and consistency conditions in the N170 component. More importantly, disgusted faces showed a significant N400 effect in the emotion-label words condition, that is, the amplitude in the emotion inconsistent condition was negative compared with that in the emotion consistent condition. The processing of happy faces was not significantly affected by the emotional information of words, and no significant effects related to consistency or to word types were found. These results found that disgusted faces exhibited emotional semantic conflict in the inconsistent emotion-label words condition, suggesting that emotion words moderated the perception of disgusted faces, supporting the hypothesis that emotional semantic information influenced the perception of facial expressions.

How and why head position changes the perception of facial expressions of emotion.

A large body of research on emotion communication has demonstrated that facial muscle movements (i.e., facial expressions) influence social perceptions made from faces. However, new research suggests that head position can also affect the way that faces are perceived, by systematically changing the appearance of the face. More specifically, according to the action-unit imposter account, tilting one's head downward causes the eyebrows to appear lower and take on a V shape-the same appearance cues associated with a particular facial muscle movement (corrugator activity, or Action Unit 4 [AU4]). Drawing on this account, four studies (two of which were preregistered) tested whether a downward head tilt intensifies perceptions of facial expressions of emotion that include V-shaped eyebrows from AU4 but weaken perceptions of expressions that do not. Supporting this hypothesis, findings showed that (a) when the head is tilted downward, anger expressions-which include V-shaped eyebrows from AU4-are perceived as more intense, whereas expressions of happiness, disgust, fear, and surprise-which do not include V-shaped eyebrows from AU4-are perceived as less intense; (b) visually apparent changes to the eyebrows caused by the action-unit imposter effect account for the effect of a downward head tilt on perceptions of anger; and (c) this head movement is spontaneously used by individuals seeking to encode facial expressions of anger. Together, findings suggest that head movements play an important role in communicating emotion expressions from the face, especially anger. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Context consistency improves ensemble perception of facial expressions.

Humans have developed the capacity to rapidly extract summary statistics from the facial expressions of a crowd, such as computing the average facial expression. Although dual-task paradigms involving memory and ensemble tasks have recently found that this ensemble coding ability is biased by visual working memory, few studies have examined whether the context-dependent nature of memory itself can influence the perceptual averaging process. In two experiments, participants made forced-choice judgments about mean facial expressions that were paired with task-irrelevant background images, and the background images either matched or mismatched across encoding and response phases. When the backgrounds matched, it was at either the perceptual level (uniformly oriented lines with the same orientation in encoding and response phases, in Experiment 1), or at the summary statistics level (uniformly oriented lines in the response phase that had the same orientation as the mean of randomly oriented lines that were seen in the encoding phase, in Experiment 2). Participants in Experiment 1 showed a higher ensemble precision and better discrimination sensitivity when the backgrounds matched than when they mismatched, which is consistent with the kind of robust contextual memory effect that has been seen in prior research. We further demonstrated that the context-matching facilitation effect occurred at both the perceptual level (Experiment 1) and at the summary statistics level (Experiment 2). These results demonstrate that the effects of visual working memory on perceptual averaging are obligatory, and they highlight the importance of memory-related context dependency in perceptual averaging.

The Relations of Attention to and Clarity of Feelings With Facial Affect Perception.

Attention to emotions and emotional clarity are core dimensions of individual differences in emotion awareness. Findings from prior research based on self-report indicate that attention to and recognition of one’s own emotions are related to attention to and recognition of other people’s emotions. In the present experimental study, we examined the relations of attention to and clarity of emotions with the efficiency of facial affect perception. Moreover, it was explored whether attention to and clarity of emotions are linked to negative interpretations of facial expressions. A perception of facial expressions (PFE) task based on schematic faces with neutral, ambiguous, or unambiguous emotional expressions and a gender decision task were administered to healthy individuals along with measures of emotion awareness, state and trait anxiety, depression, and verbal intelligence. Participants had to decide how much the faces express six basic affects. Evaluative ratings and decision latencies were analyzed. Attention to feelings was negatively correlated with evaluative decision latency, whereas clarity of feelings was not related to decision latency in the PFE task. Attention to feelings was positively correlated with the perception of negative affects in ambiguous faces. Attention to feelings and emotional clarity were not related to gender decision latency. According to our results, dispositional attention to feelings goes along with an enhanced efficiency of facial affect perception. Habitually paying attention to one’s own emotions may facilitate processing of external emotional information. Preliminary evidence was obtained suggesting a relationship of dispositional attention to feelings with negative interpretations of facial expressions.

Subjective and objective difficulty of emotional facial expression perception from dynamic stimuli.

This study aimed to discover predictors of subjective and objective difficulty in emotion perception from dynamic facial expressions. We used a multidimensional emotion perception framework, in which observers rated the perceived emotion along a number of dimensions instead of choosing from traditionally-used discrete categories of emotions. Data were collected online from 441 participants who rated facial expression stimuli in a novel paradigm designed to separately measure subjective (self-reported) and objective (deviation from the population consensus) difficulty. We targeted person-specific (sex and age of observers and actors) and stimulus-specific (valence and arousal values) predictors of those difficulty scores. Our findings suggest that increasing age of actors makes emotion perception more difficult for observers, and that perception difficulty is underestimated by men in comparison to women, and by younger and older adults in comparison to middle-aged adults. The results also yielded an increase in the objective difficulty measure for female observers and female actors. Stimulus-specific factors-valence and arousal-exhibited quadratic relationships with subjective and objective difficulties: Very positive and very negative stimuli were linked to reduced subjective and objective difficulty, whereas stimuli of very low and high arousal were linked to decreased subjective but increased objective difficulty. Exploratory analyses revealed low relevance of person-specific variables for the prediction of difficulty but highlighted the importance of valence in emotion perception, in line with functional accounts of emotions. Our findings highlight the need to complement traditional emotion recognition paradigms with novel designs, like the one presented here, to grasp the "big picture" of human emotion perception.

Correlations Between Psychological Status and Perception of Facial Expression.

Objective Facial affect recognition is associated with neuropsychological status and psychiatric diseases. We hypothesized that facial affect recognition is associated with psychological status and perception of other affects. Methods A total of 80 images depicting facial affect, including 20 Neutral, 20 Angry, 20 Fear, and 20 Sad, were screened for use in our research. A total of 100 healthy individuals were asked to rate these images using a 10-point Likert scale and complete psychological scales assessing the emotional statuses and cognitive functions. Results The participants’ emotional state of aggression, attention, and impulsivity may have been associated with their interpretation of the Angry facial expressions. The participants often rated the Angry facial expressions as Fear. The participants rated Fear images as Angry or Sad. In response to a Sad facial expression, the participants reported psychological statuses of attention and impulsivity which were associated with the facial expression rating. The participants rated the Sad expression as Angry or Fear. Conclusion The psychological statuses of the participants were significantly correlated with their interpretation of facial affects. In particular, a psychological state of attention was often correlated with incorrect affect ratings. Attention and impulsivity could affect the rating of the sad facial expressions.

Neural signatures of individual variability in context-dependent perception of ambiguous facial expression

How do we incorporate contextual information to infer others’ emotional state? Here we employed a naturalistic context-dependent facial expression estimation task where participants estimated pleasantness levels of others’ ambiguous expression faces when sniffing different contextual cues (e.g., urine, fish, water, and rose). Based on their pleasantness rating data, we placed participants on a context-dependency continuum and mapped the individual variability in the context-dependency onto the neural representation using a representational similarity analysis. We found that the individual variability in the context-dependency of facial expression estimation correlated with the activity level of the pregenual anterior cingulate cortex (pgACC) and the amygdala and was also decoded by the neural representation of the ventral anterior insula (vAI). A dynamic causal modeling revealed that those with higher context-dependency exhibited a greater degree of the modulation from vAI to the pgACC. These findings provide novel insights into the neural circuitry associated with the individual variability in context-dependent facial expression estimation and the first empirical evidence for individual variability in the predictive accounts of affective states.

Olfactory interference on the emotional processing speed of visual stimuli: The influence of facial expressions intensities.

Research on olfactory stimulation indicates that it can influence human cognition and behavior, as in the perception of facial expressions. Odors can facilitate or impair the identification of facial expressions, and apparently its hedonic valence plays an important role. However, it was also demonstrated that the presentation of happiness and disgust faces can influence the emotional appraisal of odorants, indicating a bilateral influence in this phenomenon. Hence, it’s possible that odor influences on emotional categorization vary depending on the intensity of expressions. To investigate this hypothesis, we performed an emotion recognition task using facial expressions of five emotional categories (happiness, fear, disgust, anger and sadness) with ten different intensities. Thirty-five participants completed four blocks of the task, each with a different olfactory condition, and we found that odorants’ effects varied according to the facial expressions intensity. Odorants enhanced the Reaction Time (RT) differences between threshold and high-intensity expressions for disgust and fear faces. Also, analysis of the RT means for high-intensity facial expressions revealed that the well-known advantage in recognition of happiness facial expressions, compared to other emotions, was enhanced in the positive olfactory stimulation and decreased in the negative condition. We conclude that olfactory influences on emotional processing of facial expressions vary along intensities of the latter, and the discrepancies of past research in this field may be a result of a bilateral effect in which the odorants influence the identification of emotional faces just as the facial expressions influence the emotional reaction to the odor.

Effects of wearing a transparent face mask on perception of facial expressions.

Wearing face masks in public has become the norm in many countries post-2020. Although mask-wearing is effective in controlling infection, it has the negative side effect of occluding the mask wearer’s facial expressions. The purpose of this study was to investigate the effects of wearing transparent masks on the perception of facial expressions. Participants were required to categorize the perceived facial emotion of female (Experiment 1) and male (Experiment 2) faces with different facial expressions and to rate the perceived emotion intensity of the faces. Based on the group, the participants were assigned to, the faces were presented with a surgical mask, a transparent mask, or without a mask. The results showed that wearing a surgical mask impaired the performance of reading facial expressions, both with respect to recognition and perceived intensity of facial emotions. Specifically, the impairments were robustly observed in fear and happy faces for emotion recognition, and in happy faces for perceived intensity of emotion in Experiments 1 and 2. However, the impairments were moderated by wearing a transparent mask instead of a surgical mask. During the coronavirus disease 2019 (COVID-19) pandemic, the transparent mask can be used in a range of situations where face-to-face communication is important.

Culture Shapes the Distinctiveness of Posed and Spontaneous Facial Expressions of Anger and Disgust

There is a growing consensus that culture influences the perception of facial expressions of emotion. However, relatively few studies have examined whether and how culture shapes the production of emotional facial expressions. Drawing on prior work on cultural differences in communication styles, we tested the prediction that people from the Netherlands (a low-context culture) produce facial expressions that are more distinct across emotions compared to people from China (a high-context culture). Furthermore, we examined whether the degree of distinctiveness varies across posed and spontaneous expressions. Dutch and Chinese participants were instructed to either pose facial expressions of anger and disgust, or to share autobiographical events that elicited spontaneous expressions of anger or disgust. Using a supervised machine learning approach to categorize expressions based on the patterns of activated facial action units, we showed that both posed and spontaneous facial expressions of anger and disgust were more distinct when produced by Dutch compared to Chinese participants. Yet, the distinctiveness of posed and spontaneous expressions differed in their sources. The difference in the distinctiveness of posed expressions appears to be due to a larger array of facial expression prototypes for each emotion in Chinese culture than in Dutch culture. The difference in the distinctiveness of spontaneous expressions, however, appears to reflect the greater similarity of expressions of anger and disgust from the same Chinese individual than from the same Dutch individual. The implications of these findings are discussed in relation to cross-cultural emotion communication, including via cultural products.

Does Attention to One’s Own Emotion Relate to the Emotional Interpretation of Other People’s Faces?

Do individual differences in attention to one’s own emotion relate to the way individuals interpret emotion in other people? For example, although the accuracy has been debated, people’s facial expressions are often perceived as providing information about their emotional state. Previous research on individual differences in attention to emotion has mostly looked at how individuals categorize the emotions they believe other people to have or to the extent to which individuals have dysregulated attention to emotional information. However, less is known about other facets of emotional interpretation such as perceptions of emotional intensity and genuineness. One hypothesis, suggested by previous literature on categorization, is that higher attention to one’s own emotion may result in greater differentiation of the cues perceived to relate to emotional intensity and genuineness. On the other hand, previous research on dysregulated attention to emotion suggests a second possibility: higher attention to emotion might instead result in heightened weighting of facial expressions which tend to be perceived to indicate emotional intensity and genuineness. Across two studies, participants rated their perception of emotional intensity in facial expressions and their perception of genuineness in Duchenne and non-Duchenne smiles. Contrary to both hypotheses, attention to emotion did not significantly relate to perceived emotional intensity or genuineness (Study 1). Furthermore, whereas an exploratory test suggested a significant relation between attention to emotion and the average perceived intensity of different emotion categories (Study 1), the relation was not significant in a conceptual replication study (Study 2). The current research highlights the possibility that future research may benefit from explorations of an expanded range of biased perceptions of facial expressions in relation to individual differences in attention to emotion.

Effect of skin colors due to hemoglobin or melanin modulation on facial expression recognition

Facial color is thought to contribute to the perception of emotional expression. Previous studies on facial recognition have shown that reddish and bluish faces enhance the perceptions of anger and sadness, respectively. However, the effect of facial color on the recognition of facial expressions in the range of natural skin colors remains unexplored. In this study, we examined the effect of natural skin color changes on the perception of facial expressions of anger, sadness, and happiness. We used five skin color conditions for each facial expression: standard, and hemoglobin or melanin decrease/increase (H-, H+, M-, and M + ). In the experiment, we used images of different facial expressions ranging from neutral to maximum intensity. Observers judged whether a face stimulus was perceived as “angry” (“happy” or “sad”) or “not angry” (“not happy” or “not sad”). The results showed that H+, M+, H-, and M- enhanced the perception of anger and happiness, respectively. However, the change in the perception of sadness due to these modulations was small. These results suggest that the range of natural facial colors by hemoglobin and melanin modulation affects facial expression recognition. We further consider the possibility that these results could be partially explained by the lightness of the stimulus.

Caricatured facial movements enhance perception of emotional facial expressions.

Although faces “in the wild” constantly undergo complicated movements, humans adeptly perceive facial identity and expression. Previous studies, focusing mainly on identity, used photographic caricature to show that distinctive form increases perceived dissimilarity. We tested whether distinctive facial movements showed similar effects, and we focussed on both perception of expression and identity. We caricatured the movements of an animated computer head, using physical motion metrics extracted from videos. We verified that these “ground truth” metrics showed the expected effects: Caricature increased physical dissimilarity between faces differing in expression and those differing in identity. Like the ground truth dissimilarity, participants’ dissimilarity perception was increased by caricature when faces differed in expression. We found these perceived dissimilarities to reflect the “representational geometry” of the ground truth. However, neither of these findings held for faces differing in identity. These findings replicated across two paradigms: pairwise ratings and multiarrangement. In a final study, motion caricature did not improve recognition memory for identity, whether manipulated at study or test. We report several forms of converging evidence for spatiotemporal caricature effects on dissimilarity perception of different expressions. However, more work needs to be done to discover what identity-specific movements can enhance face identification.

Emerged human-like facial expression representation in a deep convolutional neural network.

Recent studies found that the deep convolutional neural networks (DCNNs) trained to recognize facial identities spontaneously learned features that support facial expression recognition, and vice versa. Here, we showed that the self-emerged expression-selective units in a VGG-Face trained for facial identification were tuned to distinct basic expressions and, importantly, exhibited hallmarks of human expression recognition (i.e., facial expression confusion and categorical perception). We then investigated whether the emergence of expression-selective units is attributed to either face-specific experience or domain-general processing by conducting the same analysis on a VGG-16 trained for object classification and an untrained VGG-Face without any visual experience, both having the identical architecture with the pretrained VGG-Face. Although similar expression-selective units were found in both DCNNs, they did not exhibit reliable human-like characteristics of facial expression perception. Together, these findings revealed the necessity of domain-specific visual experience of face identity for the development of facial expression perception, highlighting the contribution of nurture to form human-like facial expression perception.

Simulating dynamic facial expressions of pain from visuo-haptic interactions with a robotic patient

Medical training simulators can provide a safe and controlled environment for medical students to practice their physical examination skills. An important source of information for physicians is the visual feedback of involuntary pain facial expressions in response to physical palpation on an affected area of a patient. However, most existing robotic medical training simulators that can capture physical examination behaviours in real-time cannot display facial expressions and comprise a limited range of patient identities in terms of ethnicity and gender. Together, these limitations restrict the utility of medical training simulators because they do not provide medical students with a representative sample of pain facial expressions and face identities, which could result in biased practices. Further, these limitations restrict the utility of such medical simulators to detect and correct early signs of bias in medical training. Here, for the first time, we present a robotic system that can simulate facial expressions of pain in response to palpations, displayed on a range of patient face identities. We use the unique approach of modelling dynamic pain facial expressions using a data-driven perception-based psychophysical method combined with the visuo-haptic inputs of users performing palpations on a robot medical simulator. Specifically, participants performed palpation actions on the abdomen phantom of a simulated patient, which triggered the real-time display of six pain-related facial Action Units (AUs) on a robotic face (MorphFace), each controlled by two pseudo randomly generated transient parameters: rate of change beta and activation delay tau . Participants then rated the appropriateness of the facial expression displayed in response to their palpations on a 4-point scale from “strongly disagree” to “strongly agree”. Each participant (n=16, 4 Asian females, 4 Asian males, 4 White females and 4 White males) performed 200 palpation trials on 4 patient identities (Black female, Black male, White female and White male) simulated using MorphFace. Results showed facial expressions rated as most appropriate by all participants comprise a higher rate of change and shorter delay from upper face AUs (around the eyes) to those in the lower face (around the mouth). In contrast, we found that transient parameter values of most appropriate-rated pain facial expressions, palpation forces, and delays between palpation actions varied across participant-simulated patient pairs according to gender and ethnicity. These findings suggest that gender and ethnicity biases affect palpation strategies and the perception of pain facial expressions displayed on MorphFace. We anticipate that our approach will be used to generate physical examination models with diverse patient demographics to reduce erroneous judgments in medical students, and provide focused training to address these errors.

The aftereffect of the ensemble average of facial expressions on subsequent facial expression recognition

An ensemble or statistical summary can be extracted from facial expressions presented in different spatial locations simultaneously. However, how such complicated objects are represented in the mind is not clear. It is known that the aftereffect of facial expressions, in which prolonged viewing of facial expressions biases the perception of subsequent facial expressions of the same category, occurs only when a visual representation is formed. Using this methodology, we examined whether an ensemble can be represented with visualized information. Experiment 1 revealed that the presentation of multiple facial expressions biased the perception of subsequent facial expressions to less happy as much as the presentation of a single face did. Experiment 2 compared the presentation of faces comprising strong and weak intensities of emotional expressions with an individual face as the adaptation stimulus. The results indicated that the perceptual biases were found after the presentation of four faces and a strong single face, but not after the weak single face presentation. Experiment 3 employed angry expressions, a distinct category from the test expression used as an adaptation stimulus; no aftereffect was observed. Finally, Experiment 4 clearly demonstrated the perceptual bias with a higher number of faces. Altogether, these results indicate that an ensemble average extracted from multiple faces leads to the perceptual bias, and this effect is similar in terms of its properties to that of a single face. This supports the idea that an ensemble of faces is represented with visualized information as a single face.

Face masks have emotion-dependent dissociable effects on accuracy and confidence in identifying facial expressions of emotion

The coronavirus pandemic has resulted in increased use of face masks worldwide. Here, we examined the effect of wearing a face mask on the ability to recognise facial expressions of emotion. In a within-subjects design, 100 UK-based undergraduate students were shown facial expressions of anger, disgust, fear, happiness, sadness, and neutral expression; these were either posed with or without a face mask, or with a face mask artificially imposed onto them. Participants identified the emotion portrayed in the photographs from a fixed choice array of answers and rated their confidence in their selection. While overall accuracy was higher without than with masks, the effect varied across emotions, with a clear advantage without masks in disgust, happiness, and sadness; no effect for neutral, and lower accuracy without masks for anger and fear. In contrast, confidence was generally higher without masks, with the effect clear for all emotions other than anger. These results confirm that emotion recognition is affected by face mask wearing, but reveal that the effect depends on the emotion being displayed—with this emotion-dependence not reflected in subjects’ confidence. The disparity between the effects of mask wearing on different emotions and the failure of this to be reflected in confidence ratings suggests that mask wearing not only effects emotion recognition, but may also create biases in the perception of facial expressions of emotion of which perceivers are unaware. In addition, the similarity of results between the Imposed Mask and Posed Mask conditions suggests that prior research using artificially imposed masks has not been deleteriously affected by the use of this manipulation.

Personality and the accurate perception of facial emotion expressions: What is accuracy and how does it matter?

Research into Emotion Decoding Accuracy (EDA) has revealed limited associations with personality. One possible reason could be the neglect of social context influences on the perception of emotions, which is problematic given the interplay of personality with social context. We propose a novel way to understand accuracy in emotion perception, which includes social context and the distinction between accuracy (perceiving the intended emotions) and inaccuracy (perceiving additional emotions to those expressed). In seven studies that utilized three methods, we found that personality traits that tap the social domain, consistently relate, in a meaningful way, to accuracy and inaccuracy in emotion perception. Accuracy and inaccuracy capture different aspects of the variance and do so more than traditional, hit rate based, methods and tests for assessing the accurate decoding of facial emotion expressions. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Are adaptation aftereffects for facial emotional expressions affected by prior knowledge about the emotion?

ABSTRACT Accurate perception of the emotional signals conveyed by others is crucial for successful social interaction. Such perception is influenced not only by sensory input, but also by knowledge we have about the others’ emotions. This study addresses the issue of whether knowing that the other’s emotional state is congruent or incongruent with their displayed emotional expression (“genuine” and “fake”, respectively) affects the neural mechanisms underpinning the perception of their facial emotional expressions. We used a visual adaptation paradigm to investigate this question in three experiments employing increasing adaptation durations. The adapting stimuli consisted of photographs of emotional facial expressions of joy and anger, purported to reflect (in-)congruency between felt and expressed emotion, displayed by professional actors. A Validity checking procedure ensured participants had the correct knowledge about the (in-)congruency. Significantly smaller adaptation aftereffects were obtained when participants knew that the displayed expression was incongruent with the felt emotion, following all tested adaptation periods. This study shows that knowledge relating to the congruency between felt and expressed emotion modulates face expression aftereffects. We argue that this reflects that the neural substrate responsible for the perception of facial expressions of emotion incorporates the presumed felt emotion underpinning the expression.