Perception Of Facial Expressions Research Articles

Facial icons as indexes of emotions and intentions

Various objects and artifacts incorporate representations of faces, encompassing artworks like portraits, as well as ethnographic or industrial artifacts such as masks or humanoid robots. These representations exhibit diverse degrees of human-likeness, serving different functions and objectives. Despite these variations, they share common features, particularly facial attributes that serve as building blocks for facial expressions—an effective means of communicating emotions. To provide a unified conceptualization for this broad spectrum of face representations, we propose the term “facial icons” drawing upon Peirce’s semiotic concepts. Additionally, based on these semiotic principles, we posit that facial icons function as indexes of emotions and intentions, and introduce a significant anthropological theory aligning with our proposition. Subsequently, we support our assertions by examining processes related to face and facial expression perception, as well as sensorimotor simulation processes involved in discerning others’ mental states, including emotions. Our argumentation integrates cognitive and experimental evidence, reinforcing the pivotal role of facial icons in conveying mental states.

Consistent social information perceived in animated backgrounds improves ensemble perception of facial expressions.

Observers can rapidly extract the mean emotion from a set of faces with remarkable precision, known as ensemble coding. Previous studies have demonstrated that matched physical backgrounds improve the precision of ongoing ensemble tasks. However, it remains unknown whether this facilitation effect still occurs when matched social information is perceived from the backgrounds. In two experiments, participants decided whether the test face in the retrieving phase appeared more disgusted or neutral than the mean emotion of the face set in the encoding phase. Both phases were paired with task-irrelevant animated backgrounds, which included either the forward movement trajectory carrying the "cooperatively chasing" information, or the backward movement trajectory conveying no such chasing information. The backgrounds in the encoding and retrieving phases were either mismatched (i.e., forward and backward replays of the same trajectory), or matched (i.e., two identical forward movement trajectories in Experiment 1, or two different forward movement trajectories in Experiment 2). Participants in both experiments showed higher ensemble precisions and better discrimination sensitivities when backgrounds matched. The findings suggest that consistent social information perceived from memory-related context exerts a context-matching facilitation effect on ensemble coding, and more importantly, this effect is independent of consistent physical information.

Attention modulates facial expression processing in subsyndromal depression: A behavioral and ERP study

Impaired facial expression perception is a core element in depression, but the underlying mechanism remains controversial. This event-related potential study investigated how attention modulates facial expression perception in depression using a nonclinical sample. A group of healthy controls (HC, N = 39) and a group of individuals with subsyndromal depression (SD, N = 39) categorized faces based on either facial expression (happy vs. sad) or gender (male vs. female). Behaviorally, the SD group was less sensitive to the emotional valence of facial expression than the HC group when their attention was directed to facial expression, as revealed by comparable subjective ratings and accuracy rates in response to facial expressions. When attention was directed towards facial gender, the SD group versus the HC group showed a negative bias, as revealed by a faster N170 for sad faces than happy faces. Together, our findings suggest that attention plays a role in understanding the relationship between depression and facial expression perception.

Visual perception of facial expressions of emotions and alexithymia among people suffering from schizophrenia

Aim of the studyThe aim of the study is to find out whether there is a relationship between the severity of alexithymia and the visual perception of facial expressions of emotions among people suffering from schizophrenia.Subject or material and methods41 people suffering from schizophrenia and 41 healthy people participated in the study. In the first stage, the subjects completed the Toronto Alexithymia Scale (TAS-20) questionnaire. The second stage was based on eye-tracking measurements made during a computer experiment of recognising facial expressions.ResultsPeople with schizophrenia showed significantly higher intensity of alexithymia than healthy people. In the group of patients with schizophrenia, a negative correlation between alexithymia and the length of saccades on the nose was found.DiscussionIn the group of people with schizophrenia, alexithymia is associated with less intensive exploration of the nasal area. A wrinkled nose is an important area, first of all, in the manifestation of negative emotions, in recognition of which the sick people make the most mistakes.ConclusionsProblems with recognising emotions in people suffering from schizophrenia are associated with neuronal dysfunction and abnormalities in the eye movement. These factors disrupt the early stages of sensory processing. The relationship of alexithymia with the visual perception of patients with schizophrenia suggests that top-down processes may also contribute to the development of these deficits.

On the Role of Sensorimotor Experience in Facial Expression Perception.

Humans recognize the facial expressions of others rapidly and effortlessly. Although much is known about how we perceive expressions, the role of facial experience in shaping this remarkable ability remains unclear. Is our perception of expressions linked to how we ourselves make facial expressions? Are we better at recognizing other's facial expressions if we are experts at making the same expressions ourselves? And if we could not make facial expressions at all, would it impact our ability to recognize others' facial expressions? The current article aims to examine these questions by explicating the link between facial experience and facial expression recognition. It includes a comprehensive appraisal of the related literature and examines three main theories that posit a connection between making and recognizing facial expressions. First, recent studies in individuals with Moebius syndrome support the role of facial ability (i.e., the ability to move one's face to make facial expressions) in facial expression recognition. Second, motor simulation theory suggests that humans recognize others' facial expressions by covertly mimicking the observed expression (without overt motor action) and that this facial mimicry helps us identify and feel the associated emotion. Finally, the facial feedback hypothesis provides a framework for enhanced emotional experience via proprioceptive feedback from facial muscles when mimicking a viewed facial expression. Evidence for and against these theories is presented as well as some considerations and outstanding questions for future research studies investigating the role of facial experience in facial expression perception.

Additive effects of emotional expression and stimulus size on the perception of genuine and artificial facial expressions: an ERP study

Seeing an angry individual in close physical proximity can not only result in a larger retinal representation of that individual and an enhanced resolution of emotional cues, but may also increase motivation for rapid visual processing and action preparation. The present study investigated the effects of stimulus size and emotional expression on the perception of happy, angry, non-expressive, and scrambled faces. We analyzed event-related potentials (ERPs) and behavioral responses of N = 40 participants who performed a naturalness classification task on real and artificially created facial expressions. While the emotion-related effects on accuracy for recognizing authentic expressions were modulated by stimulus size, ERPs showed only additive effects of stimulus size and emotional expression, with no significant interaction with size. This contrasts with previous research on emotional scenes and words. Effects of size were present in all included ERPs, whereas emotional expressions affected the N170, EPN, and LPC, irrespective of size. These results imply that the decoding of emotional valence in faces can occur even for small stimuli. Supra-additive effects in faces may necessitate larger size ranges or dynamic stimuli that increase arousal.

Perception and memory-based representations of facial emotions: Associations with personality functioning, affective states and recognition abilities

Personality traits and affective states are associated with biases in facial emotion perception. However, the precise personality impairments and affective states that underlie these biases remain largely unknown. To investigate how relevant factors influence facial emotion perception and recollection, Experiment 1 employed an image reconstruction approach in which community-dwelling adults (N = 89) rated the similarity of pairs of facial expressions, including those recalled from memory. Subsequently, perception- and memory-based expression representations derived from such ratings were assessed across participants and related to measures of personality impairment, state affect, and visual recognition abilities. Impairment in self-direction and level of positive affect accounted for the largest components of individual variability in perception and memory representations, respectively. Additionally, individual differences in these representations were impacted by face recognition ability. In Experiment 2, adult participants (N = 81) rated facial image reconstructions derived in Experiment 1, revealing that individual variability was associated with specific visual face properties, such as expressiveness, representation accuracy, and positivity/negativity. These findings highlight and clarify the influence of personality, affective state, and recognition abilities on individual differences in the perception and recollection of facial expressions.

A neurophysiological signature of dynamic emotion recognition associated with social communication skills and cortical gamma-aminobutyric acid levels in children.

Emotion recognition is a core feature of social perception. In particular, perception of dynamic facial emotional expressions is a major feature of the third visual pathway. However, the classical N170 visual evoked signal does not provide a pure correlate of such processing. Indeed, independent component analysis has demonstrated that the N170 component is already active at the time of the P100, and is therefore distorted by early components. Here we implemented, a dynamic face emotional paradigm to isolate a more pure face expression selective N170. We searched for a neural correlate of perception of dynamic facial emotional expressions, by starting with a face baseline from which a facial expression evolved. This allowed for a specific facial expression contrast signal which we aimed to relate with social communication abilities and cortical gamma-aminobutyric acid (GABA) levels. We recorded event-related potentials (ERPs) and Magnetic Resonance (MRS) measures in 35 typically developing (TD) children, (10-16 years) sex-matched, during emotion recognition of an avatar morphing/unmorphing from neutral to happy/sad expressions. This task allowed for the elimination of the contribution low-level visual components, in particular the P100, by morphing baseline isoluminant neutral faces into specific expressions, isolating dynamic emotion recognition. Therefore, it was possible to isolate a dynamic face sensitive N170 devoid of interactions with earlier components. We found delayed N170 and P300, with a hysteresis type of dependence on stimulus trajectory (morphing/unmorphing), with hemispheric lateralization. The delayed N170 is generated by an extrastriate source, which can be related to the third visual pathway specialized in biological motion processing. GABA levels in visual cortex were related with N170 amplitude and latency and predictive of worse social communication performance (SCQ scores). N170 latencies reflected delayed processing speed of emotional expressions and related to worse social communication scores. In sum, we found a specific N170 electrophysiological signature of dynamic face processing related to social communication abilities and cortical GABA levels. These findings have potential clinical significance supporting the hypothesis of a spectrum of social communication abilities and the identification of a specific face-expression sensitive N170 which can potentially be used in the development of diagnostic and intervention tools.

Influence of stimulus manipulation on conscious awareness of emotional facial expressions in the match-to-sample paradigm

The conscious perception of emotional facial expressions plays an indispensable role in social interaction. However, previous psychological studies have reported inconsistent findings regarding whether conscious awareness is greater for emotional expressions than for neutral expressions. Furthermore, whether this phenomenon is attributable to emotional or visual factors remains unknown. To investigate these issues, we conducted five psychological experiments to test the conscious perception of emotional and neutral facial expressions using the match-to-sample paradigm. Facial stimuli were momentarily presented in the peripheral visual fields while participants read simultaneously presented letters in the central visual fields. The participants selected a perceived face from nine samples. The results of all experiments demonstrated that emotional expressions were more accurately identified than neutral expressions. Furthermore, Experiment 4 showed that angry expressions were identified more accurately than anti-angry expressions, which expressed neutral emotions with comparable physical changes to angry expressions. Experiment 5, testing the interaction between emotional expression and face direction, showed that angry expressions looking toward participants were more accurately identified than those looking away from participants, even though they were physically identical. These results suggest that the conscious awareness of emotional facial expressions is enhanced by their emotional significance.

Trust in the police and affective evaluation of police faces: a preliminary study.

A study was conducted to investigate if an individual's trust in law enforcement affects their perception of the emotional facial expressions displayed by police officers. The study invited 77 participants to rate the valence of 360 face images. Images featured individuals without headgear (condition 1), or with a baseball cap (condition 2) or police hat (condition 3) digitally added to the original photograph. The images were balanced across sex, race/ethnicity (Asian, African American, Latine, and Caucasian), and facial expression (Happy, Neutral, and Angry). After rating the facial expressions, respondents completed a survey about their attitudes toward the police. The results showed that, on average, valence ratings for "Angry" faces were similar across all experimental conditions. However, a closer examination revealed that faces with police hats were perceived as angrier compared to the control conditions (those with no hat and those with a baseball cap) by individuals who held negative views of the police. Conversely, participants with positive attitudes toward the police perceived faces with police hats as less angry compared to the control condition. This correlation was highly significant for angry faces (p < 0.01), and stronger in response to male faces compared to female faces but was not significant for neutral or happy faces. The study emphasizes the substantial role of attitudes in shaping social perception, particularly within the context of law enforcement.

Inability to move one's face dampens facial expression perception

Humans rely heavily on facial expressions for social communication to convey their thoughts and emotions and to understand them in others. One prominent but controversial view is that humans learn to recognize the significance of facial expressions by mimicking the expressions of others. This view predicts that an inability to make facial expressions (e.g., facial paralysis) would result in reduced perceptual sensitivity to others' facial expressions. To test this hypothesis, we developed a diverse battery of sensitive emotion recognition tasks to characterize expression perception in individuals with Moebius Syndrome (MBS), a congenital neurological disorder that causes facial palsy. Using computer-based detection tasks we systematically assessed expression perception thresholds for static and dynamic face and body expressions. We found that while MBS individuals were able to perform challenging perceptual control tasks and body expression tasks, they were less efficient at extracting emotion from facial expressions, compared to matched controls. Exploratory analyses of fMRI data from a small group of MBS participants suggested potentially reduced engagement of the amygdala in MBS participants during expression processing relative to matched controls. Collectively, these results suggest a role for facial mimicry and consequent facial feedback and motor experience in the perception of others' facial expressions.

Effects of wearing an opaque or transparent face mask on the perception of facial expressions: A comparative study between Japanese school-aged children and adults.

The negative side effects of mask-wearing on reading facial emotional cues have been investigated in several studies with adults post-2020. However, little is known about children. This study aimed to determine the negative influence of mask-wearing on reading emotions of adult faces by Japanese school-aged children, compared to Japanese adults. We also examined whether this negative influence could be alleviated by using a transparent face mask instead of an opaque one (surgical mask). The performance on reading emotions was measured using emotion categorization and emotion intensity rating tasks for adult faces. As per the findings, the accuracy of emotion recognition in children was impaired for various facial expressions (disgust, fear, happy, neutral, sad, and surprise faces), except for angry faces. Conversely, in adults, it was impaired for a few facial expressions. The perceived intensity for happy faces with a surgical mask was weaker in both children and adults than in those without the mask. A negative influence of wearing surgical masks was generally not observed for faces wearing a transparent mask in both children and adults. Thus, negative side effects of mask-wearing on reading emotions are observed for more facial expressions in children than in adults; transparent masks can help remedy these.

The effect of facial colour on implicit facial expressions

ABSTRACT Humans recognise reddish-coloured faces as angry. However, does facial colour also affect “implicit” facial expression perception of which humans are not explicitly aware? In this study, we investigated the effects of facial colour on implicit facial expression perception. The experimental stimuli were “hybrid faces”, in which the low-frequency component of the neutral facial expression image was replaced with the low-frequency component of the facial expression image of happiness or anger. In Experiment 1, we confirmed that the hybrid face stimuli were perceived as neutral and, therefore, supported implicit facial expression perception. In Experiment 2, the hybrid face stimuli were adjusted to natural and reddish facial colours, and their friendliness ratings were compared. The results showed that the expression of happiness was rated as more friendly than the expression of anger. In addition, the expression of happiness was rated as friendlier when the low-frequency happy component was red, but the friendliness rating of the expression of anger did not change when it was presented in red. In Experiment 3, we affirmed the implicit facial expression perception even in reddish colours. These results suggest that facial colour modulates the perception of implicit facial expressions in hybrid facial stimuli.

Effects of emotion words activation and satiation on facial expression perception: evidence from behavioral and ERP investigations.

The present study investigated the impact of emotion concepts obtained from external environmental experiences on the perception of facial expressions by manipulating the activation and satiation of emotion words, which was based on the argument between basic emotion theory and constructed emotion theory. Experiment 1 explored the effects of emotion activation on happy, disgusted, emotion-label words and emotion-laden words in a facial expression judgment task through behavioral experimentation. Experiment 2 explored the effect of semantic satiation on emotion-label words and emotion-laden words using the event-related potential technique. Experiment 1 found that facial expression perception was influenced by both types of emotion words and showed a significant emotional consistency effect. Experiment 2 found that N170 exhibited a more negative amplitude in the consistent condition compared to the inconsistent condition in the right hemisphere. More importantly, in the later stage of facial expression processing, emotion-label words and emotion-laden words both obstructed the perception of disgusted facial expressions and elicited more negative N400 amplitude in the emotion consistency condition, showing a reversed N400 effect. These results suggested that emotion concepts in the form of language influenced the perception of facial expressions, but there were differences between happy and disgusted faces. Disgusted faces were more dependent on emotion concept information and showed different performances in semantic activation and satiation conditions.

Perception of facial expressions involves emotion specific somatosensory cortex activations which are shaped by alexithymia

Somatosensory cortex (SCx) has been shown to crucially contribute to early perceptual processes when judging other's emotional facial expressions. Here, we investigated the specificity of SCx activity to angry, happy, sad and neutral emotions and the role of personality factors. We assessed participants' alexithymia (TAS-20) and depression (BDI) levels, their cardioceptive abilities and recorded changes in neural activity in a facial emotion judgment task. During the task, we presented tactile probes to reveal neural activity in SCx which was then isolated from visual carry-over responses. We further obtain SCx emotion effects by subtracting SCx activity elicited by neutral emotion expressions from angry, happy, and sad expressions. We find preliminary evidence for distinct modulations of SCx activity to angry and happy expressions. Moreover, the SCx anger response was predicted by individual differences in trait alexithymia. Thus, emotion expressions of others may be distinctly presented in the observer's neural body representation and may be shaped by their personality trait.

Analysis of convolutional neural networks reveals the computational properties essential for subcortical processing of facial expression

Perception of facial expression is crucial for primate social interactions. This visual information is processed through the ventral cortical pathway and the subcortical pathway. However, the subcortical pathway exhibits inaccurate processing, and the responsible architectural and physiological properties remain unclear. To investigate this, we constructed and examined convolutional neural networks with three key properties of the subcortical pathway: a shallow layer architecture, concentric receptive fields at the initial processing stage, and a greater degree of spatial pooling. These neural networks achieved modest accuracy in classifying facial expressions. By replacing these properties, individually or in combination, with corresponding cortical features, performance gradually improved. Similar to amygdala neurons, some units in the final processing layer exhibited sensitivity to retina-based spatial frequencies (SFs), while others were sensitive to object-based SFs. Replacement of any of these properties affected the coordinates of the SF encoding. Therefore, all three properties limit the accuracy of facial expression information and are essential for determining the SF representation coordinate. These findings characterize the role of the subcortical computational processes in facial expression recognition.

I focus only when I see your fear-fearful faces are not prioritized by attention when processed outside of awareness.

The defensive reaction to threats consists of two components: non-specific physiological arousal and specific attentional prioritization of the threatening stimulus, both of which are assumed by the so-called "low-road" hypothesis to be induced automatically and unconsciously. Although ample evidence indicates that non-specific arousal can indeed be caused by unconscious threatening stimuli, data regarding the involvement of the attentional selection mechanism remain inconclusive. Therefore, in the present study we used ERPs to compare the potential engagement of attention in the perception of subliminal and supraliminal fearful facial expressions to that of neutral ones. In the conscious condition, fearful faces were preferentially encoded (as indicated by the N170 component) and prioritized by bottom-up (EPN) and spatial attention (N2pc) in an automatic, task-independent manner. Furthermore, consciously perceived fearful expressions engaged cognitive resources (SPCN, P3) when face stimuli were task-relevant. In the unconscious condition, fearful faces were still preferentially encoded (N170), but we found no evidence for any type of attentional prioritization. Therefore, by showing that threatening stimuli engage attention only when perceived consciously, our findings challenge the "low road" hypothesis and point to the limits of unconscious attentional selection.

Learning a crowd-powered perceptual distance metric for facial blendshapes

It is known that purely geometric distance metrics cannot reflect the human perception of facial expressions. A novel perceptually based distance metric designed for 3D facial blendshape models is proposed in this paper. To develop this metric, comparative evaluations of facial expressions were collected from a crowdsourcing experiment. Then, the weights of a distance metric, based on descriptive features of the models, were optimized to match the results with crowdsourced data, through a metric learning process. The method incorporates perceptual properties such as curvature and visual saliency. A formal analysis of the results proves the high correlation between the metric output and human perception. The effectiveness and success of the proposed metric were also compared to other distance alternatives. The proposed metric will enable intelligent processing of 3D facial blendshapes data in several ways. It will be possible to generate perceptually valid clustering and visualization of 3D facial blendshapes. It will help reduce storage and computational requirements by removing redundant expressions that are perceptually identical from the overall dataset. It can also be used to assist novice animators while creating plausible and expressive facial animations.

“When You’re Smiling”: How Posed Facial Expressions Affect Visual Recognition of Emotions

Facial imitation occurs automatically during the perception of an emotional facial expression, and preventing it may interfere with the accuracy of emotion recognition. In the present fMRI study, we evaluated the effect of posing a facial expression on the recognition of ambiguous facial expressions. Since facial activity is affected by various factors, such as empathic aptitudes, the Interpersonal Reactivity Index (IRI) questionnaire was administered and scores were correlated with brain activity. Twenty-six healthy female subjects took part in the experiment. The volunteers were asked to pose a facial expression (happy, disgusted, neutral), then to watch an ambiguous emotional face, finally to indicate whether the emotion perceived was happiness or disgust. As stimuli, blends of happy and disgusted faces were used. Behavioral results showed that posing an emotional face increased the percentage of congruence with the perceived emotion. When participants posed a facial expression and perceived a non-congruent emotion, a neural network comprising bilateral anterior insula was activated. Brain activity was also correlated with empathic traits, particularly with empathic concern, fantasy and personal distress. Our findings support the idea that facial mimicry plays a crucial role in identifying emotions, and that empathic emotional abilities can modulate the brain circuits involved in this process.

Face-Selective Patches in Marmosets Are Involved in Dynamic and Static Facial Expression Processing.

The correct identification of facial expressions is critical for understanding the intention of others during social communication in the daily life of all primates. Here we used ultra-high-field fMRI at 9.4 T to investigate the neural network activated by facial expressions in awake New World common marmosets from both male and female sex, and to determine the effect of facial motions on this network. We further explored how the face-patch network is involved in the processing of facial expressions. Our results show that dynamic and static facial expressions activate face patches in temporal and frontal areas (O, PV, PD, MD, AD, and PL) as well as in the amygdala, with stronger responses for negative faces, also associated with an increase of the respiration rates of the monkey. Processing of dynamic facial expressions involves an extended network recruiting additional regions not known to be part of the face-processing network, suggesting that face motions may facilitate the recognition of facial expressions. We report for the first time in New World marmosets that the perception and identification of changeable facial expressions, vital for social communication, recruit face-selective brain patches also involved in face detection processing and are associated with an increase of arousal.SIGNIFICANCE STATEMENT Recent research in humans and nonhuman primates has highlighted the importance to correctly recognize and process facial expressions to understand others' emotions in social interactions. The current study focuses on the fMRI responses of emotional facial expressions in the common marmoset (Callithrix jacchus), a New World primate species sharing several similarities of social behavior with humans. Our results reveal that temporal and frontal face patches are involved in both basic face detection and facial expression processing. The specific recruitment of these patches for negative faces associated with an increase of the arousal level show that marmosets process facial expressions of their congener, vital for social communication.