Decoding Of Facial Expressions Research Articles

Assessing the effectiveness of spatial PCA on SVM-based decoding of EEG data

Principal component analysis (PCA) has been widely employed for dimensionality reduction prior to multivariate pattern classification (decoding) in EEG research. The goal of the present study was to provide an evaluation of the effectiveness of PCA on decoding accuracy (using support vector machines) across a broad range of experimental paradigms. We evaluated several different PCA variations, including group-based and subject-based component decomposition and the application of Varimax rotation or no rotation. We also varied the numbers of PCs that were retained for the decoding analysis. We evaluated the resulting decoding accuracy for seven common event-related potential components (N170, mismatch negativity, N2pc, P3b, N400, lateralized readiness potential, and error-related negativity). We also examined more challenging decoding tasks, including decoding of face identity, facial expression, stimulus location, and stimulus orientation. The datasets also varied in the number and density of electrode sites. Our findings indicated that none of the PCA approaches consistently improved decoding performance related to no PCA, and the application of PCA frequently reduced decoding performance. Researchers should therefore be cautious about using PCA prior to decoding EEG data from similar experimental paradigms, populations, and recording setups.

Detection of emotional faces: The role of spatial frequencies and local features

Models of emotion processing suggest that threat-related stimuli such as fearful faces can be detected based on the rapid extraction of low spatial frequencies. However, this remains debated as other models argue that the decoding of facial expressions occurs with a more flexible use of spatial frequencies. The purpose of this study was to clarify the role of spatial frequencies and differences in luminance contrast between spatial frequencies, on the detection of facial emotions. We used a saccadic choice task in which emotional-neutral face pairs were presented and participants were asked to make a saccade toward the neutral or the emotional (happy or fearful) face. Faces were displayed either in low, high, or broad spatial frequencies. Results showed that participants were better to saccade toward the emotional face. They were also better for high or broad than low spatial frequencies, and the accuracy was higher with a happy target. An analysis of the eye and mouth saliency ofour stimuli revealed that the mouth saliency of the target correlates with participants’ performance. Overall, this study underlines the importance of local more than global information, and of the saliency of the mouth region in the detection of emotional and neutral faces.

The other-race effect in facial expression processing: Behavioral and ERP evidence from a balanced cross-cultural study in women.

Although evidence for cultural variants in facial expression decoding is accumulating, the other-race effect in facial expression processing and its neural correlates are still unclear. We investigated this question with a fully balanced design, in which a group of East Asian and a group of European Caucasian women categorized pictures of sad, happy, angry, and neutral facial expressions posed by individuals of their own-race and the other-race. Results revealed a disadvantage in categorizing expressions of anger in other-race faces in both samples, and for sad expressions in the European sample only. Partially consistent, East Asian participants showed longer latency of the N170 component in the event-related potential (ERP) and European Caucasian participants showed larger N170 amplitudes to other-race faces. The late positive complex in the ERP was less distinguishable among other-race facial expressions. Therefore, the present study observed an other-race effect in early and late stages of face processing, reflecting less efficient structural encoding and less elaborate processing for other-race than own-race faces.

Pain facial expression decoding is tuned to same-race faces

Pain facial expression decoding is tuned to same-race faces

How gender affects the decoding of facial expressions of pain.

Gender has been suggested to play a critical role in how facial expressions of pain are perceived by others. With the present study we aim to further investigate how gender might impact the decoding of facial expressions of pain, (i)by varying both the gender of the observer as well as the gender of the expressor and (ii) by considering two different aspects of the decoding process, namely intensity decoding and pain recognition. In two online-studies, videos of facial expressions of pain as well as of anger and disgust displayed by male and female avatars were presented to male and female participants. In the first study, valence and arousal ratings were assessed (intensity decoding) and in the second study, participants provided intensity ratings for different affective states, that allowed for assessing intensity decoding as well as pain recognition. The gender of the avatar significantly affected theintensity decoding of facial expressions of pain, with higher ratings (arousal, valence, pain intensity) for female compared to male avatars. In contrast, the gender of the observer had no significant impact on intensity decoding. With regard to pain recognition (differentiating pain from anger and disgust), neither the gender of the avatar, nor the gender of the observer had any affect. Only the gender of the expressor seems tohave a substantial impact on the decoding of facial expressions of pain, whereas the gender of the observer seems of less relevance. Reasons for the tendency to see more pain in female faces might be due to psychosocial factors (e.g., gender stereotypes) and require further research.

Don't look at me like that: Integration of gaze direction and facial expression.

Efficient decoding of facial expressions and gaze direction supports reactions to social environments. Although both cues are processed fast and accurately, when and how these cues are integrated is still debated. We investigated the temporal integration of gaze and emotion cues. Participants responded to letters that were randomly presented on four faces. Two of these faces initially showed direct gaze, two showed averted gaze. Upon target presentation, two faces changed gaze direction (from averted to direct and vice versa). Simultaneously, facial expressions changed from neutral to either an approach- or an avoidance-oriented emotion expression (Experiment 1a: angry/fearful; Experiment 1b: happy/disgusted). Although angry and fearful expressions diminished any effects of gaze direction (Experiment 1a), a direct gaze advantage was found for happy and an averted gaze advantage for disgusted faces (Experiment 1b). This pattern is consistent with hypotheses suggesting a processing benefit when emotion expression and gaze information are congruent in terms of approach- or avoidance-orientation. In Experiment 2, we tracked eye movements and, again, found evidence for an approach-avoidance-congruency advantage for happy and disgusted faces both in performance and gaze behavior. Gaze behavior analyses suggested an integration of gaze and emotion information that was already visible from 300 ms after target onset. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Clinical and structural brain correlates of hypomimia in early-stage Parkinson's disease.

Reduced facial expression of emotions is a very frequent symptom of Parkinson's disease (PD) and has been considered part of the motor features of the disease. However, the neural correlates of hypomimia and the relationship between hypomimia and other non-motor symptoms of PD are poorly understood. The clinical and structural brain correlates of hypomimia were studied. For this purpose, cross-sectional data from the COPPADIS study database were used. Age, disease duration, levodopa equivalent daily dose, Unified Parkinson's Disease Rating Scale part III (UPDRS-III), severity of apathy and depression and global cognitive status were collected. At the imaging level, analyses based on gray matter volume and cortical thickness were used. After controlling for multiple confounding variables such as age or disease duration, the severity of hypomimia was shown to be indissociable from the UPDRS-III speech and bradykinesia items and was significantly related to the severity of apathy (β=0.595; p < 0.0001). At the level of neural correlates, hypomimia was related to motor regions brodmann area 8 (BA 8) and to multiple fronto-temporo-parietal regions involved in the decoding, recognition and production of facial expression of emotions. Reduced facial expressivity in PD is related to the severity of symptoms of apathy and is mediated by the dysfunction of brain systems involved in motor control and in the recognition, integration and expression of emotions. Therefore, hypomimia in PD may be conceptualized not exclusively as a motor symptom but as a consequence of a multidimensional deficit leading to a symptom where motor and non-motor aspects converge.

Surgical face masks do not impair the decoding of facial expressions of negative affect more severely in older than in younger adults

Surgical face masks reduce the spread of airborne pathogens but also disturb the flow of information between individuals. The risk of getting seriously ill after infection with SARS-COV-2 during the present COVID-19 pandemic amplifies with age, suggesting that face masks should be worn especially during face-to-face contact with and between older people. However, the ability to accurately perceive and understand communication signals decreases with age, and it is currently unknown whether face masks impair facial communication more severely in older people. We compared the impact of surgical face masks on dynamic facial emotion recognition in younger (18–30 years) and older (65–85 years) adults (N = 96) in an online study. Participants watched short video clips of young women who facially expressed anger, fear, contempt or sadness. Faces of half of the women were covered by a digitally added surgical face mask. As expected, emotion recognition accuracy declined with age, and face masks reduced emotion recognition accuracy in both younger and older participants. Unexpectedly, the effect of face masks did not differ between age groups. Further analyses showed that masks also reduced the participants’ overall confidence in their emotion judgements, but not their performance awareness (the difference between their confidence ratings for correct and incorrect responses). Again, there were no mask-by-age interactions. Finally, data obtained with a newly developed questionnaire (attitudes towards face masks, atom) suggest that younger and older people do not differ in how much they feel impaired in their understanding of other people’s emotions by face masks or how useful they find face masks in confining the COVID-19 pandemic. In sum, these findings do not provide evidence that the impact of face masks on the decoding of facial signals is disproportionally larger in older people.

Personality and the accurate perception of facial emotion expressions: What is accuracy and how does it matter?

Research into Emotion Decoding Accuracy (EDA) has revealed limited associations with personality. One possible reason could be the neglect of social context influences on the perception of emotions, which is problematic given the interplay of personality with social context. We propose a novel way to understand accuracy in emotion perception, which includes social context and the distinction between accuracy (perceiving the intended emotions) and inaccuracy (perceiving additional emotions to those expressed). In seven studies that utilized three methods, we found that personality traits that tap the social domain, consistently relate, in a meaningful way, to accuracy and inaccuracy in emotion perception. Accuracy and inaccuracy capture different aspects of the variance and do so more than traditional, hit rate based, methods and tests for assessing the accurate decoding of facial emotion expressions. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Infusing Context Into Emotion Perception Impacts Emotion Decoding Accuracy.

. The accurate decoding of facial emotion expressions lies at the center of many research traditions in psychology. Much of this research, while paying lip service to the importance of context in emotion perception, has used stimuli that were carefully created to be deprived of contextual information. The participants' task is to associate the expression shown in the face with a correct label, essentially changing a social perception task into a cognitive task. In fact, in many cases, the task can be carried out correctly without engaging emotion recognition at all. The present article argues that infusing context in emotion perception does not only add an additional source of information but changes the way that participants approach the task by rendering it a social perception task rather than a cognitive task. Importantly, distinguishing between accuracy (perceiving the intended emotions) and bias (perceiving additional emotions to those intended) leads to a more nuanced understanding of social emotion perception. Results from several studies that use the Assessment of Contextual Emotions demonstrate the significance and social functionality of simultaneously considering emotion decoding accuracy and bias for social interaction in different cultures, their key personality and societal correlates, and their function for close relationships processes.

EEG Decoding of Dynamic Facial Expressions of Emotion: Evidence from SSVEP and Causal Cortical Network Dynamics

The neural cognitive mechanism in processing static facial expressions (FEs) has been well documented, whereas the one underlying perceiving dynamic faces remains unclear. In this study, Fourier transformation and time–frequency analysis of Electroencephalography (EEG) data were carried out to detect the brain activation underlying dynamic or static FEs while twenty-one participants were viewing dynamic or static faces flicking at 10 Hz. In particular, steady-state visual evoked potentials (SSVEPs) were quantified through spectral power analysis of EEG recordings. Besides, Granger causality (GC) analysis (GCA) was also performed to capture the causal cortical network dynamics during dynamic or static FEs of emotion. It was discovered that the dynamic (from neural to happy (N2H) or vice versa (H2N)) FEs elicited larger SSVEPs than the static ones. Additionally, GCA demonstrated that the H2N case, in which happy FEs were being gradually changed into neutral ones, exhibited larger GC measure during the late processing stage than that from the early stage. Consequently, enhanced SSVEPs and effective brain connectivity for dynamic FEs illustrated that participants might need consume more attentional resources to process the dynamic faces, particularly for the change from happy to neutral faces. The new neural index might facilitate us to better understand the cognitive processing of dynamic and static FEs.

Decoding of facial expressions of pain in avatars: does sex matter?

The decoding of facial expressions of pain plays a crucial role in pain diagnostic and clinical decision making. For decoding studies, it is necessary to present facial expressions of pain in a flexible and controllable fashion. Computer models (avatars) of human facial expressions of pain allow for systematically manipulating specific facial features. The aim of the present study was to investigate whether avatars can show realistic facial expressions of pain and how the sex of the avatars influence the decoding of pain by human observers. For that purpose, 40 female (mean age: 23.9 years) and 40 male (mean age: 24.6 years) observers watched 80 short videos showing computer-generated avatars, who presented the five clusters of facial expressions of pain (four active and one stoic cluster) identified by Kunz and Lautenbacher (2014). After each clip, observers were asked to provide ratings for the intensity of pain the avatars seem to experience and the certainty of judgement, i.e. if the shown expression truly represents pain. Results show that three of the four active facial clusters were similarly accepted as valid expressions of pain by the observers whereas only one cluster ("raised eyebrows") was disregarded. The sex of the observed avatars influenced the decoding of pain as indicated by increased intensity and elevated certainty ratings for femaleavatars. The assumption of different valid facial expressions of pain could be corroborated in avatars, which contradicts the idea of only one uniform pain face. The observers' rating of the avatars' pain was influenced by the avatars' sex, which resembles known observer biases for humans. The use of avatars appeared to be a suitable method in research on the decoding of the facial expression of pain, mirroring closely the known forms of human facial expressions.

Προσανατολισμός δεσμού και κοινωνική αντίληψη του συναισθήματος: Ο ρόλος των κοινωνικών κινήτρων

Despite the seminal role of emotion perception in social and personal relationships, there is limited understanding of how adult attachment organization affects the decoding of facial emotion expressions. Previousresearch has focused on how insecure attachment-related strategies for emotion regulation influence early stages of emotion information processing. However, recent studies highlight the importance of socialprocesses and motivational factors in the perception of positive and negative emotion (see e.g., Vrtička, Sander, & Vuilleumier, 2012). Based on a critical review of the relevant literature, the present articlepresents findings from a recent series of studies that reveal the effects senders' social moti ves (relational context, social goals) have on emotion decoding accuracy in adults with insecure attachment. The findings from these studies are discussed in the context of theories of motivated social cognition and the social perception of emotion.

Frequency of Recent Binge Drinking Is Associated With Sex-Specific Cognitive Deficits: Evidence for Condition-Dependent Trait Expression in Humans.

Evolutionary theory suggests that commonly found sex differences are largest in healthy populations and smaller in populations that have been exposed to stressors. We tested this idea in the context of men’s typical advantage (vs. women) in visuospatial abilities (e.g., mental rotation) and women’s typical advantage (vs. men) in social-cognitive (e.g., facial-expression decoding) abilities, as related to frequent binge drinking. Four hundred nineteen undergraduates classified as frequent or infrequent binge drinkers were assessed in these domains. Trial-level multilevel models were used to test a priori Sex × Group (binge drinking) interactions for visuospatial and social-cognitive tasks. Among infrequent binge drinkers, men’s typical advantage in visuospatial abilities and women’s typical advantage in social-cognitive abilities was confirmed. Among frequent binge drinkers, men’s advantage was reduced for one visuospatial task (Δ d = 0.29) and eliminated for another (Δ d = 0.75), and women’s advantage on the social-cognitive task was eliminated (Δ d = 0.12). Males who frequently engaged in extreme binges had exaggerated deficits on one of the visuospatial tasks, as did their female counterparts on the social-cognitive task. The results suggest sex-specific vulnerabilities associated with recent, frequent binge drinking, and support an evolutionary approach to the study of these vulnerabilities.

The Time Course of Facial Expression Recognition Using Spatial Frequency Information: Comparing Pain and Core Emotions

We are able to recognize others’ experience of pain from their facial expressions. However, little is known about what makes the recognition of pain possible and whether it is similar or different from core emotions. This study investigated the mechanisms underpinning the recognition of pain expressions, in terms of spatial frequency (SF) information analysis, and compared pain with 2 core emotions (ie, fear and happiness). Two experiments using a backward masking paradigm were conducted to examine the time course of low- and high-SF information processing, by manipulating the presentation duration of face stimuli and target-mask onset asynchrony. Overall, we found a temporal advantage of low-SF over high-SF information for expression recognition, including pain. This asynchrony between low- and high-SF happened at a very early stage of information extraction, which indicates that the decoding of low-SF expression information is not only faster but possibly occurs before the processing of high-SF information. Interestingly, the recognition of pain was also found to be slower and more difficult than core emotions. It is suggested that more complex decoding process may be involved in the successful recognition of pain from facial expressions, possibly due to the multidimensional nature of pain experiences. PerspectiveTwo studies explore the perceptual and temporal properties of the decoding of pain facial expressions. At very early stages of attention, the recognition of pain was found to be more difficult than fear and happiness. It suggests that pain is a complex expression, and requires additional time to detect and process.

Visual Behavior, Pupil Dilation, and Ability to Identify Emotions From Facial Expressions After Stroke.

Social cognition is the innate human ability to interpret the emotional state of others from contextual verbal and non-verbal information, and to self-regulate accordingly. Facial expressions are one of the most relevant sources of non-verbal communication, and their interpretation has been extensively investigated in the literature, using both behavioral and physiological measures, such as those derived from visual activity and visual responses. The decoding of facial expressions of emotion is performed by conscious and unconscious cognitive processes that involve a complex brain network that can be damaged after cerebrovascular accidents. A diminished ability to identify facial expressions of emotion has been reported after stroke, which has traditionally been attributed to impaired emotional processing. While this can be true, an alteration in visual behavior after brain injury could also negatively contribute to this ability. This study investigated the accuracy, distribution of responses, visual behavior, and pupil dilation of individuals with stroke while identifying emotional facial expressions. Our results corroborated impaired performance after stroke and exhibited decreased attention to the eyes, evidenced by a diminished time and number of fixations made in this area in comparison to healthy subjects and comparable pupil dilation. The differences in visual behavior reached statistical significance in some emotions when comparing individuals with stroke with impaired performance with healthy subjects, but not when individuals post-stroke with comparable performance were considered. The performance dependence of visual behavior, although not determinant, might indicate that altered visual behavior could be a negatively contributing factor for emotion recognition from facial expressions.

Identifying Liars Through Automatic Decoding of Children's Facial Expressions.

This study explored whether children's (N=158; 4- to 9 years old) nonverbal facial expressions can be used to identify when children are being deceptive. Using a computer vision program to automatically decode children's facial expressions according to the Facial Action Coding System, this study employed machine learning to determine whether facial expressions can be used to discriminate between children who concealed breaking a toy(liars) and those who did not break a toy(nonliars). Results found that, regardless of age or history of maltreatment, children's facial expressions could accurately (73%) be distinguished between liars and nonliars. Two emotions, surprise and fear, were more strongly expressed by liars than nonliars. These findings provide evidence to support the use of automatically coded facial expressions to detect children's deception.

Network Representations of Facial and Bodily Expressions: Evidence From Multivariate Connectivity Pattern Classification.

Emotions can be perceived from both facial and bodily expressions. Our previous study has found the successful decoding of facial expressions based on the functional connectivity (FC) patterns. However, the role of the FC patterns in the recognition of bodily expressions remained unclear, and no neuroimaging studies have adequately addressed the question of whether emotions perceiving from facial and bodily expressions are processed rely upon common or different neural networks. To address this, the present study collected functional magnetic resonance imaging (fMRI) data from a block design experiment with facial and bodily expression videos as stimuli (three emotions: anger, fear, and joy), and conducted multivariate pattern classification analysis based on the estimated FC patterns. We found that in addition to the facial expressions, bodily expressions could also be successfully decoded based on the large-scale FC patterns. The emotion classification accuracies for the facial expressions were higher than that for the bodily expressions. Further contributive FC analysis showed that emotion-discriminative networks were widely distributed in both hemispheres, containing regions that ranged from primary visual areas to higher-level cognitive areas. Moreover, for a particular emotion, discriminative FCs for facial and bodily expressions were distinct. Together, our findings highlight the key role of the FC patterns in the emotion processing, indicating how large-scale FC patterns reconfigure in processing of facial and bodily expressions, and suggest the distributed neural representation for the emotion recognition. Furthermore, our results also suggest that the human brain employs separate network representations for facial and bodily expressions of the same emotions. This study provides new evidence for the network representations for emotion perception and may further our understanding of the potential mechanisms underlying body language emotion recognition.

Facial Expression Decoding based on fMRI Brain Signal

The analysis of facial expressions is a hot topic in brain-computer interface research. To determine the facial expressions of the subjects under the corresponding stimulation, we analyze the fMRI images acquired by the Magnetic Resonance. There are six kinds of facial expressions: "anger", "disgust", "sadness", "happiness", "joy" and "surprise". We demonstrate that brain decoding is achievable through the parsing of two facial expressions ("anger" and "joy"). Support vector machine and extreme learning machine are selected to classify these expressions based on time series features. Experimental results show that the classification performance of the extreme learning machine algorithm is better than support vector machine. Among the eight participants in the trials, the classification accuracy of three subjects reached 70-80%, and the remaining five subjects also achieved accuracy of 50-60%. Therefore, we can conclude that the brain decoding can be used to help analyzing human facial expressions.

Emotional Awareness and Cognitive Performance in Borderline Intellectual Functioning Young Adolescents.

According to DSM-5 and ICD-10, borderline intellectual functioning (BIF) should not be classified properly as a disorder. However, BIF people may present relevant problems of adaptive functioning in several areas of daily activities, and they seem to be more vulnerable to mental diseases. Young adolescence may be considered a particular period for emotional information processing. The "own and others' emotions" awareness can play a crucial role in many daily life situations, such as decision making, interpersonal relationships, and decoding of facial expressions. On this background, a BIF young adolescents group underwent a neuropsychological assessment including emotional and cognitive domains, and was compared with a healthy young adolescents control group (HC). In the overall sample, a significant negative correlation between general intellectual abilities and emotional awareness was found. The BIF group showed a significantly greater level of alexithymia and a poorer performance in higher cognitive tasks than HC group. As hypothesized, a border cognitive functioning influences mentalization processes as ability to discriminate and monitor emotions, as well as higher domains of cognition.