Facial Expressions Of Basic Emotions Research Articles

Idiosyncratic fixation patterns generalize across dynamic and static facial expression recognition

Facial expression recognition (FER) is crucial for understanding the emotional state of others during human social interactions. It has been assumed that humans share universal visual sampling strategies to achieve this task. However, recent studies in face identification have revealed striking idiosyncratic fixation patterns, questioning the universality of face processing. More importantly, very little is known about whether such idiosyncrasies extend to the biological relevant recognition of static and dynamic facial expressions of emotion (FEEs). To clarify this issue, we tracked observers’ eye movements categorizing static and ecologically valid dynamic faces displaying the six basic FEEs, all normalized for time presentation (1 s), contrast and global luminance across exposure time. We then used robust data-driven analyses combining statistical fixation maps with hidden Markov Models to explore eye-movements across FEEs and stimulus modalities. Our data revealed three spatially and temporally distinct equally occurring face scanning strategies during FER. Crucially, such visual sampling strategies were mostly comparably effective in FER and highly consistent across FEEs and modalities. Our findings show that spatiotemporal idiosyncratic gaze strategies also occur for the biologically relevant recognition of FEEs, further questioning the universality of FER and, more generally, face processing.

Application of Facial Expression Analysis Technology in Violence Risk Assessment of Individuals with Mental Disorders in Supervised Settings.

To explore the association between violent behaviors and emotions in individuals with mental disorders, to evaluate the application value of facial expression analysis technology in violence risk assessment of individuals with mental disorders in supervised settings, and to provide a reference for violence risk assessment. Thirty-nine male individuals with mental disorders in supervised settings were selected, the participant risk of violence, cognitive function, psychiatric symptoms and severity were assessed using the Modified Overt Aggression Scale (MOAS), the Historical, Clinical, Risk Management-Chinese version(HCR-CV), the Positive and Negative Syndrome Scale (PANSS) and the Brief Psychiatric Rating Scale (BPRS). An emotional arousal was performed on the participants and the intensity of their emotions and facial expression action units was recorded before, during and after the arousal. One-way analysis of variance (ANOVA) was used to compare the differences in the intensity of emotions and facial expression action units before, during and after the arousal. Pearson correlation analysis was used to calculate the correlations between the intensity of the seven basic emotional facial expressions and the scores of the assessment scales. The intensity difference of sadness, surprise and fear in different time periods was statistically significant (P<0.05). The intensity of the left medial eyebrow lift action unit was found significantly different before and after the emotional arousal (P<0.05). The intensity of anger was positively correlated with the Modified Overt Aggression Scale score throughout the experiment (P<0.05). Eye action units such as eyebrow lifting, eyelid tightening and upper eyelid lifting can be used as effective action units to identify sadness, anger and other negative emotions associated with violent behaviors. Facial expression analysis technology can be used as an auxiliary tool to assess the potential risk of violence in individuals with mental disorders in supervised settings.

Facial emotion recognition deficits are associated with hypomimia and related brain correlates in Parkinson's disease.

Hypomimia is a frequent manifestation in Parkinson's disease (PD) that can affect interpersonal relationships and quality of life. Recent studies have suggested that hypomimia is not only related to motor dysfunction but also to impairment in emotional processing networks. Therefore, we hypothesized that the severity of hypomimia could be associated with performance on a task aimed at assessing facial emotion recognition. In this study, we explored the association between hypomimia, recognition of facial expressions of basic emotions using the Ekman 60 Faces Test (EF), and brain correlates of both hypomimia and performance on the EF. A total of 94 subjects underwent clinical assessments (neurological and neuropsychological examinations), and 56 of them participated in the neuroimaging study. We found significant correlation between hypomimia, EF Disgust (r = -0.242, p = 0.022) and EF Happiness (r = -0.264, p = 0.012); an independent reduction in Cortical Thickness (Cth) in the postcentral gyrus, insula, middle and superior temporal gyri, supramarginal gyrus, banks of the superior temporal sulcus, bilateral fusiform gyri, entorhinal cortex, parahippocampal gyrus, inferior and superior parietal cortex, and right cuneus and precuneus; and multiple correlations between negative emotions such as EF Disgust or EF Anger and a reduced Cth in fronto-temporo-parietal regions. In conclusion, these results suggest that the association between hypomimia and emotion recognition deficits in individuals with PD might be mediated by shared circuits, supporting the concept that hypomimia is not only the result of the dysfunction of motor circuits, but also of higher cognitive functions.

Multimodal impressions of voice quality settings: the role of vocal and visual symbolism

This study considers instances of voice quality settings under a sound-symbolic and synesthetic perspective, focusing on the auditory impressions these settings might have on listeners' attributions of meaning effects and associations between vocal and visual features related to emotional expression. Three perceptual experiments are carried out. The first experiment examined the impressionistic effects of eight voice quality settings characterized by differences in pitch. The second experiment examined the impressionistic effects of seven voice quality settings characterized by productions with the presence or absence of turbulent airflow, irregularity, and tenseness. The third experiment investigated associations between facial expressions of basic emotions and voice quality characteristics. Data are considered in terms of acoustic (fundamental frequency values), articulatory (reduced or expanded length of the vocal tract), perceptual impressions of size (big/small), strength (strong/weak), brightness (dark, clear), and distinctiveness (muffled/distinct), and visual features (facial expressions of the basic emotions sadness, happiness, anger, disgust, fear, and neutrality). The results provide corroborating evidence of existing links between sound and meaning and are discussed in relation to the frequency, production, sirenic biological codes, phonetic metaphors, and the vocal and facial gestures involved in emotional expression.

Masking our emotions: Emotion recognition and perceived intensity differ by race and use of medical masks.

Although medical masks have played a key role in decreasing the transmission of communicable disease, they simultaneously reduce the availability of nonverbal cues fundamental to social interaction. In the present study, we determined the collective impact of medical masks on emotional expression recognition and perceived intensity as a function of actor race. Participants completed an emotional expression recognition task involving stimuli with or without medical masks. Across six basic emotional facial expressions, medical masks were associated with significantly more emotional expression recognition errors. Overall, the effects associated with race varied depending on the emotion and appearance of masks. Whereas recognition accuracy was higher for White relative to Black actors for anger and sadness, the opposite pattern was observed for disgust. Medical mask-wearing exacerbated actor-race related recognition differences for anger and surprise, but attenuated these differences for fear. Emotional expression intensity ratings were significantly reduced for all emotions except fear, where masks were associated with increased perceived intensity. Masks further increased already higher intensity ratings for anger in Black versus White actors. In contrast, masks eliminated the tendency to give higher intensity ratings for Black versus White sad and happy facial expressions. Overall, our results suggest that the interaction between actor race and mask wearing status with respect to emotional expression judgements is complex, varying by emotion in both direction and degree. We consider the implications of these results particularly in the context of emotionally charged social contexts, such as in conflict, healthcare, and policing.

Eye movements and ERP biomarkers for face processing problems in avoidant attachment-style individuals.

Avoidant attachment poses a serious risk to intimate relationships and offspring. However, there are few studies on the face-processing characteristics and impairments of avoidant individuals based on basic emotion theory. Therefore, this study investigated the issues of emotional processing and deactivation strategies in individuals with avoidant attachment. Avoidant and secure individuals were recruited to participate in an eye-tracking experiment and a two-choice oddball task in which they had to distinguish facial expressions of basic emotions (sadness, anger, fear, disgust, and neutral). Eye fixation durations to various parts of the face, including the eyes, nose, and mouth, were measured, and three event-related potentials (ERP) components (P100, N170, and P300) were monitored. Avoidant individuals could not process facial expressions as easily as secure individuals. Avoidant individuals focused less on the eyes of angry faces when compared to secure individuals. They also exhibited a more positive P100 component and a less negative N170 component when processing faces and a larger amplitude of the P300 component than secure individuals when processing emotional expressions. Avoidant individuals use deactivating strategies and exhibit specific characteristics at different stages, which are of great significance in social interaction.

Construction of face databases for tasks to recognize facial expressions of basic emotions: a systematic review.

This systematic review sought to gather these studies, describing and comparing the methodologies used in their elaboration. The databases used to select the articles were the following: PubMed, Web of Science, PsycInfo, and Scopus. The following word crossing was used: "Facial expression database OR Stimulus set AND development OR Validation." A total of 36 articles showed that most of the studies used actors to express the emotions that were elicited from specific situations to generate the most spontaneous emotion possible. The databases were mainly composed of colorful and static stimuli. In addition, most of the studies sought to establish and describe patterns to record the stimuli, such as color of the garments used and background. The psychometric properties of the databases are also described. The data presented in this review point to the methodological heterogeneity among the studies. Nevertheless, we describe their patterns, contributing to the planning of new research studies that seek to create databases for new contexts.

A deep neural network model of the primate superior colliculus for emotion recognition.

Although sensory processing is pivotal to nearly every theory of emotion, the evaluation of the visual input as ‘emotional’ (e.g. a smile as signalling happiness) has been traditionally assumed to take place in supramodal ‘limbic’ brain regions. Accordingly, subcortical structures of ancient evolutionary origin that receive direct input from the retina, such as the superior colliculus (SC), are traditionally conceptualized as passive relay centres. However, mounting evidence suggests that the SC is endowed with the necessary infrastructure and computational capabilities for the innate recognition and initial categorization of emotionally salient features from retinal information. Here, we built a neurobiologically inspired convolutional deep neural network (DNN) model that approximates physiological, anatomical and connectional properties of the retino-collicular circuit. This enabled us to characterize and isolate the initial computations and discriminations that the DNN model of the SC can perform on facial expressions, based uniquely on the information it directly receives from the virtual retina. Trained to discriminate facial expressions of basic emotions, our model matches human error patterns and above chance, yet suboptimal, classification accuracy analogous to that reported in patients with V1 damage, who rely on retino-collicular pathways for non-conscious vision of emotional attributes. When presented with gratings of different spatial frequencies and orientations never ‘seen’ before, the SC model exhibits spontaneous tuning to low spatial frequencies and reduced orientation discrimination, as can be expected from the prevalence of the magnocellular (M) over parvocellular (P) projections. Likewise, face manipulation that biases processing towards the M or P pathway affects expression recognition in the SC model accordingly, an effect that dovetails with variations of activity in the human SC purposely measured with ultra-high field functional magnetic resonance imaging. Lastly, the DNN generates saliency maps and extracts visual features, demonstrating that certain face parts, like the mouth or the eyes, provide higher discriminative information than other parts as a function of emotional expressions like happiness and sadness. The present findings support the contention that the SC possesses the necessary infrastructure to analyse the visual features that define facial emotional stimuli also without additional processing stages in the visual cortex or in ‘limbic’ areas.This article is part of the theme issue ‘Cracking the laugh code: laughter through the lens of biology, psychology and neuroscience’.

Attention for Emotion-How Young Adults With Neurodevelopmental Disorders Look at Facial Expressions of Affect.

While Autism Spectrum Disorder (ASD), Attention-Deficit/Hyperactivity Disorder (ADHD) and Schizophrenia (SCZ) differ in many clinically relevant features such as symptomatology and course, they may also share genetic underpinnings, affective problems, deviancies in social interactions, and are all characterized by some kind of cognitive impairment. This situation calls for a joint investigation of the specifics of cognitive (dys-)functions of the three disorders. Such endeavor should focus, among other domains, on the inter-section of processing cognitive, affective and social information that is crucial in effective real-life interactions and can be accomplished when attentional preferences for human facial expressions of emotions is studied. To that end, attention to facial expressions of basic emotions was examined in young adults with ASD, ADHD, or SCZ in the present study. The three clinical groups were compared with an age-matched group of typically-developing participants (TD) during the free contemplation of five different facial emotions presented simultaneously, by varying identities, through the registration of eye movements. We showed, that dwell times and fixation counts differed for the different emotions in TD and in a highly similar way in ADHD. Patients with ASD differed from TD by showing a stronger differentiation between emotions and partially different attentional preferences. In contrast, the SCZ group showed an overall more restricted scanning behavior and a lack of differentiation between emotions. The ADHD group, showed an emotion-specific gazing pattern that was highly similar to that of controls. Thus, by analyzing eye movements, we were able to differentiate three different viewing patterns that allowed us to distinguish between the three clinical groups. This outcome suggests that attention for emotion may not tap into common pathophysiological processes and argues for a multi-dimensional approach to the grouping of disorders with neurodevelopmental etiology.

Facial Motion Analysis beyond Emotional Expressions.

Facial motion analysis is a research field with many practical applications, and has been strongly developed in the last years. However, most effort has been focused on the recognition of basic facial expressions of emotion and neglects the analysis of facial motions related to non-verbal communication signals. This paper focuses on the classification of facial expressions that are of the utmost importance in sign languages (Grammatical Facial Expressions) but also present in expressive spoken language. We have collected a dataset of Spanish Sign Language sentences and extracted the intervals for three types of Grammatical Facial Expressions: negation, closed queries and open queries. A study of several deep learning models using different input features on the collected dataset (LSE_GFE) and an external dataset (BUHMAP) shows that GFEs can be learned reliably with Graph Convolutional Networks simply fed with face landmarks.

Perception of Dynamic Point Light Facial Expression

Abstract This study uses point light displays both to investigate the roles of global and local motion analyses in the perception of dynamic facial expressions and to measure the information threshold for reliable recognition of emotions. We videotaped the faces of actors wearing black makeup with white dots while they dynamically produced each of 6 basic Darwin/Ekman emotional expressions. The number of point lights was varied to systematically manipulate amount of information available. For all but one of the expressions, discriminability (d′) increased approximately linearly with number of point lights, with most remaining largely discriminable with as few as only 6 point lights. This finding supports reliance on global motion patterns produced by facial muscles. However, discriminability for the happy expression was notably higher and largely unaffected by number of point lights and thus appears to rely on characteristic local motion, probably the unique upward curvature of the mouth. The findings indicate that recognition of facial expression is not a unitary process and that different expressions may be conveyed by different perceptual information, but in general, basic facial emotional expressions typically remain largely discriminable with as few as 6 dynamic point lights.

Left Hemisphere Dominance for Negative Facial Expressions: The Influence of Task.

Major theories of hemisphere asymmetries in facial expression processing predict right hemisphere dominance for negative facial expressions of disgust, fear, and sadness, however, some studies observe left hemisphere dominance for one or more of these expressions. Research suggests that tasks requiring the identification of six basic emotional facial expressions (angry, disgusted, fearful, happy, sad, and surprised) are more likely to produce left hemisphere involvement than tasks that do not require expression identification. The present research investigated this possibility in two experiments that presented six basic emotional facial expressions to the right or left hemisphere using a divided-visual field paradigm. In Experiment 1, participants identified emotional expressions by pushing a key corresponding to one of six labels. In Experiment 2, participants detected emotional expressions by pushing a key corresponding to whether an expression was emotional or not. In line with predictions, fearful facial expressions exhibited a left hemisphere advantage during the identification task but not during the detection task. In contrast to predictions, sad expressions exhibited a left hemisphere advantage during both identification and detection tasks. In addition, happy facial expressions exhibited a left hemisphere advantage during the detection task but not during the identification task. Only angry facial expressions exhibited a right hemisphere advantage, and this was only observed when data from both experiments were combined. Together, results highlight the influence of task demands on hemisphere asymmetries in facial expression processing and suggest a greater role for the left hemisphere in negative expressions than predicted by previous theories.

The recognition of facial expressions of emotion in deaf and hearing individuals

During real-life interactions, facial expressions of emotion are perceived dynamically with multimodal sensory information. In the absence of auditory sensory channel inputs, it is unclear how facial expressions are recognised and internally represented by deaf individuals. Few studies have investigated facial expression recognition in deaf signers using dynamic stimuli, and none have included all six basic facial expressions of emotion (anger, disgust, fear, happiness, sadness, and surprise) with stimuli fully controlled for their low-level visual properties, leaving the question of whether or not a dynamic advantage for deaf observers exists unresolved. We hypothesised, in line with the enhancement hypothesis, that the absence of auditory sensory information might have forced the visual system to better process visual (unimodal) signals, and predicted that this greater sensitivity to visual stimuli would result in better recognition performance for dynamic compared to static stimuli, and for deaf-signers compared to hearing non-signers in the dynamic condition. To this end, we performed a series of psychophysical studies with deaf signers with early-onset severe-to-profound deafness (dB loss >70) and hearing controls to estimate their ability to recognize the six basic facial expressions of emotion. Using static, dynamic, and shuffled (randomly permuted video frames of an expression) stimuli, we found that deaf observers showed similar categorization profiles and confusions across expressions compared to hearing controls (e.g., confusing surprise with fear). In contrast to our hypothesis, we found no recognition advantage for dynamic compared to static facial expressions for deaf observers. This observation shows that the decoding of dynamic facial expression emotional signals is not superior even in the deaf expert visual system, suggesting the existence of optimal signals in static facial expressions of emotion at the apex. Deaf individuals match hearing individuals in the recognition of facial expressions of emotion.

Facial width-to-height ratio underlies perceived dominance on facial emotional expressions

The facial width-to-height ratio (fWHR) is a perceptual cue that affects the perception of psychological traits such as dominance. The current research examined whether the fWHR would impact the perception of dominance and emotional intensity when expressing emotions. In study one, we examined whether the emotional facial expressions (EFEs) modify the visually perceivable fWHR by following a specific pattern reflecting the perception of dominance associated with basic EFEs. We found that EFEs differed from neutral poses following the expected pattern: high dominance EFEs (anger, disgust, happiness) increased the fWHR, whereas low dominance EFEs (fear, sadness, surprise) decreased the fWHR. In study two, we investigated whether manipulating the fWHR (low, average, high) would affect the perception of dominance and emotional intensity. We obtained that the fWHR influenced the perception of dominance and emotional intensity but its effect on dominance was only present with high dominance EFEs. One social implication of this effect is that individuals for which expressing dominant emotions lead to high increase of their fWHR would be perceived highly dominant. We discuss that such effect could participate in the development of individuals' dominance and further researches are still needed to determine its social impact in interaction with other factors.

Effect of Basic Emotional Facial Expressions on Time Perception

Amaç: Bu çalışmanın amacı, evrensel olarak kabul edilen 6 temel duyguya ait yüz ifadelerinin sunum süresi değişimlenmesinin zaman algısı üzerindeki etkisini, zaman aralığı yeniden oluşturma ve süreyi sözel tahmin etme yöntemleri aracılığıyla incelemektir.Yöntem: Araştırma, dört farklı sunum süresinde (500, 1000, 1500 ve 2000 ms) sunulan yüz ifadelerinin (kızgınlık, korku, üzüntü, tiksinme, şaşkınlık, mutluluk ve nötr) sözel olarak tahmin edilmesi ve yeniden üretilmesini içeren iki farklı yöntemle 75 gönüllü öğrenci (47 Kadın ve 28 Erkek) ile yürütülmüştür. Bulgular: Her bir sunum süresinde, tüm duygusal yüz ifadeleri hedef süreye göre kısa algılanmakta ve sunum süresi artıkça zamansal bozulmalar artmaktadır. Öte yandan, tüm duygusal yüz ifadeleri, üzüntü yüz ifadesi hariç, nötr yüz ifadesinden daha uzun algılanmaktadır. Bununla birlikte, tüm uyarıcı süreleri, zaman aralığı yeniden oluşturma görevinde, süreyi sözel tahmin etme görevine göre daha uzun algılanmaktadır.Sonuç: İçsel saat modeli ile dikkat ve genel uyarılmışlık düzeyi bağlamında, yüz ifadelerinin hedef süreye göre kısa algılanması dikkat kaynaklı modelle; nötre göre daha uzun algılanması ise genel uyarılmışlık düzeyi kaynaklı modelle daha uyumlu görünmektedir. Duygusal yüz ifadelerinin zaman algısı üzerindeki etkisinde, genel uyarılmışlık düzeyi ve dikkat kaynaklı ortak etkilerin yanı sıra, zamansal yöntem türlerindeki farklılıklardan kaynaklanan bellek ve karar verme süreçleri de etkili olabilmektedir.

Strategy Shift Toward Lower Spatial Frequencies in the Recognition of Dynamic Facial Expressions of Basic Emotions: When It Moves It Is Different.

Facial expressions of emotion play a key role in social interactions. While in everyday life, their dynamic and transient nature calls for a fast processing of the visual information they contain, a majority of studies investigating the visual processes underlying their recognition have focused on their static display. The present study aimed to gain a better understanding of these processes while using more ecological dynamic facial expressions. In two experiments, we directly compared the spatial frequency (SF) tuning during the recognition of static and dynamic facial expressions. Experiment 1 revealed a shift toward lower SFs for dynamic expressions in comparison to static ones. Experiment 2 was designed to verify if changes in SF tuning curves were specific to the presence of emotional information in motion by comparing the SF tuning profiles for static, dynamic, and shuffled dynamic expressions. Results showed a similar shift toward lower SFs for shuffled expressions, suggesting that the difference found between dynamic and static expressions might not be linked to informative motion per se but to the presence of motion regardless its nature.

Quantifying Facial Expression Intensity and Signal Use in Deaf Signers.

We live in a world of rich dynamic multisensory signals. Hearing individuals rapidly and effectively integrate multimodal signals to decode biologically relevant facial expressions of emotion. Yet, it remains unclear how facial expressions are decoded by deaf adults in the absence of an auditory sensory channel. We thus compared early and profoundly deaf signers (n= 46) with hearing nonsigners (n= 48) on a psychophysical task designed to quantify their recognition performance for the six basic facial expressions of emotion. Using neutral-to-expression image morphs and noise-to-full signal images, we quantified the intensity and signal levels required by observers to achieve expression recognition. Using Bayesian modeling, we found that deaf observers require more signal and intensity to recognize disgust, while reaching comparable performance for the remaining expressions. Our results provide a robust benchmark for the intensity and signal use in deafness and novel insights into the differential coding of facial expressions of emotion between hearing and deaf individuals.

Facial Expressions of Basic Emotions in Japanese Laypeople.

Facial expressions that show emotion play an important role in human social interactions. In previous theoretical studies, researchers have suggested that there are universal, prototypical facial expressions specific to basic emotions. However, the results of some empirical studies that tested the production of emotional facial expressions based on particular scenarios only partially supported the theoretical predictions. In addition, all of the previous studies were conducted in Western cultures. We investigated Japanese laypeople (n = 65) to provide further empirical evidence regarding the production of emotional facial expressions. The participants produced facial expressions for six basic emotions (anger, disgust, fear, happiness, sadness, and surprise) in specific scenarios. Under the baseline condition, the participants imitated photographs of prototypical facial expressions. The produced facial expressions were automatically coded using FaceReader in terms of the intensities of emotions and facial action units. In contrast to the photograph condition, where all target emotions were shown clearly, the scenario condition elicited the target emotions clearly only for happy and surprised expressions. The photograph and scenario conditions yielded different profiles for the intensities of emotions and facial action units associated with all of the facial expressions tested. These results provide partial support for the theory of universal, prototypical facial expressions for basic emotions but suggest the possibility that the theory may need to be modified based on empirical evidence.

RECOGNITION OF FACIAL MICRO EXPRESSIONS OF EMOTIONS DEPENDING ON PROFESSIONAL ORIENTATION, SATISFACTION WITH LIFE AND SOCIAL BACKGROUND OF STUDENTS

The aim of this research was to examine if there is a difference regarding the successfulness of observation of facial micro expressions of basic emotions among students depending on their professional orientation, social background and self-assessment of satisfaction with life. The instruments used were a computer program for examining the ability to observe basic emotions based on micro expressions of the face and SWLS- Satisfaction with life scale. The results showed that there is a connection between the self-assessment of satisfaction with life and the accuracy of observation of micro facial expressions of basic emotions. When it comes to the successfulness of observation of facial micro expressions of basic emotions, there is no difference between students from the city and the village, or between the subsamples of students of different social backgrounds.

The importance of recurrent top-down synaptic connections for the anticipation of dynamic emotions

Different studies have shown the efficiency of a feed-forward neural network in categorizing basic emotional facial expressions. However, recent findings in psychology and cognitive neuroscience suggest that visual recognition is not a pure bottom-up process but likely involves top-down recurrent connectivity. In the present computational study, we compared the performances of a pure bottom-up neural network (a standard multi-layer perceptron, MLP) with a neural network involving recurrent top-down connections (a simple recurrent network, SRN) in the anticipation of emotional expressions. In two complementary simulations, results revealed that the SRN outperformed the MLP for ambiguous intensities in the temporal sequence, when the emotions were not fully depicted but when sufficient contextual information (related to previous time frames) was provided. Taken together, these results suggest that, despite the cost of recurrent connections in terms of energy and processing time for biological organisms, they can provide a substantial advantage for the fast recognition of uncertain visual signals.