Facial Affect Research Articles

Gaze behavior in response to affect during natural social interactions.

Attention in social interactions is directed by social cues such as the face or eye region of an interaction partner. Several factors that influence these attentional biases have been identified in the past. However, most findings are based on paradigms with static stimuli and no interaction potential. Therefore, the current study investigated the influence of one of these factors, namely facial affect in natural social interactions using an evaluated eye-tracking setup. In a sample of 35 female participants, we examined how individuals' gaze behavior responds to changes in the facial affect of an interaction partner trained in affect modulation. Our goal was to analyze the effects on attention to facial features and to investigate their temporal dynamics in a natural social interaction. The study results, obtained from both aggregated and dynamic analyses, indicate that facial affect has only subtle influences on gaze behavior during social interactions. In a sample with high measurement precision, these findings highlight the difficulties of capturing the subtleties of social attention in more naturalistic settings. The methodology used in this study serves as a foundation for future research on social attention differences in more ecologically valid scenarios.

Full and Partial Facial Affect Recognition in Pediatric Brain Tumour Survivors and Typically Developing Children Following COVID-19 Pandemic.

Affect recognition has emerged as a potential mechanism underlying the social competence challenges experienced by pediatric brain tumour survivors (PBTSs). However, many social interactions were altered during the pandemic, with the widespread use of masking potentially impacting affect recognition abilities. Here, we examine affect recognition in PBTSs and typically developing youth (TD) after the onset of the global pandemic. Twenty-three PBTSs and 24 TD between 8 and 16 years old were recruited and completed two performance-based affect recognition tasks (full and partial facial features) and a self-reported questionnaire on mask exposure in their social interactions. Their parents completed parent proxy questionnaires on their child's social adjustment and sociodemographics. The scores between the PBTSs and TD did not differ significantly in full (t(45) = 1.33, p = 0.19, d = 0.39, 95% CI [-0.69, 3.40]) or partial (t(37.36) = 1.56, p = 0.13, d = 0.46, 95% CI [-0.47, 3.60]) affect recognition, suggesting similar affect recognition between the two groups. These skills were also not significantly correlated with social adjustment or mask exposure (p > 0.05). However, the combined sample had significantly better scores in affect recognition when exposed to partial facial cues versus full. Additionally, participants obtained lower scores on a measure of full facial affect recognition and higher scores on a measure of partial affect recognition compared to pre-pandemic data. The pandemic may have influenced affect recognition across youth, underscoring the importance of further research into its lasting impact on the social competence of youth.

Both partners’ negative emotion drives aggression during couples’ conflict

Researchers examining conflict between intimate partners believe that the experience and expression of emotion drives aggressive behaviour. Intra-personally, increases in negative affect make aggression more likely. Inter-personally, theoretical models suggest that each individuals’ perception of their partners’ emotion also influences aggression, potentially creating a Violence Escalation Cycle. Here, using a lab-based aggression task across a primary study (n = 104, number of trials = 3095) and a replication (n = 58, number of trials = 3167), we show that both intra- and inter-personal experiences of negative emotion predict reactive aggression within couples, revealing retaliation but not escalation. Critically, analyses of facial affect reveal that prototypic displays of negative emotions have a compounding effect, leading to dramatic changes in aggression depending on whether one, both, or neither partner expressed negative emotion. We propose a mechanism by which temporal delays (i.e., experimentally imposed forced breaks) reduce aggression by decreasing negative emotional arousal and limiting impulsive action. Our results show that both forced breaks and elective breaks (i.e., extra participant-initiated extensions of the forced break time) reduce aggression, providing exciting evidence that interventions focused on preventing impulsive action when people are in a provoked state can reduce aggression within couples.

Social Cognition, Executive Functioning, Mood, and Disability in Cervical Dystonia.

Individuals with idiopathic adult-onset isolated cervical dystonia (CD) may have cognitive difficulties and increased mood challenges. Social cognition and executive functioning may be particularly affected. To explore social cognition and executive functioning performance in individuals with CD, using the Cambridge Neuropsychological Test Automated Battery (CANTAB), as previous research has used traditional, nondigital neuropsychological assessments. We sought to investigate the relationships between social cognition, executive functioning, mood, and disability in individuals with CD. We recruited 37 individuals with CD, including 26 women with an age range of 33 to 69 years (M = 56.64, SD = 8.31) from a dystonia clinic in a hospital neurology department. The individuals completed selected tasks from the CANTAB measuring social cognition and executive functioning. We compared the individuals' performance with CANTAB normative data. Depression, anxiety, disease severity, and disability were measured. The individuals with CD had significantly lower scores than the CANTAB normative data in both social cognition and executive functioning tasks, with the largest differences evident in problem-solving, attention, and positive emotion bias tasks. Poorer emotion recognition was associated with increased difficulties in problem-solving tasks. The individuals demonstrated a bias toward identifying happiness in facial affect, which was related to a poorer recognition of emotions. Cognitive performance was not related to CD severity or disability or to current mood symptoms. Difficulties with both social cognition and executive functioning were identified in the individuals with CD, and are likely important targets for clinical interventions.

Phase-Transitions in Disgust During Self-Initiated Eating Among Adolescents With Anorexia Nervosa and Related Low-Weight Eating Disorders and Matched Controls.

Understanding the emotional context of feeding behavior may help identify causal mechanisms of food avoidance among individuals with anorexia nervosa. Although predominant food avoidance models assume fear of fat drives feeding behavior, disgust may be more theoretically and proximally relevant to moment-to-moment experiences of feeding. This study, therefore, aimed to examine affect and food avoidance using automated affect analysis from facial response by measuring time-specific transitions in disgust during a laboratory eating paradigm. We hypothesized that phase transitions in disgust would distinguish temporally self-initiated eating from food avoidance. Sixty-three adolescents with anorexia nervosa or another low-weight eating disorder (LWED) and 27 age- and sex-matched controls were recruited as part of a larger study; 45 patients and 22 controls provided data on autonomous eating and facial affect during a laboratory meal. Dynamic structural equation models quantified moment-to-moment relationships between disgust and feeding behavior. Self-initiated eating was associated with greater increases in disgust, but not fear, intensity among those with LWED relative to control participants and greater disgust intensity predicted lower likelihood of self-initiated eating. Phasic transitions in disgust provide moment-to-moment evidence of affective influence on self-initiated eating and lend credibility to the hypothesis that disgust contributes to food avoidance and initiation in individuals with LWED.

Infants' salivary oxytocin and positive affective reactions to people

Oxytocin is a neuropeptide positively associated with prosociality in adults. Here, we studied whether infants' salivary oxytocin can be reliably measured, is developmentally stable, and is linked to social behavior. We longitudinally collected saliva from 62 U.S. infants (44 % female, 56 % Hispanic/Latino, 24 % Black, 18 % non-Hispanic White, 11 % multiracial) at 4, 8, and 14 months of age and offline-video-coded the valence of their facial affect in response to a video of a smiling woman. We also captured infants' affective reactions in terms of excitement/joyfulness during a live, structured interaction with a singing woman in the Early Social Communication Scales at 14 months. We detected stable individual differences in infants' oxytocin levels over time (over minutes and months) and in infants' positive affect over months and across contexts (video-based and in live interactions). We detected no statistically significant changes in oxytocin levels between 4 and 8 months but found an increase from 8 to 14 months. Infants with higher oxytocin levels showed more positive facial affect to a smiling person video at 4 months; however, this association disappeared at 8 months, and reversed at 14 months (i.e., higher oxytocin was associated with less positive facial affect). Infant salivary oxytocin may be a reliable physiological measure of individual differences related to socio-emotional development.

Differentiating neural sensitivity and bias during face-emotion processing in youth: a computational approach.

The ability to interpret face-emotion displays is critical for the development of adaptive social interactions. Using a novel variant of a computational model and fMRI data, we examined behavioral and neural associations between two metrics of face-emotion labeling (sensitivity and bias) and age in youth. Youth and adults (n = 44, M age = 20.02, s.d. = 7.44, range = 8-36) completed an explicit face-emotion labeling fMRI task including happy to angry morphed face emotions. A drift-diffusion model was applied to choice and reaction time distributions to examine sensitivity and bias in interpreting face emotions. Model fit and reliability of parameters were assessed on adult data (n = 42). Linear and quadratic slopes modeled brain activity associated with dimensions of face-emotion valence and ambiguity during interpretation. Behaviorally, age was associated with sensitivity. The bilateral anterior insula exhibited a more pronounced neural response to ambiguity with older age. Associations between sensitivity and bias metrics and activation patterns indicated that systems encoding face-emotion valence and ambiguity both contribute to the ability to discriminate face emotions. The current study provides evidence for age-related improvement in perceptual sensitivity to facial affect across adolescence and young adulthood.

An ecological momentary cognitive assessment study of over-attribution of threat and suicide risk factors in people with serious mental illness

IntroductionPeople with serious mental illness (SMI; schizophrenia, schizoaffective disorder, bipolar disorder) are at increased risk of suicidal ideation (SI). Over-attribution of social threat, or attributing threatening emotions to neutral faces, may contribute to social isolation through increased social avoidance and decreased social approach motivation. These factors are related to suicide, as well as perceived burdensomeness (PB) and thwarted belongingness (TB). This study examined how over-attribution of threat relates to PB, TB, and social motivations. MethodN = 273 participants with SMI were assessed for current SI and behavior, and were stratified into SI (N = 130) vs. non-SI (N = 143) groups. Participants completed smartphone surveys (via ecological momentary assessments [EMA]) 3×/day for 10 days. They also completed the Mobile Ecological Test of Emotion Recognition (METER) 1×/day. Linear mixed models and multi-level mediation tested the relationships between over-attribution of threat, METER performance, PB/TB, and social motivations. ResultsParticipants with and without SI did not significantly differ in over-attribution of threat or METER performance. In separate models, there was a relationship of over-attribution of threat with increased PB (B = 1.00, SE = 0.21, t = 4.72, p < .001), reduced social approach motivation (B = -0.74, SE = 0.22, t = −3.33, p < .001), and increased social avoidance (B = 0.90, SE = 0.24, t = 3.70, p < .001), all significant when adjusting for facial affect recognition ability. A model examining social motivations as a mediator between over-attribution of threat and PB/TB was not significant. ConclusionThese results suggest that over-attribution of threat relates to interpersonal constructs related to SI irrespective of facial affect abilities. This study may inform understanding of social cognitive processes related to suicide in SMI.

Mother-infant self- and interactive contingency at four months and infant cognition at one year: A view from microanalysis

Although a considerable literature documents associations between early mother-infant interaction and cognitive outcomes in the first years of life, few studies examine the contributions of contingently coordinated mother-infant interaction to infant cognitive development. This study examined associations between the temporal dynamics of the contingent coordination of mother-infant face-to-face interaction at 4 months and cognitive performance on the Bayley Scales of Infant Development at age one year in a sample of (N = 100) Latina mother-infant pairs. Split-screen videotaped interactions were coded on a one second time base for the communication modalities of infant and mother gaze and facial affect, infant vocal affect, and mother touch. Multi-level time-series models evaluated self- and interactive contingent processes in these modalities and revealed 4-month patterns of interaction associated with higher one-year cognitive performance, not identified in prior studies. Infant and mother self-contingency, the moment-to-moment probability that the individual’s prior behavior predicts the individual’s future behavior, was the most robust measure associated with infant cognitive performance. Self-contingency findings showed that more varying infant behavior was optimal for higher infant cognitive performance, namely, greater modulation of negative affect; more stable maternal behavior was optimal for higher infant cognitive performance, namely, greater likelihood of sustaining positive facial affect. Although interactive contingency findings were sparse, they showed that, when mothers looked away, or dampened their faces to interest or mild negative facial affect, infants with higher 12-month cognitive performance were less likely to show negative vocal affect. We suggest that infant ability to modulate negative affect, and maternal ability to sustain positive affect, may be mutually reinforcing, together creating a dyadic climate that is associated with more optimal infant cognitive development.

Measuring facial mimicry: Affdex vs. EMG.

Facial mimicry is the automatic imitation of the facial affect expressions of others. It serves as an important component of interpersonal communication and affective co-experience. Facial mimicry has so far been measured by Electromyography (EMG), which requires a complex measuring apparatus. Recently, software for measuring facial expressions have become available, but it is still unclear how well it is suited for measuring facial mimicry. This study investigates the comparability of the automated facial coding software Affdex with EMG for measuring facial mimicry. For this purpose, facial mimicry was induced in 33 subjects by presenting naturalistic affect-expressive video sequences (anger, joy). The response of the subjects is measured simultaneously by facial EMG (corrugator supercilii muscle, zygomaticus major muscle) and by Affdex (action units lip corner puller and brow lowerer and affects joy and anger). Subsequently, the correlations between the measurement results of EMG and Affdex were calculated. After the presentation of the joy stimulus, there was an increase in zygomaticus muscle activity (EMG) about 400 ms after stimulus onset and an increase in joy and lip corner puller activity (Affdex) about 1200 ms after stimulus onset. The joy and the lip corner puller activity detected by Affdex correlate significantly with the EMG activity. After presentation of the anger stimulus, corrugator muscle activity (EMG) also increased approximately 400 ms after stimulus onset, whereas anger and brow lowerer activity (Affdex) showed no response. During the entire measurement interval, anger activity and brow lowerer activity (Affdex) did not correlate with corrugator muscle activity (EMG). Using Affdex, the facial mimicry response to a joy stimulus can be measured, but it is detected approximately 800 ms later compared to the EMG. Thus, electromyography remains the tool of choice for studying subtle mimic processes like facial mimicry.

Pre-scan state anxiety is associated with greater right amygdala-hippocampal response to fearful versus happy faces among trait-anxious Latina girls.

Unfamiliarity with academic research may contribute to higher levels of anticipatory state anxiety about affective neuroimaging tasks. Children with high trait anxiety display differences in brain response to fearful facial affect compared to non-anxious youth, but little is known about the influence of state anxiety on this association. Because reduced engagement in scientific research and greater mistrust among minoritized groups may lead to systematic differences in pre-scan state anxiety, it is crucial to understand the neural correlates of state anxiety during emotion processing so as to disambiguate sources of individual differences. The present study probed the interactive effects of pre-scan state anxiety, trait anxiety, and emotional valence (fearful vs. happy faces)on neural activation during implicit emotion processing in a community sample of 46 preadolescent Latina girls (8-13 years). Among girls with mean and high levels of trait anxiety, pre-scan state anxiety was associated with greater right amygdala-hippocampal and left inferior parietal lobe response to fearful faces relative to happy faces. Anticipatory state anxiety in the scanning context may cause children with moderate and high trait anxiety to be hypervigilant to threats, further compounding the effects of trait anxiety. Neuroimaging researchers should control for state anxiety so that systematic differences in brain activation resulting from MRI apprehension are not misleadingly attributed to demographic or environmental characteristics.

Altering experienced utility by incidental affect: The interplay of valence and arousal in incidental affect infusion processes.

The way we evaluate an experience can be influenced by contextual factors that are unrelated to the experience at hand. A prominent factor that has been shown to infuse into the evaluation processes is incidental affect. Prior research has examined the role of such incidental affect by either focusing on its valence or its arousal, while neglecting the interplay of these two components in the affect infusion process. Based on the affect-integration-motivation (AIM) framework from affective neuroscience, our research proposes a novel arousal transport hypothesis (ATH) that describes how valence and arousal of an affective state jointly influence the evaluation of experiences. We test the ATH in a set of multimethod studies combining functional magnetic resonance imaging (fMRI), skin conductance recording, automated facial affect recording, and behavioral approaches across a range of sensory modalities including auditory, gustatory, and visual. We find that positive incidental affect, induced by viewing affect-laden pictures (vs. neutral pictures) or winning (vs. not winning) monetary rewards, enhances how much an experience (i.e., listening to music, consuming wines, or looking at images) is enjoyed. Tracking moment-based changes of affective states at the neurophysiological level, we demonstrate that valence mediates reported enjoyment and that arousal is necessary to implement and moderate these mediating effects. We rule out alternative explanations for these mediation patterns such as the excitation transfer account and the attention narrowing account. Finally, we discuss how the ATH framework provides a new perspective to explain divergent decision outcomes caused by discrete emotions and its implications for effort-based decision-making. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Social participation of school-aged survivors of pediatric brain tumors: A daily diary report.

Difficulties with social functioning are common among survivors of pediatric brain tumors. Social participation is an understudied measure of social functioning that is associated with emotional health across the lifespan. This paper uses a diary method to assess the social participation of survivors of pediatric brain tumors in middle childhood. Survivors of pediatric brain tumors (N=47; age 10.6±1.4years; 51.1% male, 89.4% White) who were 5.3 (SD=2.4, range: 2-9.9) years post therapy completed a daily diary assessment of social interaction (5-7days) and an objective measure of facial affect recognition. The participant's caregiver completed the NIH Toolbox Emotion Measures and a background information questionnaire. Overall, frequency and quality of reported social interactions were low for survivors, with a large subset of survivors (n=16, 34%) endorsing fewer than 10 social interactions over the course of a typical school week, and almost half of parents (48.9%) reporting that their child participates in zero social activities outside of school during a typical week. Participants engaged in more positive social participation exhibited stronger social skills (facial affect recognition (F(2,44) =4.85, p<.05). School-aged survivors of pediatric brain tumors seemed to be infrequently engaged in social participation and quality interaction with peers. More specifically, the interactions most commonly reported on the diary assessment are not indicative of friendship development and maintenance. Survivors of pediatric brain tumors would likely benefit from interventions designed to increase quality time spent with peers.

38 Assessing Memory for Emotions Separately from Emotion Recognition

Objective:Accurate processing of facial displays of emotion is critical for effective communication. A robust literature has documented impairment in the ability to recognize facial affect in people with traumatic brain injury (TBI), but research is scarce about memory for facial affect. Disruptions in recognizing and remembering the emotions of others can undermine relationship quality and may result in psychosocial dysfunction. Importantly, the extant literature indicates that facial affect recognition dissociates from other cognitive abilities such that it is likely a distinct neuronal process. Thus, explicit measurement of affect recognition and memory for emotions may be critical for implementing and refining rehabilitation interventions. The present study examined the relationship between recognition and memory for emotions using a novel computerized task and explored its associations with other cognitive abilities.Participants and Methods:Participants were adults who were neurologically healthy (n = 31) or had a history of moderate to severe TBI (n = 26). The battery included the novel Assessment of Facial Affect Recognition and Memory (AFARM), Cambridge Face Memory Test (face memory without emotion), Wechsler Test of Adult Reading, Rey Auditory Verbal Learning Test, Judgment of Line Orientation, Oral Symbol Digit Modalities, Digit Span, FAS, Animal Fluency, and the Affect Intensity Measure (experienced emotion). Spearman correlations examined the relationship of AFARM performance with the test battery. Logistic regression models examined whether immediate-delay (ID-EM) and long-delay face emotion-memory (LD-EM) accounted for unique variance in group membership beyond recognition accuracy of facial affect and memory for faces.Results:AFARM demonstrated relationships with neuropsychological and mood variables in the expected directions across and within groups, with the strongest associations observed for memory for verbal information (rs = .51 to .58) and processing speed (rs = .48 to .57). Consistent with traditional list-learning tests, ID- and LD-EM were highly correlated (r = .85). Experienced affect intensity was inversely associated with ID-EM (r = -.29) and LD-EM (r = -.38) but not with recognition accuracy (r = -.10). Logistic regression examining ID-EM was significant, χ2(3) = 26.05, p &lt; .001, Nagelkerke R2 = .49. ID-EM accounted for unique variance in group status (p = .006; OR = 0.65) after accounting for recognition accuracy and face memory. Similarly, the model examining LD-EM was significant χ2(3) = 27.70, p &lt; .001, Nagelkerke R2 = .43; LD-EM was significant after accounting for other variables (p = .017; OR = 0.69).Conclusions:The findings are consistent with the hypothesis that memory for emotions represents a unique component of social cognition that is separate from recognition. Accuracy in identifying emotions, face recognition memory, and memory for emotions are strongly related but not wholly redundant processes. Consistent with prior literature, subjective experience of emotion had substantial effects on objective performance tasks, indicating that an individual's intense experience of their own emotions can disrupt sensitivity to the emotions of others. Future research should assess the extent to which memory for emotions relates to psychosocial outcomes such as the quality and quantity of interpersonal relationships.

Computerized analysis of facial expression reveals objective indices of blunted facial affect.

Blunted affect is associated with severe mental illness, particularly schizophrenia. Mechanisms of blunted affect are poorly understood, potentially due to a lack of phenomenological clarity. Here, we examine clinician rated blunted affect and computerized facial metrics derived from ambulatory video assessment using machine learning. With high predictive accuracy (80-82%), we found that head orientation, eye movement, and facets of mouth movement were associated with clinical ratings of blunted affect. Features denoting larger muscle movements were associated with social cognition (R2 = 0.37) and cognition (R2 = 0.40). Findings provide potential insights on psychological and pathophysiological contributors to blunted affect.

B - 14 Facial Affect Recognition among Individuals with Autism Spectrum Conditions (ASC) Compared to Individuals without ASC.

This study aims to assess whether individuals with autism spectrum conditions (ASC) are less effective in recognizing facial affect (emotions) when compared to other individuals. 1675 adult volunteers (ages 18+) were recruited via Research Match. All participants were asked to indicate their mental health history and were placed into one of two groups: (1) individuals with ASC (n = 586) and (2) individuals without ASC (n = 1089). Both groups took the Ekman Friesen Picture of Facial Affect scale (POFA). The POFA scale consists of 110 grayscale photos who posed with seven facial expressions: happy, sad, angry, fearful, surprise, disgust, and neutral. Participants were instructed to choose which emotion the photos best represented. Using the scoring manual, we recoded the participant's responses to represent whether they accurately determined the emotion presented in each photo. Next, we computed a total score for each emotion. Finally, we conducted an Independent Samples T-test for each of the emotions to compare the mean of each group's accuracy in determining the correct emotions. For all emotions, the ASD group was significantly less accurate in determining facial affect compared to the control group. Happy: t(1673) = 3.50, p = <0.001. Sad: t(1673) = 3.10, p = 0.002. Angry: t(1673) = 3.57, p = <0.001. Fearful: t(1673) = 3.55, p = <0.001. Surprised: t(1673) = 3.48, p = <0.001. Neutral: t(1673) = 5.60, p = <0.001. These findings indicate that individuals with ASC may be less accurate in determining facial affect when compared to individuals without ASC.

Associations of maternal postpartum depressive and anxiety symptoms with 4-month infant and mother self- and interactive contingency of gaze, affect, and touch.

Maternal depression and anxiety are associated with infant and mother self- and interactive difficulties. Although maternal depression and anxiety usually co-occur, studies taking this comorbidity into account are few. Despite some literature, we lack a detailed understanding of how maternal depressive and anxiety symptoms may be associated with patterns of mother-infant interaction. We examined associations of maternal postpartum depressive and anxiety symptoms with infant and mother self- and interactive patterns by conducting multi-level time-series models in a sample of 56 Turkish mothers and their 4-month infants. Time-series models assessed the temporal dynamics of interaction via infant and mother self- and interactive contingency. Videotaped face-to-face interaction was coded on a 1s time base for infant and mother gaze and facial affect, infant vocal affect, and mother touch. Results indicated that mothers with high depressive symptoms were vulnerable to infants looking away, reacting with negative touch; their infants remained affectively midrange, metaphorically distancing themselves from mothers' affect. Mothers with high anxiety symptoms were vulnerable to infants becoming facially dampened and mothers reacted with negative facial affect. Altered infant and mother self-contingency patterns were largely opposite for maternal depressive and anxiety symptoms. These patterns describe foundational processes by which maternal postpartum mood is transmitted to the infant and which may affect infant development.

Graph-Based Facial Affect Analysis: A Review

As one of the most important affective signals, facial affect analysis (FAA) is essential for developing human-computer interaction systems. Early methods focus on extracting appearance and geometry features associated with human affects while ignoring the latent semantic information among individual facial changes, leading to limited performance and generalization. Recent work attempts to establish a graph-based representation to model these semantic relationships and develop frameworks to leverage them for various FAA tasks. This paper provides a comprehensive review of graph-based FAA, including the evolution of algorithms and their applications. First, the FAA background knowledge is introduced, especially on the role of the graph. We then discuss approaches widely used for graph-based affective representation in literature and show a trend towards graph construction. For the relational reasoning in graph-based FAA, existing studies are categorized according to their non-deep or deep learning methods, emphasizing the latest graph neural networks. Performance comparisons of the state-of-the-art graph-based FAA methods are also summarized. Finally, we discuss the challenges and potential directions. As far as we know, this is the first survey of graph-based FAA methods. Our findings can serve as a reference for future research in this field.

Effect of Action Units, Viewpoint and Immersion on Emotion Recognition Using Dynamic Virtual Faces.

Facial affect recognition is a critical skill in human interactions that is often impaired in psychiatric disorders. To address this challenge, tests have been developed to measure and train this skill. Recently, virtual human (VH) and virtual reality (VR) technologies have emerged as novel tools for this purpose. This study investigates the unique contributions of different factors in the communication and perception of emotions conveyed by VHs. Specifically, it examines the effects of the use of action units (AUs) in virtual faces, the positioning of the VH (frontal or mid-profile), and the level of immersion in the VR environment (desktop screen versus immersive VR). Thirty-six healthy subjects participated in each condition. Dynamic virtual faces (DVFs), VHs with facial animations, were used to represent the six basic emotions and the neutral expression. The results highlight the important role of the accurate implementation of AUs in virtual faces for emotion recognition. Furthermore, it is observed that frontal views outperform mid-profile views in both test conditions, while immersive VR shows a slight improvement in emotion recognition. This study provides novel insights into the influence of these factors on emotion perception and advances the understanding and application of these technologies for effective facial emotion recognition training.

How do Individuals With and Without Traumatic Brain Injury Interpret Emoji? Similarities and Differences in Perceived Valence, Arousal, and Emotion Representation

Impaired facial affect recognition is common after traumatic brain injury (TBI) and linked to poor social outcomes. We explored whether perception of emotions depicted by emoji is also impaired after TBI. Fifty participants with TBI and 50 non-injured peers generated free-text labels to describe emotions depicted by emoji and rated their levels of valence and arousal on nine-point rating scales. We compared how the two groups’ valence and arousal ratings were clustered and examined agreement in the words participants used to describe emoji. Hierarchical clustering of affect ratings produced four emoji clusters in the non-injured group and three emoji clusters in the TBI group. Whereas the non-injured group had a strongly positive and a moderately positive cluster, the TBI group had a single positive valence cluster, undifferentiated by arousal. Despite differences in cluster numbers, hierarchical structures of the two groups’ emoji ratings were significantly correlated. Most emoji had high agreement in the words participants with and without TBI used to describe them. Participants with TBI perceived emoji similarly to non-injured peers, used similar words to describe emoji, and rated emoji similarly on the valence dimension. Individuals with TBI showed small differences in perceived arousal for a minority of emoji. Overall, results suggest that basic recognition processes do not explain challenges in computer-mediated communication reported by adults with TBI. Examining perception of emoji in context by people with TBI is an essential next step for advancing our understanding of functional communication in computer-mediated contexts after brain injury.