Target Emotion Research Articles

FEDA: Fine-grained emotion difference analysis for facial expression recognition

Facial expression recognition (FER) has an important role in intelligent human–computer interaction. The complexity and confusion of target emotions and the subjectivity of observers make the definition of emotion categories controversial, and low accuracy has become a bottleneck in facial expression recognition analysis. To establish an emotion representation model with efficient facial expression recognition, this study presents FEDA, a fine-grained emotion difference analysis based on correlation, and explores the issue of emotion categories with appropriate intraclass correlation and interclass differences. First, a clustering algorithm is employed to obtain variable fine-grained emotion representations. Second, the correlation is objectively analysed through recognition and facial action units. Last, an emotion representation model that supports high-efficiency FER is obtained. Through a test with the FERPlus public dataset, the recognition accuracy rate reached 91.5% for the first time, verifying the rationality of our emotion representation model. Our experimental results can also support the effective establishment of emotion representation models based on facial expression recognition and have a role in promoting the diagnosis and treatment of mental illness, as well as technological development in the fields of human–computer interaction, security, and robotics services. The codes and training logs are publicly available at https://github.com/liuhw01/FEDA.

Reading emotions in Dogs’ eyes and Dogs’ faces

Our primary goal was to investigate human ability to recognize basic emotions from only the eyes of dogs in comparison to the whole face. Simultaneously, we replicated and extended previous research (Bloom et al., 2021), while validating American canine emotional facial expression photographs cross-culturally to Brazil. Participants (N = 120) viewed behaviorally-anchored photographs of three breeds. Half the participants in each condition (faces or eyes-only) viewed two-word forced choice items while the other half viewed four-word forced choice items. Participants identified target emotions from images of both dogs’ faces and eyes-only at a higher rate than chance. Fear was accurately recognized more than the other emotions. When dogs are afraid, they open their eyes and expose the sclera, a conspicuous signal. Emotion identification accuracy was highest for the Rhodesian Ridgeback, who is similar in morphology to common Brazilian stray dogs (Vira-Latas Carmelo). We conjectured that Brazilians were more accustomed to seeing dogs with the Rhodesian Ridgeback morphology than the erect-eared breeds, thus increasing accuracy for this breed. Further studies with additional dog morphologies are desirable. In addition to research interest, our Canine Eyes task has the potential to become a test of individual differences in Theory of Mind with clinical applications.

Exploration and Application of Graphic Design Language Based on Artificial Intelligence Visual Communication

Graphic design is one of the design disciplines with the longest history in the history of human design. As a marginal discipline, it integrates science and art organically. Graphic design has three basic characteristics: information, art, and economy. Information refers to the fact that graphic design is a carrier of information communication, and graphic design works have the practical function of specific information transmission. Among them, graphic design language is the direct form of information communication, but there is a relative lack of research on the intelligent recognition of graphic design language. To address this problem, this paper focuses on the analysis method of graphic design language based on artificial intelligence visual communication. First, in order to segment different parts of the image more accurately, this paper improves a deep learning-based image segmentation method. The method uses a three-branch network structure to learn semantic information, detail information, and fusion information, respectively. The coding network uses a lightweight convolutional neural network and adds an attention mechanism in the branches to weight the importance of image feature channels, and the features extracted from different perceptual fields of the image are multiscale fused to fuse the features extracted from different stages of the coding network. Then, the salient target regions are detected on the basis of segmented images, the salient target emotions are analyzed by using feature pyramids to improve the convolutional neural network, the emotions expressed by graphic design language are analyzed by constructing a weighted loss convolutional neural network on a multilayer supervised module, and the salient target emotions are fused to obtain the final emotion classification results. The experimental results show that our proposed method outperforms traditional segmentation methods in the dataset, and the sentiment analysis based on segmented images can obtain higher sentiment classification accuracy than the sentiment analysis method that directly identifies the whole image, which is beneficial to the research and application of graphic design language.

Investigating basic emotion knowledge of people with Williams syndrome

BackgroundEmotional recognition is a key aspect of mentalizing other people’s minds. People with Williams syndrome (WS) are reported to be hypersocial and empathetic; however, few studies have investigated their emotion processing ability. AimsThis study aimed to examine emotion knowledge in people with WS and to further understand their emotion-related mentalizing ability. Methods and ProceduresSix basic emotions (happiness, sadness, anger, fear, disgust, and surprise) were tested by narrating scenarios targeting specific emotions. Participants with WS (n = 16, 8F/8 M, CA = 8.46, MA = 5.27) listened to each scenario and pointed to an emoticon displayed on a card. Correct percentages of each target emotion and replacement by other emotions were analyzed. Comparison groups comprised mental age-matched controls (n = 16, 8F/8 M, CA = 5.26) and chronological age-matched controls (n = 16, 8 F/8 M, CA = 8.31), sixth graders (n = 20, 10F/10M, CA = 12.04), and college students (n = 20, 10F/10M, CA = 20.03). All were Chinese-speaking participants with right handedness. Outcomes and ResultsPeople with WS were delayed compared to the mental age-matched group but differed from the chronological age-matched group in emotion processing. The 6th graders showed different patterns from college students, suggesting that development of emotion processing takes time. Conclusions and ImplicationsThese findings suggest (1) a gap in the maturation of emotion knowledge between people with WS and typically developing controls, (2) a nonequivalent chronological emotion-related mentalizing ability in people with WS, and (3) developmental changes in emotion processing from childhood to adulthood.

Processing emotional prosody in a foreign language: the case of German and Hebrew

This study investigated the universality of emotional prosody in perception of discrete emotions when semantics is not available. In two experiments the perception of emotional prosody in Hebrew and German by listeners who speak one of the languages but not the other was investigated. Having a parallel tool in both languages allowed to conduct controlled comparisons. In Experiment 1, 39 native German speakers with no knowledge of Hebrew and 80 native Israeli speakers rated Hebrew sentences spoken with four different emotional prosodies (anger, fear, happiness, sadness) or neutral. The Hebrew version of the Test for Rating of Emotions in Speech (T-RES) was used for this purpose. Ratings indicated participants’ agreement on how much the sentence conveyed each of four discrete emotions (anger, fear, happiness and sadness). In Experient 2, 30 native speakers of German, and 24 Israeli native speakers of Hebrew who had no knowledge of German rated sentences of the German version of the T-RES. Based only on the prosody, German-speaking participants were able to accurately identify the emotions in the Hebrew sentences and Hebrew-speaking participants were able to identify the emotions in the German sentences. In both experiments ratings between the groups were similar. These findings show that individuals are able to identify emotions in a foreign language even if they do not have access to semantics. This ability goes beyond identification of target emotion; similarities between languages exist even for “wrong” perception. This adds to accumulating evidence in the literature on the universality of emotional prosody.

Research on Classroom Emotion Recognition Algorithm Based on Visual Emotion Classification.

In this paper, we construct a classroom emotion recognition algorithm by classifying visual emotions for improving the quality of classroom teaching. We assign weights to the training images through an attention mechanism network and then add a designed loss function so that it can focus on the feature parts of face images that are not obscured and can characterize the target emotion, thus improving the accuracy of facial emotion recognition under obscuration. Analyze the salient expression features of classroom students and establish a classification criteria and criteria library. The videos of classroom students' facial expressions are collected, a multi-task convolutional neural network (MTCNN) is used for face detection and image segmentation, and the ones with better feature morphology are selected to build a standard database. A visual motion analysis method with the fusion of overall and local features of the image is proposed. To validate the effectiveness of the designed MTCNN model, two mainstream classification networks, VGG16 and ResNet18, were tested and compared with MTCNN by training on RAF-DB, masked dataset, and the classroom dataset constructed in this paper, and the final accuracy after training was 78.26% and 75.03% for ResNet18 and VGG16, respectively. The results show that the MTCNN proposed in this paper has a better recognition effect. The test results of the loss function also show that it can effectively improve the recognition accuracy, and the MTCNN model has an accuracy of 93.53% for recognizing students' facial emotions. Finally, the dataset is extended with the training method of expression features, and the experimental study shows that the method performs well and can carry out recognition effectively.

One-shot emotional voice conversion based on feature separation

The task of emotional voice conversion (EVC) aims to convert speech from one emotional state into another, while keeping linguistic content, speaker identity and other emotion-independent information unchanged. Because previous studies were limited to a specific set of emotions, it is challenging to realize the conversion of emotions never seen in training stage. In this paper, we propose a one-shot emotional voice conversion model based on feature separation. The proposed method could control emotional characteristics with Global Emotion Embeddings (GEEs), and introduce activation guidance (AG) and mutual information (MI) minimization to reduce the correlations between emotion embedding and emotion-independent representation. At run-time conversion, it could produce the desired emotional utterance from a single pairwise utterance without any emotion labels, whether the target emotion appears in the training set or not. The subjective and objective evaluations validate the effectiveness of our proposed model for both the degree of feature separation and emotion expression, even it could achieve unseen emotion conversion.

Emotion Recognition Based on Energy-related Features of Peripheral Physiological Signals.

The interest in development of methods and tools for recognizing human emotions has increased continuously. Using physiological information, especially the peripheral physiological signals, to identify emotions is an important direction for this area. This paper proposes an approach for emotion recognition based on energy-related features extracted from peripheral physiological signals. Three emotions: calm, happiness and fear, were elicited in 54 volunteers using video clips while three peripheral physiological signals were recorded: Electrocardiography (ECG), Photoplethysmography (PPG) and Respiration. Given that energy-related features of physiological signals are closely related to autonomic nervous systems activities, nine energy-related features were extracted from the recorded physiological signals. To find the optimal feature subset to represent the target emotions, the correlation between features and emotion state, as well as the discrimination ability of feature for emotion recognition were both analyzed. Four optimal features were then selected for further classification. Moreover, models based on Decision Tree (DT) were built to evaluate the performance of these features for purpose of recognition of emotion states of calm, happiness, and fear. The results show that the DT models based on these four optimal features could distinguish fear from calm (AUC=0.879, Accuracy=87.8%), happiness from calm (AUC=0.915, Accuracy=91.8%), and fear from happiness (AUC=0.822, Accuracy=81.8%), with a global recognition accuracy of 70.8%. These results indicate that energy-related features of peripheral physiological signals can reliably identify emotions, especially intense emotions.

Do Surrounding People's Emotions Affect Judgment of the Central Person's Emotion? Comparing Within Cultural Variation in Holistic Patterns of Emotion Perception in the Multicultural Canadian Society.

Previous studies in cultural psychology have suggested that when assessing a target person's emotion, East Asians are more likely to incorporate the background figure's emotion into the judgment of the target's emotion compared to North Americans. The objective of this study was to further examine cultural variation in emotion perception within a culturally diverse population that is representative of Canada's multicultural society. We aimed to see whether East-Asian Canadians tended to keep holistic tendencies of their heritage culture regarding emotion perception. Participants were presented with 60 cartoon images consisting of a central figure and four surrounding figures and were then asked to rate the central figure's emotion; out of the four cartoon figures, two were female and two were male. Each character was prepared with 5 different emotional settings with corresponding facial expressions including: extremely sad, moderately sad, neutral, moderately happy, and extremely happy. Each central figure was surrounded by a group of 4 background figures. As a group, the background figures either displayed a sad, happy, or neutral expression. The participant's task was to judge the intensity of the central figures' happiness or sadness on a 10-point Likert scale ranging from 0 (not at all) to 9 (extremely). For analysis, we divided the participants into three groups: European Canadians (N = 105), East Asian Canadians' (N = 104) and Non-East Asian/Non-European Canadians (N = 161). The breakdown for the Non-East Asian/Non-European Canadian group is as follows: 94 South Asian Canadians, 25 Middle Eastern Canadians, 23 African Canadians, 9 Indigenous Canadians, and 10 Latin/Central/South American Canadians. Results comparing European Canadians and East Asian Canadians demonstrated cultural variation in emotion judgment, indicating that East Asian Canadians were in general more likely than their European Canadian counterparts to be affected by the background figures' emotion. The study highlights important cultural variations in holistic and analytic patterns of emotional attention in the ethnically diverse Canadian society. We discussed future studies which broaden the scope of research to incorporate a variety of diverse cultural backgrounds outside of the Western educational context to fully comprehend cultural variations in context related attentional patterns.

Transformation of Emotions in Images Using Poisson Blended Generative Adversarial Networks (Student Abstract)

We propose a novel method for transforming the emotional content in an image to a specified target emotion. Existing techniques such as a single generative adversarial network (GAN) struggle to perform well on unconstrained images, especially when data is limited. Our method addresses this limitation by blending the outputs from two networks to better transform fine details (e.g., faces) while still operating on the broader styles of the full image. We demonstrate our method's potential through a proof-of-concept implementation.

The avatar will see you now: Support from a virtual human provides socio-emotional benefits

When people are in emotional distress, they often seek support. Virtual humans may provide unique and complementary benefits to human support provision, given that virtual humans are readily available and help sharers overcome socio-evaluative threats, thereby increasing willingness to disclose. Here, we examined whether talking to a virtual human elicits socio-emotional benefits, and whether this is moderated by the type of support provided. To examine the scope of the potential effect, we compared two key types of support (emotional and cognitive), across two emotions (anger and worry). Participants (N = 115) shared two personal emotional experiences with a virtual human, who provided either emotional or cognitive support via the Wizard-of-Oz method (i.e., a human-operated avatar). Results showed that participants felt better after talking to the virtual human, as evidenced by reduced intensity of the target emotion and generally improved affect. The emotional improvement was similar for emotional and cognitive support. Cognitive support was also experienced as equally effective as emotional support, and led to similar levels of experienced closeness and desire to interact with the virtual human again. These findings suggest that talking to a virtual human can be a valuable form of support at times of distress.

Applying Meta-Learning and Iso Principle for Development of EEG-Based Emotion Induction System.

Music is often used for emotion induction. ince the emotions felt when listening to it vary from person to person, customized music is required. Our previous work designed a music generation system that created personalized music based on participants' emotions predicted from EEG data. Although our system effectively induced emotions, unfortunately, it suffered from two problems. The first is that a long EEG recording is required to train emotion prediction models. In this paper, we trained models with a small amount of EEG data. We proposed emotion prediction with meta-learning and compared its performance with two other training methods. The second problem is that the generated music failed to consider the participants' emotions before they listened to music. We solved this challenge by constructing a system that adapted an iso principle that gradually changed the music from close to the participants' emotions to the target emotion. Our results showed that emotion prediction with meta-learning had the lowest RMSE among three methods (p < 0.016). Both a music generation system based on the iso principle and our conventional music generation system more effectively induced emotion than music generation that was not based on the emotions of the participants (p < 0.016).

DEF-Net: A Face Aging Model by Using Different Emotional Learnings

Face aging has attracted widespread attention in recent years, but most studies are based on the same emotional situation. Is the same person&#x2019;s aging in different emotional situations the same? To solve the above confusion, this paper proposes a novel face aging model DEF-Net, which consists of two parts: different emotional learnings (Emotion-Net) and face aging (Age-Net). Given a target emotion category, DEF-Net first assists the image from the original dataset to learn the emotion features through Emotion-Net and the generated dataset is used as the inputs of Age-Net. At the same time, multiple loss functions are used to ensure that the crucial information of the original image is not lost. Secondly, Age-Net, which has been pre-trained on the original dataset, began to adopt the generated dataset to learn the aging distribution under different emotions. Designed loss functions are utilized to ensure that the realistic target images generated by Age-Net do not lose the learned emotional characteristics. Finally, extensive experiments are used to verify the performance of DEF-Net. Compared with other state-of-the-art methods: (1) DEF-Net can learn different facial emotions across different datasets and generate corresponding realistic aging images; (2) the results achieved by our DEF-Net are demonstrated to be better than those by the model that performs face aging first and then learns different emotional characteristics.

Does emotion dysregulation mediate the association between ADHD symptoms and internalizing problems? A longitudinal within-person analysis in a large population-representative study.

Previous research has suggested that children with attention-deficit/hyperactivity disorder (ADHD) symptoms commonly show emotion dysregulation difficulties. These difficulties may partly explain the strong tendency for internalizing problems such as anxiety and depression to co-occur with ADHD symptoms. However, no study has yet provided a longitudinal analysis of the within-person links between ADHD symptoms, emotion dysregulation, and internalizing problems necessary to examine this hypothesis from a developmental perspective. We used data from the age 3, 5, and 7 waves of the large UK population-representative Millennium Cohort Study (n = 9,619, 4,885 males) and fit gender-stratified autoregressive latent trajectory models with structured residuals (ALT-SR) to disaggregate within- and between-person relations between ADHD symptom, emotion dysregulation, and internalizing problem symptoms. We found that emotion dysregulation significantly mediated the longitudinal within-person association between ADHD symptoms and internalizing problems. Results underline the promise of targeting emotion dysregulation as a means of preventing internalizing problems co-occurring with ADHD symptoms.

The Influence of Emotion Regulation on the Association Between Depression and Heart Rate Variability in Cardiac Patients.

Poor vagally mediated heart rate variability (vmHRV) is a mechanism linking depression to coronary heart disease (CHD). Reduced vmHRV is also considered an index of emotion dysregulation-the frequent use of maladaptive emotion regulation strategies, one of the most important being expressive suppression-which is a key component of depression. Therefore, this study aimed to investigate the moderating role of expressive suppression in the relation between depression and vmHRV in patients with CHD. The sample included 235 patients with CHD (mean [standard deviation] age = 61.6 [9.8] years; 12% women) admitted to cardiac rehabilitation after a cardiac intervention. The Beck Depression Inventory-II was administered to assess depressive symptoms. Emotion regulation strategies based on either expressive suppression or cognitive reappraisal were assessed through the Emotion Regulation Questionnaire. Resting electrocardiographic recordings were collected for 5 minutes to compute HRV indices. Expressive suppression moderated the relation between depressive symptoms and vmHRV ( b = -0.03, p = .012). Patients with lower expressive suppression scores showed no association between depressive symptoms and vmHRV ( b = -0.00, p = .94), whereas those with higher expressive suppression scores showed a significant negative association between depressive symptoms and vmHRV ( b = -0.05, p = .015). The use of expressive suppression is likely to potentiate the relation between depressive symptoms and poor vmHRV, which could increase the cardiac risk in these patients. Targeting emotion regulation skills in cardiac rehabilitation programs may be useful for reducing the impact of depression in cardiac patients.

Teaching Emotion Vocabulary to Children With Autism Spectrum Disorder

Research suggests that reading for understanding, especially narrative text, is difficult for children with autism spectrum disorder (ASD); yet research on how to effectively teach skills associated with reading comprehension is limited and has generally deemphasized vocabulary interventions. This single case research design study (i.e., a multiple probe across behaviors design) examined the effectiveness of teaching emotion vocabulary words to children with ASD ages 6 to 7 years. Vocabulary knowledge was taught explicitly during the repeated reading of storybooks illustrating the target emotion words. Results showed all participants increased their ability to label and define the target emotion words. Moreover, the classroom teacher indicated the intervention improved participant understanding of emotion words. Implications and future research are discussed.

HeBERT and HebEMO: A Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition

Sentiment analysis of user-generated content (UGC) can provide valuable information across numerous domains, including marketing, psychology, and public health. Currently, there are very few Hebrew models for natural language processing in general, and for sentiment analysis in particular; indeed, it is not straightforward to develop such models because Hebrew is a morphologically rich language (MRL) with challenging characteristics. Moreover, the only available Hebrew sentiment analysis model, based on a recurrent neural network, was developed for polarity analysis (classifying text as positive, negative, or neutral) and was not used for detection of finer-grained emotions (e.g., anger, fear, or joy). To address these gaps, this paper introduces HeBERT and HebEMO. HeBERT is a transformer-based model for modern Hebrew text, which relies on a BERT (bidirectional encoder representations from transformers) architecture. BERT has been shown to outperform alternative architectures in sentiment analysis and is suggested to be particularly appropriate for MRLs. Analyzing multiple BERT specifications, we find that whereas model complexity correlates with high performance on language tasks that aim to understand terms in a sentence, a more parsimonious model better captures the sentiment of an entire sentence. Notably, regardless of the complexity of the BERT specification, our BERT-based language model outperforms all existing Hebrew alternatives on all language tasks examined. HebEMO is a tool that uses HeBERT to detect polarity and extract emotions from Hebrew UGC. HebEMO is trained on a unique COVID-19-related UGC data set that we collected and annotated for this study. Data collection and annotation followed an active learning procedure that aimed to maximize predictability. We show that HebEMO yields a better performance accuracy for polarity classification. Emotion detection reaches high performance for various target emotions, with the exception of surprise, which the model failed to capture. These results are better than the best reported performance, even among English-language models of emotion detection.

Mindfulness as a Protective Factor Against Depression, Anxiety and Psychological Distress During the COVID-19 Pandemic: Emotion Regulation and Insomnia Symptoms as Mediators.

ObjectivesResearch has linked mindfulness to improved mental health, yet the mechanisms underlying this relationship are not well understood. This study explored the mediating role of emotion regulation strategies and sleep in the relationship between mindfulness and symptoms of depression, anxiety and psychological distress during the COVID-19 pandemic.MethodsAs detailed in this study’s pre-registration (osf.io/k9qtw), a cross-sectional research design was used to investigate the impact of mindfulness on mental health and the mediating role of emotion regulation strategies (i.e., cognitive reappraisal, rumination and suppression) and insomnia. A total of 493 participants from the general population answered an online survey and were included in the final analysis. The online survey consisted of the short form of the Five-Facets Mindfulness Questionnaire (FFMQ-SF), the Impact of Event Scale-revised (IES-R), the Generalised Anxiety Disorder Scale (GAD-7), the Patient Health Questionnaire (PHQ-8), the Emotion Regulation Questionnaire (ERQ), the short form of the Rumination Response Scale (RSS-SF), and the Insomnia Severity Index (ISI).ResultsStructural equation modelling revealed that mindfulness was related to lower symptoms of depression, anxiety and psychological distress, both directly and indirectly. Mindfulness was negatively associated with rumination and insomnia. As hypothesised, models revealed that the associations between mindfulness and depression, anxiety and psychological distress were significantly mediated by its negative associations with rumination and insomnia. Our findings also demonstrated that rumination was related to increased insomnia symptoms, which in turn was associated with increased mental health problems, indicating a mediated mediation. Mindfulness was also positively associated with cognitive reappraisal and negatively associated with suppression, which were, respectively, negatively and positively associated with depressive symptoms, and thus functioned as specific mediators of the association between mindfulness and depression.ConclusionOur findings suggest that rumination and insomnia operate transdiagnostically as interrelated mediators of the effects of mindfulness on mental health, whereas cognitive reappraisal and suppression function as specific mediators for depression. These insights emphasise the importance of targeting emotion regulation and sleep in mindfulness interventions for improving mental health. Limitations and implications for practice are discussed.

Who Needs Values When We Have Valuing? Comments on Jean Moritz Müller, The World-Directedness of Emotional Feeling.

Müller argues that the perceptual or “Axiological Receptivity” (AR) model of emotions is incoherent, because it requires an emotion to apprehend and respond to its formal object at the same time. He defends a contrasting view of emotions as “Position-Takings" (PT) towards “formal objects”, aspects of an emotion's target pertinent to the subject's concerns. I first cast doubt on the cogency of Müller's attack on AR as begging questions about the temporal characteristics of perceptual events. I then argue that Müller's version of PT is not radical enough. On my attitudinal view, formal objects are not values but natural properties that justify specific affective or behavioral responses. Values are constituted only by a negotiated social aggregation of individual evaluative attitudes.

Multichannel Fusion Based on modified CNN for Image Emotion Recognition

&lt;p&gt;Social media networks are an integral part of people’s daily life. Users share images and texts to express their emotions and opinions. Analyzing the image and text content published by these users can help understand and predict user behavior, so as to carry out marketing, public opinion monitoring and personalized recommendation. Weibo, Wechat and other social media are important ways of self-expression. Images are more intuitive than text. Therefore, more scholars begin to pay attention to the research of image emotion analysis. At present, image emotion analysis methods pay seldom attention to the influence of saliency object and face on image emotion expression. Therefore, we propose a multichannel fusion method based on modified CNN for image emotion recognition. Firstly, saliency target and face target region are detected in the whole image. Then feature pyramid is used to improve CNN to recognize saliency target emotion. Weighted loss CNN emotion recognition is constructed on multi-layer supervision module. Finally, the saliency target emotion, face target emotion and the directly recognized emotion on the whole image are fused to get the final result of emotion classification. Experimental results show that the proposed method can improve the accuracy of image emotion recognition.&lt;/p&gt; &lt;p&gt;&amp;nbsp;&lt;/p&gt;