Emotion Recognition Processing Research Articles

Emotion Recognition and Self-Reported Emotion Processing in Alcohol and Cannabis Co-Using Young Adults.

Alcohol and cannabis use are each associated with impairments in emotion recognition accuracy, which may promote interpersonal problems. It is unclear if emotion recognition or self-reported emotion processing differs between young adult alcohol and cannabis co-users (ACCs) and healthy controls (HCs). This study examined whether ACCs and HCs differed in their emotion recognition across two different behavioral tasks with static or dynamic faces and determined if there were differences in self-reported socio-emotional processing and alexithymia. 22 ACCs (mean age = 21.27 ± 1.75) and 25 HCs (mean age = 21.48 ± 2.68), matched on age, sex, and IQ, completed the Metrisquare Emotion Recognition Task and CANTAB Emotion Recognition Task. The ACCs and HCs were compared on task accuracy and self-reported measures, including the Social Emotional Questionnaire (SEQ) and the Perth Alexithymia Questionnaire (PAQ). No significant main effects of the Group variable or the Emotion-Group interaction variable were present for either task. The ACCs had lower SEQ (p = 0.014) and higher PAQ (p = 0.024) scores relative to the HCs, indicating greater difficulties in socio-emotional processing and identifying one's own emotions, respectively. Understanding the behavioral correlates of the self-reported difficulties in emotion processing reported by ACCs is needed to develop interventions to reduce these symptoms and promote healthy socio-emotional functioning in this population.

The role of emotion recognition in reappraisal affordances.

Emotion regulation is essential for psychological well-being. One strategy that is commonly researched is reappraisal. Individual differences regarding the tendency to use reappraisal, as well as its implications for affective experience, were extensively studied. In recent years, interest has emerged in the choice to use reappraisal, based on stimuli properties. Recently, we suggested that reappraisal is related to emotion recognition processes. Emotion recognition (and affective labeling, as an explicit form of emotion recognition) is regarded as a form of emotion regulation, however, the relations between emotion recognition and reappraisal have not been previously investigated. The aim of the current study was to explore the relationship between reappraisal affordances (the opportunities of re-interpretation that are inherent in a stimulus) and emotion recognition. For this purpose, we used the Categorized Affective Picture Database, a database that provides data regarding the emotional category of each picture, agreement levels for each category, and intensity ratings. Agreement levels were used to assess the certainty regarding the emotion evoked by the pictures. Findings suggest that reappraisal affordance is predicted by both agreement levels and intensity, in negative pictures alone. In negative pictures, intensity was negatively correlated with the difficulty to reappraise. These findings strengthen the hypothesis regarding the relationship between emotion recognition and reappraisal, and provide evidence for the role of emotion recognition in reappraisal affordances.

Improving Speech Recognition with Convolutional Neural Networks

This project explores advanced techniques in speech recognition, focusing on emotion identification using Convolutional Neural Networks for improved accuracy and real-time processing efficiency. Emotion recognition from speech signals plays a crucial role in various applications, including human-computer interaction, customer service, mental health monitoring, and entertainment. This project proposes an innovative approach to emotion recognition using Convolutional Neural Networks (CNNs) applied to speech data. By leveraging advanced deep learning techniques, the proposed system aims to accurately identify and classify emotions conveyed through vocal expressions. The project begins with a comprehensive review of existing literature on emotion recognition and speech processing, identifying key challenges and opportunities in the field. Building upon prior research, the project introduces a novel CNN architecture optimized for emotion recognition tasks. This architecture is designed to extract relevant features from speech signals and capture subtle nuances indicative of different emotional states. One of the distinguishing features of the proposed approach is its multi-modal integration, which combines information from both audio and visual modalities to enhance emotion recognition accuracy. In addition to analysing speech signals, the system incorporates visual cues such as facial expressions and gestures, providing a more comprehensive understanding of the speaker's emotional state. Real-time processing efficiency is prioritized in the design of the system, ensuring prompt and responsive emotion recognition in interactive applications. Optimization techniques such as model quantization and lightweight architecture design are employed to minimize computational overhead while maintaining high accuracy. To address the variability and subjectivity of emotional expression, the system incorporates user-specific adaptation mechanisms. Through continuous learning and feedback integration, the system dynamically adapts to individual speakers' speech patterns and emotional characteristics, enhancing its ability to accurately recognize emotions in diverse contexts. The project also explores ensemble learning strategies to improve robustness and generalization performance. By combining predictions from multiple CNN models trained on diverse datasets, the system achieves greater resilience to variations in emotional expression and environmental factors. Ethical considerations, including privacy protection and responsible data handling, are integral aspects of the project's design and implementation. Measures are implemented to ensure the ethical collection, storage, and usage of speech data, safeguarding user privacy and maintaining trust in the system. Overall, the proposed system represents a significant advancement in emotion recognition technology, offering a sophisticated and versatile solution for accurately identifying emotions from speech signals. By leveraging deep learning techniques, multi-modal integration, real-time processing optimization, user-specific adaptation, and ensemble learning, the system demonstrates promising potential for various practical applications requiring robust and context-aware emotion recognition capabilities. Keywords: Speech Recognition, Emotion Identification, Convolutional Neural Networks (CNNs), Real-time Processing, Multi-modal Integration, User-specific Adaptation, Ensemble Learning, Deep Learning, Emotional Expression, Ethical Data Handling

Deep learning-based sentiment analysis for social media: A focus on multimodal and aspect-based approaches

Commonly referred to as opinion mining, sentiment analysis harnesses the power of deep learning systems to discern human emotions and subjective sentiments towards a wide array of subjects. As such, it has become an integral tool in identifying and distinguishing sentences that harbor emotional biases or trends. By systematically examining sentiment-tinged data, researchers can unearth pivotal insights that not only reflect current perspectives but also predict future behaviors and trends. This process involves intricate computational models that analyze and interpret the emotional undertones embedded within a body of text. Whether these undertones are positive, negative, or neutral, sentiment analysis allows us to delve into the subtle nuances of human communication. This ability to "understand" and quantify sentiment is particularly vital in our modern digital age, where opinions and reviews shared through social media and online platforms can greatly influence public sentiment and consumer behavior. By extending beyond the literal meanings of words and phrases, sentiment analysis can provide a more comprehensive understanding of how people truly feel. It is instrumental in fields as diverse as marketing, politics, social science, and even artificial intelligence development, given its potential to gauge public opinion and predict societal trends. This paper aims to consolidate relevant research within the field of sentiment analysis conducted in recent years. Furthermore, it seeks to prognosticate the future trajectories and impacts of this rapidly evolving domain. Emphasis is placed on the role of deep learning and its transformational effects on the approach and capabilities of sentiment analysis, anticipating how its further advancement will continue to refine this intricate process of emotion recognition and interpretation.

Emotion Recognition From Gait Analyses: Current Research and Future Directions

Human gait refers to a daily motion that represents not only mobility but can also be used to identify the walker by either human observers or computers. Recent studies reveal that gait even conveys information about the walker’s emotion. Individuals in different emotion states may show different gait patterns. The mapping between various emotions and gait patterns provides a new source for automated emotion recognition. Compared to traditional emotion detection biometrics, such as facial expression, speech, and physiological parameters, gait is remotely observable, more difficult to imitate, and requires less cooperation from the subject. These advantages make gait a promising source for emotion detection. This article reviews current research on gait-based emotion detection, particularly on how gait parameters can be affected by different emotion states and how the emotion states can be recognized through distinct gait patterns. We focus on the detailed methods and techniques applied in the whole process of emotion recognition: data collection, preprocessing, and classification. Finally, we discuss possible future developments of efficient and effective gait-based emotion recognition using state-of-the-art techniques in intelligent computation and big data.

The role of leadership level in college students’ facial emotion recognition: evidence from event-related potential analysis

While the role of emotion in leadership practice is well-acknowledged, there is still a lack of clarity regarding the behavioral distinctions between individuals with varying levels of leadership and the underlying neurocognitive mechanisms at play. This study utilizes facial emotion recognition in conjunction with electroencephalograms to explore the temporal dynamics of facial emotion recognition processes among college students with high and low levels of leadership. The results showed no significant differences in the amplitude of P1 during the early stage of facial emotion recognition between the two groups. In the middle stage of facial emotion recognition, the main effect of group was significant on the N170 component, with higher N170 amplitude evoked in high-leadership students than low-leadership students. In the late stage of facial emotion recognition, low-leadership students evoked greater LPP amplitude in the temporal-parietal lobe when recognizing happy facial emotions compared to high-leadership students. In addition, time–frequency results revealed a difference in the alpha frequency band, with high-leadership students exhibiting lower alpha power than low-leadership students. The results suggest differences in the brain temporal courses of facial emotion recognition between students with different leadership levels, which are mainly manifested in the middle stage of structural encoding and the late stage of delicate emotional processing during facial emotion recognition.

Analysis of emotion recognition based on brain-computer interface technology

The lifetime prevalence of social anxiety disorder is pretty high, and Brain-computer Interface (BCI) technology has become the main solution to the above problems. In order to solve this problem, this paper firstly introduced the sentiment classification and the theoretical basis of BCI emotion recognition, and bring up the specific mechanism and detailed process of emotion recognition. Secondly the selection of data sets and the detailed process of EEG signal analysis are mainly discussed, including pre-processing, feature extraction, classification of signal recognition, accuracy analysis and so on. Thirdly, the paper summarized some examples of current representative applications and the existing problems, improvements and breakthroughs in recent years. In the end, the future development trend of affective brain-computer interface is prospected. Although the brain computer interface (BCI) emotion recognition is still an immature technology in the initial stage and has so many challenging problems, but this paper wish and believe it can have a brilliant development prospect.

Research on the Comparative Development of Modern Popular Music and Traditional Music Culture in Colleges and Universities in the Age of Artificial Intelligence

Abstract In this paper, the forward neural network multi-feature fusion algorithm is used to extract the emotional features of music culture on artificial intelligence technology, considering the diversity and intermittency of the emotional features of the study, which needs to be parameterized. In the forward neural network architecture, the activation value obtained by using the nonlinear activation function is used, and the results obtained are passed to the next layer of data to realize layer-by-layer forward computation, which leads to the back-propagation activation function. The music culture emotion classification model is constructed based on the propagation mode of the forward neural network to determine the emotion recognition process. The research object is selected, the research process is determined, and in order to ensure the true validity of the research, it is necessary to test the reliability and validity of the research design scheme and to develop an empirical analysis of the comparison between popular music and traditional music culture. The results show that on the model, especially in the recognition of sacred, sad, passionate emotion type of music classification accuracy reached more than 88.2%. This paper’s model can improve the classification accuracy of music emotion to a certain extent. In the ontological knowledge analysis of popular music and traditional music culture, all three editions of textbooks show that general knowledge of music is predominant and has a large proportion, appreciation knowledge and extended knowledge are also considerable, and music knowledge is the least and has a small proportion. This study demonstrates the synergistic development of traditional culture and modern popular music, which is of great significance to the development of music education in colleges and universities.

Facial Emotion Recognition usign CNN

Nowadays where most of the works are being carried online the demand for face recognition technique is elevated. Computerized software is assisting in identifying human feelings such as happiness, sadness, anger, fear, disgust, etc. Over the decades, various research has been carried on the facial expression and emotion recognition. Emotion detection has extended applications. It is not merely related to any specific field nut instead the approach ranges from communication, advertising to hospital requisition and many more. To exist collective mechanisms through which we can accomplish the process of facial emotion recognition. In this paper we are using Convolutional Neural Network for the implementation. Upon exploring numerous datasets for the procedure of experiment we have chosen to go with Kaggle dataset.

Can orange colour facilitate the processing of happiness? An exploration study on happiness metaphor.

Colour is a ubiquitous perceptual stimulus, and theoretical models of colour and psychological functioning posit that colour plays a key role in influencing the behaviour and mental function of a person. One investigation and two experiments investigated the colour metaphor representation of happiness concepts and the mapping mode of the colour metaphor of happiness concepts. A questionnaire was conducted to explore the relationship between colour preference and happiness. Study 2 shows that the identification of happiness words was facilitated more when words were viewed on an orange background than when viewed on a blue background. Study 3 further verifies the links of the connection between colour and happiness at the sentence level, and the orange-happiness facilitation effect was replicated. These results document a novel influence of colour on emotion recognition processes, where an orange background may facilitate the processing of the concept of happiness and provide support for conceptual metaphor theory and colour-in-context theory.

To see or not to see: the parallel processing of self-relevance and facial expressions

The self, like the concept of central "gravity", facilitates the processing of information that is directly relevant to the self. This phenomenon is known as the self-prioritization effect. However, it remains unclear whether the self-prioritization effect extends to the processing of emotional facial expressions. To fill this gap, we used a self-association paradigm to investigate the impact of self-relevance on the recognition of emotional facial expressions while controlling for confounding factors such as familiarity and overlearning. Using a large and diverse sample, we replicated the effect of self-relevance on face processing but found no evidence for a modulation of self-relevance on facial emotion recognition. We propose two potential theoretical explanations to account for these findings and emphasize that further research with different experimental designs and a multitasks measurement approach is needed to understand this mechanism fully. Overall, our study contributes to the literature on the parallel cognitive processing of self-relevance and facial emotion recognition, with implications for both social and cognitive psychology.

Research on face emotion recognition algorithm based on deep learning neural network

Abstract This paper explores face emotion recognition based on deep learning neural networks in order to make the computer recognize facial emotion more accurately. Firstly, we introduce the convolutional neural network, which consists of a convolutional layer, pooling layer and fully connected layer in the deep learning neural network, and propose the variational self-encoder face emotion recognition algorithm based on the convolutional neural network in combination with the facial emotion recognition process. The correct recognition rates achieved by EVA using 32×32 pixels, LBP features and 50-dimensional Gabor wavelet features are 95.13%, 96.74% and 98.84%, respectively. In terms of emotion recognition ability, the recognition accuracy percentages were around 70% for large emotion fluctuations and around 30% for small emotion fluctuations. Since the neutral facial emotion is not well distinguished, the recognition degree of neutral facial emotion is only 80%, and the recognition degree using the self-encoder-based face emotion recognition algorithm in several facial features extraction is above 80%, and the recognition degree with neutral emotion removed is up to about 90%. Therefore, the algorithm in this paper has good classification performance and can recognize and analyze facial emotion features accurately.

Establishing Relationship between Emotional Prosody and Affective Pragmatics

This work aims to provide theoretical grounds to establish a relationship between emotional prosody and affective pragmatics. Affective pragmatics is a significant yet least studied area of research. This theoretical framework focuses how emotional expressions are constructive in channelizing the pragmatic meaning under the umbrella of affective pragmatics and also encompasses the speech engineering that conveys unabridged abstract emotions in the phenomenal process of emotion recognition. Since speech act theory focuses meaning at utterance level and not at emotional level, thus, there is a need to reflect on emotional expressions that function as paralinguistic features. There are several studies carried out on identification of emotions using prosodic modeling; however, there is no meticulous study that shows relationship between emotions and their pragmatic meanings. The current study will be an application of Theory of Affective Pragmatics (TAP) proposed by Andrea Scarantino in 2017, a theory analogous to speech act theory. The objectives of the study can be achieved through analysis of quantitative data procured taking advantage of the emotional recordings from Emotional Prosody Speech and Transcripts (EPST), a worldwide database used in emotion recognition processes, along with employment of appropriate prosodic features using a Hidden Markov Model (HMM), a popular emotion recognition statistical model. The outcomes of this study will contribute to benefit new researchers in the field of linguistics to understand affective pragmatics at a profound level as a novel area of research.

Optimizing 1D-CNN-Based Emotion Recognition Process through Channel and Feature Selection from EEG Signals.

EEG-based emotion recognition has numerous real-world applications in fields such as affective computing, human-computer interaction, and mental health monitoring. This offers the potential for developing IOT-based, emotion-aware systems and personalized interventions using real-time EEG data. This study focused on unique EEG channel selection and feature selection methods to remove unnecessary data from high-quality features. This helped improve the overall efficiency of a deep learning model in terms of memory, time, and accuracy. Moreover, this work utilized a lightweight deep learning method, specifically one-dimensional convolutional neural networks (1D-CNN), to analyze EEG signals and classify emotional states. By capturing intricate patterns and relationships within the data, the 1D-CNN model accurately distinguished between emotional states (HV/LV and HA/LA). Moreover, an efficient method for data augmentation was used to increase the sample size and observe the performance deep learning model using additional data. The study conducted EEG-based emotion recognition tests on SEED, DEAP, and MAHNOB-HCI datasets. Consequently, this approach achieved mean accuracies of 97.6, 95.3, and 89.0 on MAHNOB-HCI, SEED, and DEAP datasets, respectively. The results have demonstrated significant potential for the implementation of a cost-effective IoT device to collect EEG signals, thereby enhancing the feasibility and applicability of the data.

Facial emotion recognition processes according to schizotypal personality traits: An eye-tracking study

Facial emotion recognition has been shown to be impaired among patients with schizophrenia and, to a lesser extent, among individuals with high levels of schizotypal personality traits. However, aspects of gaze behavior during facial emotion recognition among the latter are still unclear. This study therefore investigated the relations between eye movements and facial emotion recognition among nonclinical individuals with schizotypal personality traits.A total of 83 nonclinical participants completed the Schizotypal Personality Questionnaire (SPQ) and performed a facial emotion recognition task. Their gaze behavior was recorded by an eye-tracker. Self-report questionnaires measuring anxiety, depressive symptoms, and alexithymia were administered.At the behavioral level, correlation analyses showed that higher SPQ scores were associated with lower surprise recognition accuracy scores. Eye-tracking data revealed that higher SPQ scores were associated with shorter dwell time on relevant facial features during sadness recognition. Regression analyses revealed that the total SPQ score was the only significant predictor of eye movements during sadness recognition, and depressive symptoms were the only significant predictor of surprise recognition accuracy. Furthermore, dwell time predicted response times for sadness recognition in that shorter dwell time on relevant facial features was associated with longer response times.Schizotypal traits may be associated with decreased attentional engagement in relevant facial features during sadness recognition and impede participants' response times. Slower processing and altered gaze patterns during the processing of sad faces could lead to difficulties in everyday social situations in which information must be rapidly processed to enable the successful interpretation of other people's behavior.

Emotion recognition in autism spectrum disorder across age groups: A cross-sectional investigation of various visual and auditory communicative domains.

Previous research on emotion processing in autism spectrum disorder (ASD) has predominantly focused on human faces and speech prosody, with little attention paid to other domains such as nonhuman faces and music. In addition, emotion processing in different domains was often examined in separate studies, making it challenging to evaluate whether emotion recognition difficulties in ASD generalize across domains and age cohorts. The present study investigated: (i) the recognition of basic emotions (angry, scared, happy, and sad) across four domains (human faces, face-like objects, speech prosody, and song) in 38 autistic and 38 neurotypical (NT) children, adolescents, and adults in a forced-choice labeling task, and (ii) the impact of pitch and visual processing profiles on this ability. Results showed similar recognition accuracy between the ASD and NT groups across age groups for all domains and emotion types, although processing speed was slower in the ASD compared to the NT group. Age-related differences were seen in both groups, which varied by emotion, domain, and performance index. Visual processing style was associated with facial emotion recognition speed and pitch perception ability with auditory emotion recognition in the NT group but not in the ASD group. These findings suggest that autistic individuals may employ different emotion processing strategies compared to NT individuals, and that emotion recognition difficulties as manifested by slower response times may result from a generalized, rather than a domain-specific underlying mechanism that governs emotion recognition processes across domains in ASD.

A Deep Learning Method Using Gender-Specific Features for Emotion Recognition.

Speech reflects people's mental state and using a microphone sensor is a potential method for human-computer interaction. Speech recognition using this sensor is conducive to the diagnosis of mental illnesses. The gender difference of speakers affects the process of speech emotion recognition based on specific acoustic features, resulting in the decline of emotion recognition accuracy. Therefore, we believe that the accuracy of speech emotion recognition can be effectively improved by selecting different features of speech for emotion recognition based on the speech representations of different genders. In this paper, we propose a speech emotion recognition method based on gender classification. First, we use MLP to classify the original speech by gender. Second, based on the different acoustic features of male and female speech, we analyze the influence weights of multiple speech emotion features in male and female speech, and establish the optimal feature sets for male and female emotion recognition, respectively. Finally, we train and test CNN and BiLSTM, respectively, by using the male and the female speech emotion feature sets. The results show that the proposed emotion recognition models have an advantage in terms of average recognition accuracy compared with gender-mixed recognition models.

Are you angry? Neural basis of impaired facial expression recognition in pre-manifest Huntington's

IntroductionEarly non-motor symptoms in Huntington's disease (HD), including visual perceptual difficulties, can have profound negative impacts on quality of life. In particular, deficits in emotion recognition may contribute to misinterpretation of social cues, and may adversely affect interpersonal relationships, work relationships and/or general well-being. This may be particularly salient during the pre-manifest period, a period prior to the onset of motor symptoms. We sought to evaluate impairments in emotion recognition in gene-positive individuals who did not meet criterial for a diagnosis of HD; we also sought to determine associations between emotion recognition processing and altered cortico-striatal circuitry. MethodsWe used a standardized battery to evaluate performance on a facial expression recognition task in a cohort of motor pre-manifest HD (Pre-HD) individuals (N = 21). Functional MRI (fMRI) was then used to assess the face processing network in a subset (N = 15). ResultsWe found significantly decreased response accuracy to certain facial expressions, particularly of negative emotions (p < 0.001) in Pre-HDs. When Pre-HDs viewed faces with different emotions, activation within the Superior Temporal Sulcus (fSTS) was reduced compared to controls; in contrast, the level of evoked response within other face-selective cortical regions was comparable. ConclusionEarly deficits in emotion recognition in Pre-HD appear to be associated with alterations in the fSTS response, a distinctly different pathway from that involved in face perception and provide support for early cognitive and behavioral interventions.

Development of Emotion Recognition from Facial Expressions with Different Eye and Mouth Cues in Japanese People

Research has reported that Japanese people are more likely to focus on and look longer at eyes when reading emotions from facial expressions than their western counterparts. However, how these tendencies develop and whether there is a relationship between the two tendencies (to focus on the eyes and to look longer at the eyes) is unclear. The present study examined emotion recognition and gaze patterns in Japanese preschool children (n = 51) and university students (n = 57), using facial expressions with different eye and mouth cues. The results showed developmental changes in emotion recognition, with adults being more sensitive to negative emotions, whereas gaze patterns showed no developmental changes. Furthermore, there was no relationship between emotion recognition and gaze patterns. This suggests that the implicit and explicit processing of emotion recognition develops at different times, and that there is no direct relationship between the two processes.

Design of a team emotion forecast tool for bettering individual emotional well-being and team performance

Individual emotion can significantly impact one's well-being and a team's emotional environment can affect team performance. In addition, one's emotions in a team can affect others’ emotions. Thus, in order to have a positive team emotional state (environment) and avoid one's negative emotions propagating to others, a team emotion forecast tool is needed for teamwork management. How to design an easy-to-use team emotion forecast tool is our research question. In this study, we first propose to use keystroke and mouse dynamics as potentially effective signals to recognize emotions because of their ubiquity in the work environment, especially in remote work. And the emotion recognition process is non-intrusive and unobtrusive, which helps to identify emotions more objectively. Second, the user-centered design method is used to design such a tool/system. Third, a qualitative study has been conducted to analyze users’ demands and concerns. At last, a digital prototype has been designed and developed.