Emotion Classification Research Articles

TPRO-NET: an EEG-based emotion recognition method reflecting subtle changes in emotion

Emotion recognition based on Electroencephalogram (EEG) has been applied in various fields, including human–computer interaction and healthcare. However, for the popular Valence-Arousal-Dominance emotion model, researchers often classify the dimensions into high and low categories, which cannot reflect subtle changes in emotion. Furthermore, there are issues with the design of EEG features and the efficiency of transformer. To address these issues, we have designed TPRO-NET, a neural network that takes differential entropy and enhanced differential entropy features as input and outputs emotion categories through convolutional layers and improved transformer encoders. For our experiments, we categorized the emotions in the DEAP dataset into 8 classes and those in the DREAMER dataset into 5 classes. On the DEAP and the DREAMER datasets, TPRO-NET achieved average accuracy rates of 97.63%/97.47%/97.88% and 98.18%/98.37%/98.40%, respectively, on the Valence/Arousal/Dominance dimension for the subject-dependent experiments. Compared to other advanced methods, TPRO-NET demonstrates superior performance.

Using Vocal-Based Emotions as a Human Error Prevention System with Convolutional Neural Networks

Human error is a mark assigned to an event that has negative effects or does not produce a desired result, with emotions playing an important role in how humans think and behave. If we detect feelings early, it may decrease human error. The human voice is one of the most powerful tools that can be used for emotion recognition. This study aims to reduce human error by building a system that detects positive or negative emotions of a user like (happy, sad, fear, and anger) through the analysis of the proposed vocal emotion component using Convolutional Neural Networks. By applying the proposed method to an emotional voice database (RAVDESS) using Librosa for voice processing and PyTorch, with the emotion classification of (happy/angry), the results show a better accuracy (98%) in comparison to the literature with regard to making a decision to deny or allow a user to access sensitive operations or send a warning to the system administrator prior to accessing system resources.

Exercise-induced inflammation alters the perception and visual exploration of emotional interactions

IntroductionThe study aimed to investigate whether an exercise-induced pro-inflammatory response alters the perception as well as visual exploration of emotional body language in social interactions. MethodsIn a within-subject design, 19 male, healthy adults aged between 19 and 33 years performed a downhill run for 45 min at 70% of their VO2max on a treadmill to induce maximal myokine blood elevations, leading to a pro-inflammatory status. Two control conditions were selected: a control run with no decline and a rest condition without physical exercise. Blood samples were taken before (T0), directly after (T1), 3 h after (T3), and 24 h after (T24) each exercise for analyzing the inflammatory response. 3 h after exercise, participants observed point-light displays (PLDs) of human interactions portraying four emotions (happiness, affection, sadness, and anger). Participants categorized the emotional content, assessed the emotional intensity of the stimuli, and indicated their confidence in their ratings. Eye movements during the entire paradigm and self-reported current mood were also recorded. ResultsThe downhill exercise condition resulted in significant elevations of measured cytokines (IL6, CRP, MCP-1) and markers for muscle damage (Myoglobin) compared to the control running condition, indicating a pro-inflammatory state after the downhill run. Emotion recognition rates decreased significantly after the downhill run, whereas no such effect was observed after control running. Participants' sensitivity to emotion-specific cues also declined. However, the downhill run had no effect on the perceived emotional intensity or the subjective confidence in the given ratings. Visual scanning behavior was affected after the downhill run, with participants fixating more on sad stimuli, in contrast to the control conditions, where participants exhibited more fixations while observing happy stimuli. ConclusionOur study demonstrates that inflammation, induced through a downhill running model, impairs perception and emotional recognition abilities. Specifically, inflammation leads to decreased recognition rates of emotional content of social interactions, attributable to diminished discrimination capabilities across all emotional categories. Additionally, we observed alterations in visual exploration behavior. This confirms that inflammation significantly affects an individual's responsiveness to social and affective stimuli.

Gender differences in the quantitative and qualitative assessment of chronic pain among older people.

Pain, regardless of its causes, is a subjective and multidimensional experience that consists of sensory, emotional and cognitive factors that cannot be adequately captured by a single number on a pain scale. The aim of the study was to understand gender differences in the assessment of quantitative and qualitative chronic pain among older people. The study used a questionnaire that included questions about demographic and social characteristics as well as the following scales: Abbreviated Mental Score (AMTS), Personal Activities of Daily Living (PADL) by Katz, Instrumental Activities of Daily Living (IADL) by Lawton, Geriatric Depression Scale (GDS-15), McGill Pain Questionnaire (MPQ). The pain rating index based on rank values of adjectives was higher among women than men (18.36 ± 7.81 vs. 17.17 ± 9.69, p = 0.04). The analysis of the frequency of selection of individual adjectives describing the sensory aspects of pain showed that men described the pain as "stabbing" more often than women (26.1% vs. 14.3%, p < 0.05). Women chose adjectives from the emotional category more often than men (59.8% vs. 75.4%, p < 0.05), describing the pain as "disgusting" (8.9% vs. 1.4%, p < 0.05), "unbearable" (19.6 vs. 4.3, p < 0.05). In the subjective category, there was a difference between women and men in terms of describing pain as "terrible" (23.2% vs. 7.2%, p < 0.05) and as "unpleasant" (11.6% vs. 23.3%, p < 0 0.05). When referring to pain, women tend to employ more detailed and factual language, indicative of heightened emotional sensitivity. Men tend to use fewer words and focus on the sensory aspects of pain. Subjective aspects of pain were demonstrated by both women and men.

Women’s cycling rides: Collective pathways to optimal experience

Women’s cycling rides are leisure events where individual and group elements may jointly contribute to optimal experience. This study was aimed at building knowledge about the narratives underlying optimal experience of these collective (noncompetitive) events. Twenty-nine women who participated in group cycling rides and challenges in Spain and in Colombia were interviewed. Csíkszentmihályi’s seminal work on shared flow and Durkheim’s collective gatherings guided the analysis in the coding stages. By considering participation in these events from a time-perspective, we examined previous circumstances, observed the evolution of social identification with the group from a relational perspective, and explored outcomes from a well-being standpoint. Central to the case that the experience is optimal is the interconnection of individual and shared flow, as well as emotional synchrony, social categorization, and empowerment. Group commitment, sorority, prosociality, and eudaimonic well-being and identity are revealed post-event.

Optimizing Emotional Insight through Unimodal and Multimodal Long Short-term Memory Models

The field of multimodal emotion recognition is increasingly gaining popularity as a research area. It involves analyzing human emotions across multiple modalities, such as acoustic, visual, and language. Emotion recognition is more effective as a multimodal learning task than relying on a single modality. In this paper, we present an unimodal and multimodal long short-term memory model with a class weight parameter technique for emotion recognition on the CMU-Multimodal Opinion Sentiment and Emotion Intensity dataset. In addition, a critical challenge lies in selecting the most effective fusion method for integrating multiple modalities. To address this, we applied four different fusion techniques: Early fusion, late fusion, deep fusion, and tensor fusion. These fusion methods improved the performance of multimodal emotion recognition compared to unimodal approaches. With the highly imbalanced number of samples per emotion class in the MOSEI dataset, adding a class weight parameter technique leads our model to outperform the state of the art on all three modalities — acoustic, visual, and language — as well as on all the fusion models. The challenges of class imbalance, which can lead to biased model performance, and using an effective fusion method for integrating multiple modalities often result in decreased accuracy in recognizing less frequent emotion classes. Our proposed model shows 2–3% performance improvement in the unimodal and 2% in the multimodal over the state-of-the-art achieved results.

Emotion classification using electrocardiogram and machine learning: A study on the effect of windowing techniques

Automated emotion recognition using physiological signals has gained significant attention in recent years due to its potential applications in human–computer interaction, healthcare, and psychology. Electrocardiogram (ECG) signals are widely used due to their non-invasiveness, high temporal resolution, and direct relationship with the autonomic nervous system. In this study, we propose a novel approach for ECG-based emotion recognition using time-series to image encoding techniques and texture-based features combined with machine learning algorithms and deep learning architectures. The ECG data used in this study were obtained from the Continuously Annotated Signals of Emotion (CASE) and Wearable Stress and Affect Detection (WESAD) datasets. We categorized emotional states based on valence and arousal annotations into four classes: High-Valence High-Arousal, High-Valence Low-Arousal, Low-Valence High-Arousal, and Low-Valence Low-Arousal. The ECGs were segmented into 5 and 7-window segments and transformed into 2D representations using Gramian Angular Summation Field, Markov Transition Field, Recurrence Plot (RP), and the triple-channel fusion of all these images. Total of 85 textural features based on the Gray-Level Co-occurrence Matrix (GLCM), Gray-Level Run Length Matrix, Zernike’s Moments, Hu’s Moments, Fractal Dimension Texture Analysis (FDTA), and First-Order Statistics were extracted. Classifiers, namely Random Forest, Support Vector Machine (SVM), eXtreme Gradient Boosting (XGB), 1D Convolutional Neural Network (CNN), and Multi-head Attention Network were considered to classify the emotional states. The performance of the classifiers varied depending on the time-series to image encoding technique, segmentation approaches, and the classifier employed. We achieved the highest Weighted F-measure (F-m) of 94.91% (RP + XGB) and 86.78% (RP + SVM) using the 7-window and 5-window approaches, respectively. Our proposed 1D CNN architecture achieved the highest classification metrics (F-m = 92.52%, Balanced accuracy = 92.0%, Recall = 91.96%, and Precision = 93.16%) with RP images in a 7-window approach. GLCM and FDTA features made significant contributions to the classification of emotional states. Overall, our results suggest that the proposed method holds promise for developing more accurate and efficient emotion recognition systems.

Human Face Emotion Recognition Using Deep Learning

Human Emotion detection from image is one of the most powerful and challenging research task in social communication. Deep learning based emotion detection gives performance better than traditional methods with image processing. This system is an artificial intelligence systemcapableofemotion detectionthrough facial expressions. It discusses about the procedure of emotion detection, which includes basically three main steps: face detection, features extraction, and emotion classification. This model proposed a convolutionalneural networks(CNN)and uses 3D face models based on deep learning architecture for emotion detection fromimages. Applicationsof face recognition detection extend across diverse fields and addressing challenges related to robustness, privacy, and ethical considerations is essential for the widespread and responsible adoption of face recognition technologyinthesefields.

Exploring Human Emotions: A Virtual Reality-Based Experimental Approach Integrating Physiological and Facial Analysis

This paper researches the classification of human emotions in a virtual reality (VR) context by analysing psychophysiological signals and facial expressions. Key objectives include exploring emotion categorisation models, identifying critical human signals for assessing emotions, and evaluating the accuracy of these signals in VR environments. A systematic literature review was performed through peer-reviewed articles, forming the basis for our methodologies. The integration of various emotion classifiers employs a ‘late fusion’ technique due to varying accuracies among classifiers. Notably, facial expression analysis faces challenges from VR equipment occluding crucial facial regions like the eyes, which significantly impacts emotion recognition accuracy. A weighted averaging system prioritises the psychophysiological classifier over the facial recognition classifiers due to its higher accuracy. Findings suggest that while combined techniques are promising, they struggle with mixed emotional states as well as with fear and trust emotions. The research underscores the potential and limitations of current technologies, recommending enhanced algorithms for effective interpretation of complex emotional expressions in VR. The study provides a groundwork for future advancements, aiming to refine emotion recognition systems through systematic data collection and algorithm optimisation.

The meaning of touch: Relational and individual variables shape emotions and intentions associated with imagined social touch.

Touch is a key channel for conveying meaning in social interactions. The affective quality of touch and its effects on well-being are shaped by relational context (relationship between touch giver vs. recipient) and person variables (e.g. adult attachment style). Yet, such effects have not been explored in relation to the meaning ascribed to touch. We used data from the Touch Test, the world's largest touch survey, which included questions on the degree to which people felt and related specific emotions and intentions to imagined gentle stroking touch and hugs. In N = 23,428, we examined how relational context (imagined source of touch) and person variables (gender, recalled positive childhood touch and adult attachment style) were associated with positive (e.g. love, desire, support) and negative (e.g. fear, anger, warning) emotions and intentions related to imagined touch. Love, desire and support were endorsed more when participants had had their partner (vs. someone else) in mind, and women (vs. men) gave lower ratings for desire overall. Gentle stroking touch was most linked with arousal when participants had had their partner in mind. Further, more positive childhood touch and secure and anxious attachment scores were associated with more positive emotions and intentions, while the opposite was found for avoidant attachment scores. Lastly, positive childhood touch and higher anxious attachment scores were related to greater discrimination between distinct emotion and intention categories, while higher attachment avoidance was associated with reduced discriminability. Thus, contextual and person variables matter in shaping the meaning of social touch.

Implementation of Text Mining to Detect Emotions of Fuel Price Increase Using BERT-LSTM Method

Fuel is essential in everyday life to be used as a means of transportation, which generally uses fuel oil. In early April 2022, there was an issue that caused an uproar in Indonesian society: rising fuel prices. Based on the problems described, this research uses emotion classification with the BERT and LSTM methods using social media data, such as YouTube, to classify types of emotions based on community responses to government decisions. The highest results were obtained when oversampling was used, with an accuracy of 95%, while without oversampling, the accuracy was only 47%. In the experiments that have been carried out, it turns out that using the oversampling method and the BERT method for emotion classification will cause a reduction in accuracy for the test section.

AI Driven Chatbot Counsellor

Speech and written information are fundamental to human communication. As a result, the majority of spoken and textual communication takes place on digital platforms like Twitter, Facebook, and WhatsApp, among others. Our model employs dual recurrent neural networks (RNNs) to encode the information from text and audio sequences, as spoken language and sound constitute emotional discourse. The emotion class is then predicted by combining the data from the two sources. Due to the complexity of speech emotion recognition, models that use audio properties to generate powerful classifiers have become increasingly important. Completing sentences or crafting sentences from a given starting word is a major aspect of natural language processing. In a way, it shows if a machine is capable of human creativity and mental processes. In order to assist handle diverse phrase generation challenges, we use natural language processing to train the machine for certain tasks. This is especially useful for application situations such as machine translation, automatic question answering, and summary creation. At the moment, OpenAI GPT and BERT are the most widely used language models for text generation and prediction. The approaches based on handwritten instructions, patterns, or statistical methods have been quickly superseded by the latest developments in deep learning and artificial intelligence, such as end-to-end trainable neural networks. This research presents a novel approach to deep neural learning-based chatbot creation. This approach builds a multilayer neural network to analyse and learn from the data. Furthermore, we employ supplementary limitations on the generation model to generate the correct response, which is capable of discerning the context of the discussion, the user's mood, and the anticipated response. This enables us to provide customised counselling replies depending on customer feedback. Through this study, two new corpora will be used to train the OpenAI GPT model, which will then be used to generate articles and long sentences. Finally, a comparison study will be carried out. Concurrently, we will use the BERT model to complete the task of context-based intermediate word prediction. Keywords : Artificial Intelligence, Data Science, NLP, Deep Learning, Machine Learning, GPT, Generative AI, Speech Synthesis

Development of Driving Simulation Experiment Protocol for the Study of Drivers’ Emotions by using EEG Signal

The Brain-Computer Interface (BCI) is a field of research that studies the EEG signal in order to elevate our understanding of the human brain. The applications of BCI are not limited to the study of the brain wave but also include its applications. The studies of human emotions specific to the vehicle driver are limited and not vastly explored. The EEG signal is used in this study to classify the emotions of drivers. This research aims to study the emotion classifications (surprise, relax/neutral, focus, fear, and nervousness) while driving the simulated vehicle by analyse the EEG signals. The experiments were conducted in 2 conditions, autonomous and manual drive in the simulated environment. In autonomous driving, vehicle control is disabled. While in manual drive, the subjects are able to control the steering angle, acceleration, and brake pedal. During the experiments, the EEG data of the subjects is recorded and then analyzed.

Mapping aging phenotypes in childhood cancer survivors: A report from the St. Jude Lifetime Cohort.

10069 Background: Childhood cancer survivors experience accelerated aging manifest as physical, cognitive and/or emotional impairments. Aging populations respond to behavioral interventions that address impairment combinations. We aimed to identify and characterize phenotypes (classes) of physical, cognitive and emotional impairments in childhood cancer survivors to inform targets for future intervention. Methods: Factor analysis using St. Jude Lifetime Cohort data (N=4051, 52.2% male, mean [SD] age at diagnosis 8.7 [5.7], assessment 33.9 [10.1] years) generated 9 impairment domains (Table) using 40 items from the Neurocognitive Questionnaire, Brief Symptom Inventory 18 and NHIS Health Status/Limitation of Activities. Latent class analysis estimated impairment patterns and class membership probabilities. Descriptive statistics characterized survivors by class for deficit accumulation index (DAI) score (range 0-1, &gt;0.25 frail), Fried Frailty (FF) score (range 0-5, ≥3 frail), diagnosis and treatment exposures. Logistic regression comparing emotional or cognitive impairment class to the physical impairment class examined associations between survivor characteristics and class membership, adjusting for diagnosis and assessment ages, sex and race/ethnicity. Results: Three classes (phenotypes) differentiated survivors reporting only physical from those reporting both physical and cognitive or emotional impairments. Mean [SD] DAI (cognitive 0.40 [0.10], emotional 0.56 [0.16], physical 0.16 [0.11]) and FF scores (cognitive 1.5 [1.0], emotional 1.8 [1.0], physical 0.9 [0.9]) were highest in the emotional impairment class. Compared to solid tumor survivors (17.1%), brain tumor (33.7%, OR 3.5; 95% CI 2.7-4.4) and leukemia (21.7%, OR 1.5; 95% CI 1.2-1.8) survivors were more likely to be in the cognitive impairment class. Leukemia (OR 1.4; 95% CI 1.1-1.7) and lymphoma (OR 1.4; 95% CI 1.1-1.9) diagnoses increased risk for emotional impairment class membership. Cranial radiation was associated with cognitive (OR 1.8; 95% CI 1.5-2.2) or emotional (OR 1.3; 95% CI 1.1-1.6) impairment class membership. Chest radiation (1.4; 95% CI 1.2-1.7) was associated with emotional impairment class membership. Conclusions: These data provide foundational information for development and testing of targeted behavioral interventions to remediate accelerated aging in survivors. [Table: see text]

Bimodal deep learning model for subjectively enhanced emotion classification in films

This research delves into the concept of color grading in film, focusing on how color influences the emotional response of the audience. The study commenced by recalling state-of-the-art works that process audio–video signals and associated emotions by machine learning. Then, assumptions of subjective tests for refining and validating an emotion model for assigning specific emotional labels to selected film excerpts were presented. The insights gained from these subjective evaluations facilitated the creation of a comprehensive database of movie excerpts. This database was subsequently employed to both train and evaluate the efficacy of deep learning models. The latter half of the study shifts focus to the intelligent analysis of audio and video signals that form film excerpts. This involved exploring diverse methodologies for parameterizing these signals. Models that demonstrated the highest accuracy on the test dataset on audio/video only were amalgamated to forge a bimodal model, which integrates both audio and video signals for emotion classification. The bimodal model exhibited superior accuracy in tests compared to a model that solely relied on video signal classification. This enhancement in performance was achieved with only a marginal increase in the complexity and the number of parameters within the model.

Masked self‐supervised pre‐training model for EEG‐based emotion recognition

AbstractElectroencephalogram (EEG), as a tool capable of objectively recording brain electrical signals during emotional expression, has been extensively utilized. Current technology heavily relies on datasets, with its performance being limited by the size of the dataset and the accuracy of its annotations. At the same time, unsupervised learning and contrastive learning methods largely depend on the feature distribution within datasets, thus requiring training tailored to specific datasets for optimal results. However, the collection of EEG signals is influenced by factors such as equipment, settings, individuals, and experimental procedures, resulting in significant variability. Consequently, the effectiveness of models is heavily dependent on dataset collection efforts conducted under stringent objective conditions. To address these challenges, we introduce a novel approach: employing a self‐supervised pre‐training model, to process data across different datasets. This model is capable of operating effectively across multiple datasets. The model conducts self‐supervised pre‐training without the need for direct access to specific emotion category labels, enabling it to pre‐train and extract universally useful features without predefined downstream tasks. To tackle the issue of semantic expression confusion, we employed a masked prediction model that guides the model to generate richer semantic information through learning bidirectional feature combinations in sequence. Addressing challenges such as significant differences in data distribution, we introduced adaptive clustering techniques that manage by generating pseudo‐labels across multiple categories. The model is capable of enhancing the expression of hidden features in intermediate layers during the self‐supervised training process, enabling it to learn common hidden features across different datasets. This study, by constructing a hybrid dataset and conducting extensive experiments, demonstrated two key findings: (1) our model performs best on multiple evaluation metrics; (2) the model can effectively integrate critical features from different datasets, significantly enhancing the accuracy of emotion recognition.

Enhancing speech emotion recognition with deep learning using multi-feature stacking and data augmentation

This study evaluates the effectiveness of data augmentation on 1D convolutional neural network (CNN) and transformer models for speech emotion recognition (SER) on the Ryerson audio-visual database of emotional speech and song (RAVDESS) dataset. The results show that data augmentation has a positive impact on improving emotion classification accuracy. Techniques such as noising, pitching, stretching, shifting, and speeding are applied to increase data variation and overcome class imbalance. The 1D CNN model with data augmentation achieved 94.5% accuracy, while the transformer model with data augmentation performed even better at 97.5%. This research is expected to contribute better insights for the development of accurate emotion recognition methods by using data augmentation with these models to improve classification accuracy on the RAVDESS dataset. Further research can explore larger and more diverse datasets and alternative model approaches.

Exploration of a method for insight into accessibility design flaws based on touch dynamics

The widespread use of smartphones implies that intelligent systems must address the negative emotions experienced by a diverse user group when encountering usability difficulties. Therefore, exploring the relationship between users’ emotions and operational behaviors is crucial for further enhancing the accessibility of intelligent systems. Touch dynamics, as a method of dynamic touch data monitoring, has already provided many models for emotion recognition, which are beneficial in gaining insight into users’ accessibility needs for smartphones and exploring intelligent solutions to design issues. This paper validates this aspect through interviews and experimental studies. The study first conducted an open-ended interview, and then used a touch behavior experiment with emotional labels and a questionnaire to validate the association between accessibility design flaws in intelligent systems and user emotions. The interview results show that middle-aged and older smartphone users have difficulty activating accessibility tools independently. Consequently, this paper establishes a dataset for emotion classification based on touch gesture data, which is used to gain insight into implicit difficulties in the interaction process through negative emotions. From a touch experiment and questionnaire survey of 57 sample users, there is approximately a 78.57% match between the accessibility needs of intelligent systems and users’ negative emotions, with interaction fluency and interface cognition being the main reasons for this phenomenon. Therefore, touch dynamics, as a non-invasive method of emotion recognition, can use negative emotional feedback to gain insights into user needs that are difficult to detect by other methods. This has application value for enhancing users’ emotional experience and promoting a transformation in interaction design thinking.

Emotion Recognition in Reddit Comments Using Recurrent Neural Networks

Background: Reddit comments are a valuable source of natural language data where emotion plays a key role in human communication. However, emotion recognition is a difficult task that requires understanding the context and sentiment of the texts. In this paper, we aim to compare the effectiveness of four recurrent neural network (RNN) models for classifying the emotions of Reddit comments. Methods: We use a small dataset of 4,922 comments labeled with four emotions: approval, disapproval, love, and annoyance. We also use pre-trained Glove.840B.300d embeddings as the input representation for all models. The models we compare are SimpleRNN, Long ShortTerm Memory (LSTM), bidirectional LSTM, and Gated Recurrent Unit (GRU). We experiment with different text preprocessing steps, such as removing stopwords and applying stemming, removing negation from stopwords, and the effect of setting the embedding layer as trainable on the models. Results: We find that GRU outperforms all other models, achieving an accuracy of 74%. Bidirectional LSTM and LSTM are close behind, while SimpleRNN performs the worst. We observe that the low accuracy is likely due to the presence of sarcasm, irony, and complexity in the texts. We also notice that setting the embedding layer as trainable improves the performance of LSTM but increases the computational cost and training time significantly. We analyze some examples of misclassified texts by GRU and identify the challenges and limitations of the dataset and the models Conclusion: In our study GRU was found to be the best model for emotion classification of Reddit comments among the four RNN models we compared. We also discuss some future directions for research to improve the emotion recognition task on Reddit comments. Furthermore, we provide an extensive discussion of the applications and methods behind each technique in the context of the paper.

Comparing the Performance of Machine Learning Algorithms for Emotion Classification on Tweets

Comparing the Performance of Machine Learning Algorithms for Emotion Classification on Tweets