Emotional Signals Research Articles

Machine Learning Techniques Towards Accurate Emotion Classification from EEG Signals

This article delves into using machine learning algorithms for emotion classification via EEG brain signals. The goal is to discover an accurate model beyond traditional methods, necessitating AI for classifying emotional EEG signals. This study, motivated by the complex link between emotions and neural activity, employs Random Forest, Support Vector Machines, and K-Nearest Neighbors. Notably, Random Forest achieves 99% accuracy, SVM 98%, and KNN 94%. These impressive results, backed by performance metrics like confusion matrices, reveal each model’s effectiveness in emotion classification. The dataset, rich in varied emotional stimuli and EEG placements, provides a robust foundation for detailed analysis. This research underscores significant applications in affective computing and mental health, offering a promising path to understanding the intricate relationship between EEG signals and human emotions.

Posthumanism, Artificial Intelligence and Ishiguro's Klara & the Sun: Can Humanoid Machines attain consciousness?

This paper examines Kazuo Ishiguro's Klara and The Sun through the lens of posthumanism. It uses the textual analysis method to analyze Ishiguro's text as a posthuman novel that depicts the posthuman society where the boundaries between what is human and the nonhuman is blurred. The basic argument is that the aim of Ishiguro's text is two-fold, while it clearly illustrates the inability of the humanoid robot to attain human consciousness, it attempts also to dismantle the anthropocentric view of man. The findings show that Klara, the narrator-protagonist is used as a tool to raise certain questions such as, can humanoids act humanly? And/or can a 'humanoid machine' attain consciousness? More importantly, what it means to be human, in the first place. In doing so, the story attempts to showcase the ruptured boundaries between human and nonhuman and the changing ideas of humankind and its entanglement with the nonhuman world. Further, the interaction between Klara (AF) and other characters in the story is developed in such a way as to illustrate not only the shortcomings of humans regarding faith and affection but, more importantly, the limits of the nonhuman machine. It dismisses the current debate among technology experts that artificial intelligence would soon be able to develop a human-like robot that enjoys similar human emotional signals and reacts exactly like humans. The story simply puts it, despite the defects of humans, nothing can replace humans as those artificial friends (AI) fundamentally lack the kinds of experience that give rise to human-like affect and emotion.

Sources and processes of social influence on health-related choices: A systematic review based on a social-interdependent choice paradigm

BackgroundMost choices in healthcare are not made in social isolation. However, current econometric models treat patients' preferences as the sole determinants of their choices.Through the lens of sociology and medical sociology theories, this paper presents a systematic literature review of identifiable social influences on patients’ choices, serving as a first step in developing a social-interdependent choice paradigm. MethodsFollowing the PRISMA guideline and using nine databases, we identified the individual agents or groups involved in health-related choices, the functional content through which social relationships influence patients, and the choice constructs affected by these processes. From 9036 screened articles, we selected 208 to develop an analytical framework connecting social relationships with choice constructs. ResultsSocial influences predominantly come from family, friends, specialized physicians, and general practitioners. We decomposed the functional content of social relationships into functions and contents. Dyadic interactions and expert knowledge were prominent functions, followed by social control. Prescriptive and informational contents were prevalent, followed by instrumental and emotional ones. Expert knowledge and social norms aligned with prescriptive and informational signals, while dyadic interactions provide emotional and instrumental signals. Reference points for social norms included friends, coworkers, and patients. Social relationships primarily impact which alternatives are evaluated, followed by alternative evaluation strategies and goal selection. Distinctions between medical domains and dimensions emerged, highlighting how the medical area conditions the social influence process. ConclusionThis systematic review presents a comprehensive framework that elucidates the social influence process in healthcare patient decision-making. By detailing the functional content of social relationships into functions and contents and linking these components to the elements of the choice process, we created a structured approach to understanding how social relationships impact patient choices. This will facilitate the systematic integration of social relationships into econometric models of patient choice.

Zero-Shot Image Caption Inference System Based on Pretrained Models

Recently, zero-shot image captioning (ZSIC) has gained significant attention, given its potential to describe unseen objects in images. This is important for real-world applications such as human–computer interaction, intelligent education, and service robots. However, the zero-shot image captioning method based on large-scale pretrained models may generate descriptions containing objects that are not present in the image, which is a phenomenon termed “object hallucination”. This is because large-scale models tend to predict words or phrases with high frequency, as seen in the training phase. Additionally, the method set a limitation to the description length, which often leads to an improper ending. In this paper, a novel approach is proposed to address and reduce the object hallucination and improper ending problem in the ZSIC task. We introduce additional emotion signals as guidance for sentence generation, and we find that proper emotion will filter words that do not appear in the image. Moreover, we propose a novel strategy that gradually extends the number of words in a sentence to confirm the generated sentence is properly completed. Experimental results show that the proposed method achieves the leading performance on unsupervised metrics. More importantly, the subjective examples illustrate the effect of our method in improving hallucination and generating properly ending sentences.

Research on Dual-Emotion Feature Fusion and Performance Improvement in Rumor Detection

At present, a large number of rumors are mixed in with various kinds of news, such as current affairs, politics, social economy, and military activities, which seriously reduces the credibility of Internet information and hinders the positive development of various fields. In previous research on rumors, most scholars have focused their attention on the textual features, contextual semantic features, or single-emotion features of rumors but have not paid attention to the chain reaction caused by the hidden emotions in comments in social groups. Therefore, this paper comprehensively uses the emotional signals in rumor texts and comments to extract emotional features and determines the relationship between them to establish dual-emotion features. The main research achievements include the following aspects: (1) this study verifies that, in the field of affective characteristics, the combination of rumor-text emotion and comment emotion is superior to other baseline affective characteristics, and the detection performance of each component is outstanding; (2) the results prove that the combination of dual-emotion features and a semantic-feature-based detector (BiGRU and CNN) can improve the effectiveness of the detector; (3) this paper proposes reconstructing the dataset according to time series to verify the generalization ability of dual affective features; (4) the attention mechanism is used to combine domain features and semantic features to extract more fine-grained features. A large number of data experiments show that the dual-emotion features can be effectively compatible with an existing rumor detector, enhance the detector’s performance, and improve the detection accuracy.

Distinct neural correlates of accuracy and bias in the perception of facial emotion expressions

ABSTRACT We investigated neural correlates of Emotion Recognition Accuracy (ERA) using the Assessment of Contextualized Emotions (ACE). ACE infuses context by presenting emotion expressions in a naturalistic group setting and distinguishes between accurately perceiving intended emotions (signal), and bias due to perceiving additional, secondary emotions (noise). This social perception process is argued to induce perspective taking in addition to pattern matching in ERA. Thirty participants were presented with an fMRI-compatible adaptation of the ACE consisting of blocks of neutral and emotional faces in single and group-embedded settings. Participants rated the central character’s expressions categorically or using scalar scales in consequent fMRI scans. Distinct brain activations were associated with the perception of emotional vs. neutral faces in the four conditions. Moreover, accuracy and bias scores from the original ACE task performed on another day were associated with brain activation during the scalar (vs. categorical) condition for emotional (vs. neutral) faces embedded in group. These findings suggest distinct cognitive mechanisms linked to each type of emotional rating and highlight the importance of considering cognitive bias in the assessment of social emotion perception.

Altered emotion perception in insomnia, anxiety, depression, mania, psychotic experiences and schizotypal symptoms: a dataset

This data resource provides evidence concerning the prevalence of perceptual alterations of emotional faces amongst individuals experiencing symptoms of insomnia, anxiety, depression, mania, psychotic experiences, and schizotypal tendencies. More specifically, we explored the categorisation accuracy (whether the displayed emotion was correctly identified), misperception (which emotion an incorrect judgment was perceived to be), intensity (extent of the emotion signal strength) and emotional valence (the extent and direction of perceived affect) of six facial expressions of emotion from the Karolinska Directed Emotional Faces database. Complete data from N = 572 respondents are included. The dataset is available to other researchers and is provided on Figshare. Information concerning the data records, usage notes, code availability and technical validation are presented. Finally, we present demographic and correlational data concerning psychiatric symptoms and alterations in the perception of emotional faces.

EEG Based Emotion Analysis using Reinforced Spatio-Temporal Attentive Graph Neural and Context Net Techniques

EEG-based emotion Classification is used to distinguish and observe the mental state or emotions. Emotion classification using EEG is used for medical, security, and various other purposes. There are several deep learning and machine learning techniques that are used to classify EEG emotion signals. They do not provide sufficient accuracy and have higher complexity and a high error rate. In this manuscript, a novel Reinforced Spatio-Temporal Attentive Graph Neural Networks (RSTAGNN)and ContextNet for emotion classification with EEG signals is proposed (RSTAGNN-ContextNet-GWOA-EEG-EA). Here the input EEG signals are taken from two benchmark datasets namely as DEAP and K-EmoCon datasets. Then the input EEG signals are pre-processed and the features are extracted using ContextNet with Global principal component Analysis (GPCA). After that, the EEG signal emotions are classified using Reinforced Spatio-Temporal Attentive Graph Neural Networks (RSTAGNN) method. Then the weight parameters of the RSTAGNN are optimized using Glowworm Swarm Optimization Algorithm (GWOA). The proposed model classifies the EEG signal emotions with high accuracy. The performance of the proposed method using DEAP dataset attains higher accuracy by 24.05%, 12.64%relatedto existing systems, like DWT-SVM-EEG-EA-DEAP and GCNN-LSTM-EEG-EA-DEAP respectively and the performance of the proposed system using K-EmoCon dataset attains higher accuracy 32.64%, 15.65% related to existing systems, like SigRep-EEG-EA-K- EEG Based Emotion Analysis using Reinforced Spatio-Temporal Attentive Graph Neural and Context Net Techniques on Deap and K- EmoCon Article and CAT-EEG-EA-K-EmoCon respectively.

Age-related changes in emotion recognition across childhood: A meta-analytic review.

Children's ability to accurately recognize the external emotional signals produced by those around them represents a milestone in their socioemotional development and is associated with a number of important psychosocial outcomes. A plethora of individual studies have examined when, and in which order, children acquire emotion knowledge over the course of their development. Yet, very few attempts have been made to summarize this body of work quantitatively. To address this, the present meta-analysis examined the age-related trajectories of emotion recognition across childhood and the extent to which typically developing children's recognition of external emotional cues (in the face, voice, and body) is influenced by a host of participant-, task-, and stimulus-related factors. We analyzed children's emotion recognition overall (independent of specific emotion categories) and for specific basic emotions. In total, k = 129 individual studies, investigating a total of N = 31,101 2-12-year-old children's emotion recognition abilities were included in our analyses. Children's recognition accuracy across all emotion categories was significantly above chance and improved with age in the same manner for all emotions. Emotion recognition accuracy was also moderated by region of study and task type. The order in which children became proficient at identifying specific emotions was consistent with previous qualitative reviews: Happiness was the easiest emotion to recognize, and disgust and fear were the most difficult to recognize across age. Task- and stimulus-related moderator variables also influenced specific emotion categories in different ways. We contextualize these results with regard to children's socioemotional development more broadly, and we discuss how our findings can be used to guide researchers and practitioners interested in children's social skills. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

A Methodical Framework Utilizing Transforms and Biomimetic Intelligence-Based Optimization with Machine Learning for Speech Emotion Recognition.

Speech emotion recognition (SER) tasks are conducted to extract emotional features from speech signals. The characteristic parameters are analyzed, and the speech emotional states are judged. At present, SER is an important aspect of artificial psychology and artificial intelligence, as it is widely implemented in many applications in the human-computer interface, medical, and entertainment fields. In this work, six transforms, namely, the synchrosqueezing transform, fractional Stockwell transform (FST), K-sine transform-dependent integrated system (KSTDIS), flexible analytic wavelet transform (FAWT), chirplet transform, and superlet transform, are initially applied to speech emotion signals. Once the transforms are applied and the features are extracted, the essential features are selected using three techniques: the Overlapping Information Feature Selection (OIFS) technique followed by two biomimetic intelligence-based optimization techniques, namely, Harris Hawks Optimization (HHO) and the Chameleon Swarm Algorithm (CSA). The selected features are then classified with the help of ten basic machine learning classifiers, with special emphasis given to the extreme learning machine (ELM) and twin extreme learning machine (TELM) classifiers. An experiment is conducted on four publicly available datasets, namely, EMOVO, RAVDESS, SAVEE, and Berlin Emo-DB. The best results are obtained as follows: the Chirplet + CSA + TELM combination obtains a classification accuracy of 80.63% on the EMOVO dataset, the FAWT + HHO + TELM combination obtains a classification accuracy of 85.76% on the RAVDESS dataset, the Chirplet + OIFS + TELM combination obtains a classification accuracy of 83.94% on the SAVEE dataset, and, finally, the KSTDIS + CSA + TELM combination obtains a classification accuracy of 89.77% on the Berlin Emo-DB dataset.

Horses can learn to identify joy and sadness against other basic emotions from human facial expressions

Recently, horses and other domestic mammals have been shown to perceive and react to human emotional signals, with most studies focusing on joy and anger. In this study, we tested whether horses can learn to identify human joyful and sad expressions against other emotions. We used a touchscreen-based automated device that presented pairs of human portraits and distributed pellets when the horse touched the rewarded face. Six horses were trained to touch the sad face and 5 the joyful face. By the end of training, horses’ performances at the group level were significantly higher than chance level, with higher scores for horses trained with the sad face. At the individual level, evidence of task learning varied among horses, which could be explained by individual variations in horses’ ability to identify different human facial expressions or attention issues during the tests. In a generalization test, we introduced portraits of different humans than those presented during training. Horses trained with the joyful face performed better than chance, demonstrating generalization. Conversely, horses trained with the sad face did not. Horses also showed differences in learning performance according to the non-rewarded emotion, providing insights into horses’ cognitive processing of facial expressions.

Self-Supervised EEG Representation Learning for Robust Emotion Recognition

Emotion recognition based on electroencephalography (EEG) is becoming a growing concern of researchers due to its various applications and portable devices. Existing methods are mainly dedicated to EEG feature representation and have made impressive progress. However, the problem of scarce labels restricts their further promotion. In light of this, we propose a self-supervised framework with contrastive learning for robust EEG-based emotion recognition, which can effectively leverage both readily available unlabeled EEG signals and labeled ones to learn highly discriminative EEG features. Firstly, we construct a specific pretext task according to the sequential non-stationarity of emotional EEG signals for contrastive learning, which aims at extracting pseudo-label information from all EEG data. Meanwhile, we propose a novel negative segment selection algorithm to reduce the noise of unlabeled data during the contrastive learning process. Secondly, to mitigate the overfitting issue induced by a small number of labeled samples during learning, we originate a loss function with label smoothing regularization that can guide the model to learn generalizable features. Extensive experiments over three benchmark datasets demonstrate the effectiveness and superiority of our model on EEG-based emotion recognition task. Besides, the generalization and robustness of the model have also been proved through sufficient experiments.

The recognition and the initial reaction to children's signals after experiencing maltreatment during childhood

AbstractMaltreatment during childhood has many long‐lasting effects. Victims often become parents who experience parenting difficulties, therefore contributing to the intergenerational cycle of maltreatment. This paper highlights the consequences of severe trauma during childhood on two processes critical for a sensitive response, namely the recognition of and the initial reaction to emotional signals. We argue that most interventions focus on the observed parental response to children's behaviors or on parental interpretation of children's signals, that is, the attribution of meaning or intent to the signal. Interventions should better integrate research indicating that a history of childhood maltreatment could impair the capacity to correctly identify the emotion and the immediate reaction to these signals. By failing to acknowledge these long‐lasting consequences of childhood maltreatment, scholars and practitioners might never achieve true success in intervening with families in the hope of breaking the intergenerational cycle of maltreatment.

Optimizing Hyperparameters of CNN and DNN for Emotion Classification Based on EEG Signals

EEG emotion is a research topic that has gained significant attention in the development of emotion classification systems. This study focuses on optimizing the hyperparameters of CNN (Convolutional Neural Network) and DNN (Deep Neural Network) for classifying EEG emotion signals. The data is divided into three train-test data ratio scenarios: 80:20, 70:30, and 60:40. After modeling and the classification process, hyperparameter tuning was conducted on both models to achieve the best results. Experimental results showed the highest accuracy of 98.36% for CNN, while DNN reached 98.18% in the 80:20 data ratio scenario. Despite the high accuracy, the differences in the loss curves between CNN and DNN reflect the complexity of the performance of both models. The train-test data ratio was also found to significantly impact the performance of both models, with the 80:20 data split yielding the best results, while the 70:30 and 60:40 splits resulted in slightly lower accuracies.

Emotion classification using electrocardiogram and machine learning: A study on the effect of windowing techniques

Automated emotion recognition using physiological signals has gained significant attention in recent years due to its potential applications in human–computer interaction, healthcare, and psychology. Electrocardiogram (ECG) signals are widely used due to their non-invasiveness, high temporal resolution, and direct relationship with the autonomic nervous system. In this study, we propose a novel approach for ECG-based emotion recognition using time-series to image encoding techniques and texture-based features combined with machine learning algorithms and deep learning architectures. The ECG data used in this study were obtained from the Continuously Annotated Signals of Emotion (CASE) and Wearable Stress and Affect Detection (WESAD) datasets. We categorized emotional states based on valence and arousal annotations into four classes: High-Valence High-Arousal, High-Valence Low-Arousal, Low-Valence High-Arousal, and Low-Valence Low-Arousal. The ECGs were segmented into 5 and 7-window segments and transformed into 2D representations using Gramian Angular Summation Field, Markov Transition Field, Recurrence Plot (RP), and the triple-channel fusion of all these images. Total of 85 textural features based on the Gray-Level Co-occurrence Matrix (GLCM), Gray-Level Run Length Matrix, Zernike’s Moments, Hu’s Moments, Fractal Dimension Texture Analysis (FDTA), and First-Order Statistics were extracted. Classifiers, namely Random Forest, Support Vector Machine (SVM), eXtreme Gradient Boosting (XGB), 1D Convolutional Neural Network (CNN), and Multi-head Attention Network were considered to classify the emotional states. The performance of the classifiers varied depending on the time-series to image encoding technique, segmentation approaches, and the classifier employed. We achieved the highest Weighted F-measure (F-m) of 94.91% (RP + XGB) and 86.78% (RP + SVM) using the 7-window and 5-window approaches, respectively. Our proposed 1D CNN architecture achieved the highest classification metrics (F-m = 92.52%, Balanced accuracy = 92.0%, Recall = 91.96%, and Precision = 93.16%) with RP images in a 7-window approach. GLCM and FDTA features made significant contributions to the classification of emotional states. Overall, our results suggest that the proposed method holds promise for developing more accurate and efficient emotion recognition systems.

Multimodal Content Analysis Using Deep Learning

Abstract: The multimodal content analysis platform combines sentiment analysis and neural style transfertechniques to process and improve various types of digital content. The sentiment analysis module utilizes natural language processing (NLP) algorithms, such as recurrent neural networks (RNNs) or transformer models like BERT, to extract emotional signals from textual, visual, and auditory inputs. Signals are classified into predefined sentiment categories, providing granular insights into the emotional context of the content. The platform employs neural style transfer algorithms, such as style transfer networks (NSTNs) or generative adversarial networks (GANs), to transfer stylistic attributes between texts. By training on a diverse range of artistic styles, the system learns to apply these styles to input text while preserving semantic meaning. This process enhances the visual representation of textual content, making it more appealing and engaging to users.

Variation in social referencing behavior according to infants’ understanding of indirect emotional signals

Variation in social referencing behavior according to infants’ understanding of indirect emotional signals

Zooming into the Complex Dynamics of Electrodermal Activity Recorded during Emotional Stimuli: A Multiscale Approach.

Physiological phenomena exhibit complex behaviours arising at multiple time scales. To investigate them, techniques derived from chaos theory were applied to physiological signals, providing promising results in distinguishing between healthy and pathological states. Fractal-like properties of electrodermal activity (EDA), a well-validated tool for monitoring the autonomic nervous system state, have been reported in previous literature. This study proposes the multiscale complexity index of electrodermal activity (MComEDA) to discern different autonomic responses based on EDA signals. This method builds upon our previously proposed algorithm, ComEDA, and it is empowered with a coarse-graining procedure to provide a view at multiple time scales of the EDA response. We tested MComEDA's performance on the EDA signals of two publicly available datasets, i.e., the Continuously Annotated Signals of Emotion (CASE) dataset and the Affect, Personality and Mood Research on Individuals and Groups (AMIGOS) dataset, both containing physiological data recorded from healthy participants during the view of ultra-short emotional video clips. Our results highlighted that the values of MComEDA were significantly different (p-value < 0.05 after Wilcoxon signed rank test with Bonferroni's correction) when comparing high- and low-arousal stimuli. Furthermore, MComEDA outperformed the single-scale approach in discriminating among different valence levels of high-arousal stimuli, e.g., showing significantly different values for scary and amusing stimuli (p-value = 0.024). These findings suggest that a multiscale approach to the nonlinear analysis of EDA signals can improve the information gathered on task-specific autonomic response, even when ultra-short time series are considered.

The role of pre-supplementary motor cortex in action control with emotional stimuli: Arepetitive transcranial magnetic stimulationstudy.

Swiftly halting ongoing motor actions is essential to react to unforeseen and potentially perilous circumstances. However, the neural bases subtending the complex interplay between emotions and motor control have been scarcely investigated. Here, we used an emotional stop signal task (SST) to investigate whether specific neural circuits engaged by action suppression are differently modulated by emotional signals with respect to neutral ones. Participants performed an SST before and after the administration of one session of repetitive transcranial magnetic stimulation (rTMS) over the pre-supplementary motor cortex (pre-SMA), the right inferior frontal gyrus (rIFG), and the left primary motor cortex (lM1). Results show that rTMS over the pre-SMA improved the ability to inhibit prepotent action (i.e., better action control) when emotional stimuli were presented. In contrast, action control in a neutral context was fostered by rTMS over the rIFG. No changes were observed after lM1 stimulation. Intriguingly, individuals with higher impulsivity traits exhibited enhanced motor control when facing neutral stimuli following rIFG stimulation. These results further our understanding of the interplay between emotions and motor functions, shedding light on the selective modulation of neural pathways underpinning these processes.

Addressing subjectivity in paralinguistic data labeling for improved classification performance: A case study with Spanish-speaking Mexican children using data balancing and semi-supervised learning

Paralinguistics is an essential component of verbal communication, comprising elements that provide additional information to the language, such as emotional signals. However, the subjective nature of perceiving affective aspects, such as emotions, poses a significant challenge to the development of quality resources for training recognition models of paralinguistic features. Labelers may have different opinions and perceive different emotions from others, making it difficult to achieve a diverse and sufficient representation of considered categories. In this study, we focused on the automatic classification of paralinguistic aspects in Spanish-speaking Mexican children of elementary school age. However, the dataset presents a strong imbalance in all labeled aspects and a low agreement between the labelers. Furthermore, the audio samples were too short, making it challenging to accurately classify affective speech. To address these challenges, we propose a novel method that combines data balancing algorithms and semisupervised learning to improve the classification performance of the trained models. Our method aims to mitigate the subjectivity involved in labeling paralinguistic data, thus advancing the development of robust and accurate recognition models of affective aspects in speech.