Detection Of Emotional States Research Articles

A new emotion detection algorithm using extracted features of the different time-series generated from ST intervals Poincaré map

Emotion recognition using biological signals plays an essential role in studying psychological states. In many studies, the distinct effect of each physiological signal usually is ignored. By limiting the physiological signals and just evaluating the ECG signals, the paper aims to study the dynamical behavior of the Poincaré map in the form of the feature extraction from the generated time-series. For this purpose, the detection of emotional states in the two-dimensional model of emotion (Arousal-Valence) is utilized using the electrocardiogram signals recorded on the MAHNOB-HCI tagging database. So after signal processing, the waves of Q, R, S, and T are detected by processing the ECG signals using the Pan-Tompkins algorithm. The Poincaré map is adopted for the RR, QT, and ST intervals, then five different time-series are extracted from this mapping, subsequently, various features are extracted from these different time-series in the time domain, frequency domain, time-frequency domain, and nonlinear domain analysis. Finally, the classification of emotional states is performed using three classifiers: KNN, SVM, and MLP. In this study, the extracted optimal features of the different time-series generated from Poincaré map of the ST Intervals are achieved the best average accuracies of 82.17 % ± 4.73 and 78.07 % ± 3.59 in the arousal and valence model, respectively. The superiority of the obtained results from the extracted features of the five different time-series generated from ST-Intervals Poincaré map expresses that in comparison with RR and QT Intervals, the ST Intervals are more affected by ANS response to the emotional stimuli.

Detection and Classification of Emotional State Based on Speech Signal

In this research, an algorithm was proposed to automatically classify the mood of the speaker by referring to his speech. Three moods were adopted in this study, namely joy, sadness and anger in order to distinguish between them. The principle of the algorithm work includes the initial treatment of the signal of by removing the silence and then cut the signal to a number of sections length of each 512 sample, and then treatment by window (Hamming window) followed by the process of extracting the characteristics such as energy, the basic frequency, resonance frequencies of each section and for all speech signals that Were recorded, which included 30 signals of persons between 15 and 25 years of age in order to prepare the database for the three moods and to draw the characteristic curves and for each mood. The selection of signals was done from training and testing set for detectingthe mood of these signals by performing the previous steps and then comparing the resulting curve with the previous curves using the correlation coefficient and the Euclidean distance. The algorithm gave good results when these characteristics were adopted in the classification process and by about 75%.

Simultaneous Prediction of Valence/Arousal and Emotions on AffectNet, Aff-Wild and AFEW-VA

Abstract We address the problem of emotional state detection from facial expressions. Our proposed approach simultaneously detects faces and predicts both discrete emotion categories and continuous valence/arousal values from raw input images. We train and evaluate our approach on 3 different datasets, compare our approach to other state-of-the-art approaches and perform a cross-database evaluation. In this way, we found, that our approach generalizes well and is suitable for real-time applications.

Emotional state detection based on common spatial patterns of EEG

The application of EEG-based emotional states is one of the most vital phases in the context of neural response decoding. Emotional response mostly appears in the presence of visual, auditory, tactile, and gustatory arousals. In our work, we use visual stimuli to evaluate the emotional feedback. One of the best performing methods in emotion estimation applications is the common spatial patterns (CSP). We implement CSP method in addition to the conventional Welch power spectral density-based analysis. Experimental results and topographies on the collected EEG data show that the CSP spatial filtering method implies the relationship between EEG bands, EEG channels, neural efficiency and emotional stimuli types.

Effective EEG Channels for Emotion Identification over the Brain Regions using Differential Evolution Algorithm.

The motivation of this study was to detect the most effective electroencephalogram (EEG) channels for various emotional states of the brain regions (i.e. frontal, temporal, parietal and occipital). The EEGs of ten volunteer participants without health conditions were captured while the participants were shown seven, short, emotional video clips with audio (i.e. anger, anxiety, disgust, happiness, sadness, surprise and neutral). The Savitzky-Golay (SG) filter was adopted for smoothing and denoising the EEG dataset. The spectral features were performed by employing the relative spectral powers of delta (δRP), theta (θRP), alpha (αRP), beta (βRP), and gamma (γRP). The differential evolution-based channel selection algorithm (DEFS_Ch) was computed to find the most suitable EEG channels that have the greatest efficacy for identifying the various emotional states of the brain regions. The results revealed that all seven emotions previously mentioned were represented by at least two frontal and two temporal channels. Moreover, some emotional states could be identified by channels from the parietal region such as disgust, happiness and sadness. Furthermore, the right and left occipital channels may help in identifying happiness, sadness, surprise and neutral emotional states. The DEFS_Ch algorithm raised the linear discriminant analysis (LDA) classification accuracy from 80% to 86.85%, indicating that DEFS_Ch may offer a useful way for reliable enhancement of the detection of different emotional states of the brain regions.

Emotional Stress State Detection Using Genetic Algorithm-Based Feature Selection on EEG Signals.

In recent years, stress analysis by using electro-encephalography (EEG) signals incorporating machine learning techniques has emerged as an important area of research. EEG signals are one of the most important means of indirectly measuring the state of the brain. The existing stress algorithms lack efficient feature selection techniques to improve the performance of a subsequent classifier. In this paper, genetic algorithm (GA)-based feature selection and k-nearest neighbor (k-NN) classifier are used to identify stress in human beings by analyzing electro-encephalography (EEG) signals. GA is incorporated in the stress analysis pipeline to effectively select subset of features that are suitable to enhance the performance of the k-NN classifier. The performance of the proposed method is evaluated using the Database for Emotion Analysis using Physiological Signals (DEAP), which is a public EEG dataset. A feature set is extracted in 32 EEG channels, which consists of statistical features, Hjorth parameters, band power, and frontal alpha asymmetry. The selected features through GA are used as input to the k-NN classifier to distinguish whether each EEG datapoint represents a stress state. To further consolidate, the effectiveness of the proposed method is compared with that of a state-of-the-art principle component analysis (PCA) method. Experimental results show that the proposed GA-based method outperforms PCA, with GA demonstrating 71.76% classification accuracy compared with 65.3% for PCA. Thus, it can be concluded that the proposed method can be effectively used for stress analysis with high classification accuracy.

Remote Detection of Human Emotional States by Facial Areas

Remote Detection of Human Emotional States by Facial Areas

Measuring and monitoring emotional changes in children who stutter

The assessment of clients with speech disorders presents challenges for speech-language pathologists. For example, having a reliable way of measuring the severity of the case, determining which remedial program is aligned with a patient's needs, and measuring of treatment processes. There is potential for brain-computer interface (BCI) applications to enhance speech therapy sessions by providing objective insights and real-time visualization of brain activity during the sessions. This paper presents a study on emotional state detection during speech pathology. The goal of this study is to investigate affective–motivational brain responses to stimuli in children who stutter. To this end, we conducted an experiment that involved recording frontal electroencephalography (EEG) activity from fifteen children with stuttering whilst they looked at visual stimuli. The contribution of our study is to provide a comprehensive background and a framework for emotional state detection experiments as assessment and monitoring tool in speech pathology. It mainly discusses the feasibility and potential benefits of applying EEG-based emotion detection in speech-language therapy contexts of use. The findings of our research indicate that emotional recognition using non-invasive EEG-based BCI system is sufficient to differentiate between affective states of individuals in treatment contexts.

Investigation of the effect of spectrogram images and different texture analysis methods on speech emotion recognition

Emotional state detection is an important part of human-machine interaction studies. The features used in emotion recognition are derived from the changes in facial mimics and speech signals. In emotion recognition from facial expressions, facial expressions are processed by image processing methods. If emotion recognition is performed via speech, speech is digitized by signal processing methods, and various features of speech are obtained by acoustic analysis. However, since the change in the features obtained by acoustic analysis is different according to emotion, the general success rate is changing. To overcome this limitation, the study of the effect of spectrogram images on emotional recognition is a current field of study. The purpose of this study is to investigate the effects of texture analysis methods and spectrogram images on speech emotion recognition. For this purpose, spectrogram images of speech were processed by four different texture analysis methods to obtain feature sets. The success rates for the emotion recognition of the obtained feature sets were experimentally investigated using support vector machines. In addition, the success of texture analysis methods was compared with acoustic analysis methods. The results have shown that texture analysis methods can be used for speech emotion recognition. When the results of the texture analysis methods were compared with those of the acoustic analysis, the texture analysis methods resulted in a 0.4% reduction in emotion recognition success rate. However, the combined use of both methods increased the success rate.

Towards Aiding Decision-Making in Social Networks by Using Sentiment and Stress Combined Analysis

The present work is a study of the detection of negative emotional states that people have using social network sites (SNSs), and the effect that this negative state has on the repercussions of posted messages. We aim to discover in which grade a user having an affective state considered negative by an Analyzer can affect other users and generate bad repercussions. Those Analyzers that we propose are a Sentiment Analyzer, a Stress Analyzer and a novel combined Analyzer. We also want to discover what Analyzer is more suitable to predict a bad future situation, and in what context. We designed a Multi-Agent System (MAS) that uses different Analyzers to protect or advise users. This MAS uses the trained and tested Analyzers to predict future bad situations in social media, which could be triggered by the actions of a user that has an emotional state considered negative. We conducted an experimentation with different datasets of text messages from Twitter.com to examine the ability of the system to predict bad repercussions, by comparing the polarity, stress level or combined value classification of the messages that are replies to the ones of the messages that originated them.

Convolutional Neural Network-Based Classification of Driver's Emotion during Aggressive and Smooth Driving Using Multi-Modal Camera Sensors.

Because aggressive driving often causes large-scale loss of life and property, techniques for advance detection of adverse driver emotional states have become important for the prevention of aggressive driving behaviors. Previous studies have primarily focused on systems for detecting aggressive driver emotion via smart-phone accelerometers and gyro-sensors, or they focused on methods of detecting physiological signals using electroencephalography (EEG) or electrocardiogram (ECG) sensors. Because EEG and ECG sensors cause discomfort to drivers and can be detached from the driver’s body, it becomes difficult to focus on bio-signals to determine their emotional state. Gyro-sensors and accelerometers depend on the performance of GPS receivers and cannot be used in areas where GPS signals are blocked. Moreover, if driving on a mountain road with many quick turns, a driver’s emotional state can easily be misrecognized as that of an aggressive driver. To resolve these problems, we propose a convolutional neural network (CNN)-based method of detecting emotion to identify aggressive driving using input images of the driver’s face, obtained using near-infrared (NIR) light and thermal camera sensors. In this research, we conducted an experiment using our own database, which provides a high classification accuracy for detecting driver emotion leading to either aggressive or smooth (i.e., relaxed) driving. Our proposed method demonstrates better performance than existing methods.

Multi-Sensor Based State Prediction for Personal Mobility Vehicles.

This paper presents a study on multi-modal human emotional state detection while riding a powered wheelchair (PMV; Personal Mobility Vehicle) in an indoor labyrinth-like environment. The study reports findings on the habituation of human stress response during self-driving. In addition, the effects of “loss of controllability”, change in the role of the driver to a passenger, are investigated via an autonomous driving modality. The multi-modal emotional state detector sensing framework consists of four sensing devices: electroencephalograph (EEG), heart inter-beat interval (IBI), galvanic skin response (GSR) and stressor level lever (in the case of autonomous riding). Physiological emotional state measurement characteristics are organized by time-scale, in terms of capturing slower changes (long-term) and quicker changes from moment-to-moment. Experimental results with fifteen participants regarding subjective emotional state reports and commercial software measurements validated the proposed emotional state detector. Short-term GSR and heart signal characterizations captured moment-to-moment emotional state during autonomous riding (Spearman correlation; ρ = 0.6, p < 0.001). Short-term GSR and EEG characterizations reliably captured moment-to-moment emotional state during self-driving (Classification accuracy; 69.7). Finally, long-term GSR and heart characterizations were confirmed to reliably capture slow changes during autonomous riding and also of emotional state during participant resting state. The purpose of this study and the exploration of various algorithms and sensors in a structured framework is to provide a comprehensive background for multi-modal emotional state prediction experiments and/or applications. Additional discussion regarding the feasibility and utility of the possibilities of these concepts are given.

Webcam-based detection of emotional states

Game designers have to deal with the complex task of monitoring the emotional state of players in games. There are different elements with the game, which effect the player's emotional status. Since the game play experience occurs almost unconsciously, traditional methods such as think aloud may disrupt the playing experience, thus skewing the results obtained. Other methods include fitting cables and electrodes to the player to monitor biological information. Although such devices can offer significant accurate results, they are not commonly found and may cause discomfort while playing games. Because of this, we propose a webcam-based heart rate monitoring method that can be used to predict the player's emotional state. We first analyzed the change in heart rate with respect to the players emotional state. This allowed us to find a correlation between emotional states, such as frustration, fun, challenge, and boredom. The second objective was to create a webcam-based method to monitor the heart rate. This was performed by extracting the RGB channels from the face region and then retrieving the underlying components using a dimensionality-reduction method. The results obtained from the webcam-based method were far from perfect, but this was expected, since we were performing the tests under realistic conditions. The last objective was to predict the player's emotional state using the heart rate obtained from the webcam-based method. The accuracy of the prediction was up to 76 %, which exceeded our initial aim. This system will be implemented in Unity 3D to make its integration and adoption easier.

Automatic Emotional State Detection using Facial Expression Dynamic in Videos

In this paper, an automatic emotion detection system is built for a computer or machine to detect the emotional state from facial expressions in human computer communication. Firstly, dynamic motion features are extracted from facial expression videos and then advanced machine learning methods for classification and regression are used to predict the emotional states. The system is evaluated on two publicly available datasets, i.e. GEMEP_FERA and AVEC2013, and satisfied performances are achieved in comparison with the baseline results provided. With this emotional state detection capability, a machine can read the facial expression of its user automatically. This technique can be integrated into applications such as smart robots, interactive games and smart surveillance systems.

Real Time Fear Detection Using Wearable Single Channel Electroencephalogram

Real time detection of emotional state has multiple applications for security, safety and identification of dangerous situations. Traditionally electroencephalogram (EEG) based emotion studies are conducted in controlled lab environment with multi-channel systems and large signal processing power. In order to be useful in real world situation the system for emotion detection has to be miniature, portable and working in real time supported by calculations that can be provided by a processor power of a mobile phone. Here we present our results on real time fear detection using portable single electrode EEG system conducted on 10 subjects. We studied possibility of translation of the markers previously identified for complex multi-electrode system - ratios of slow waves to fast waves into real time portable system. It was demonstrated using Student’s t-test that the average value of the monitored parameter during normal state was significantly higher than that of during a scary stimulus with a P value of (0.027) ~ 0.03 for Theta/ Beta. The framework for portable fear detection together with the markers discussed in this study can enable many applications important for the soldiers in the battlefield or police officers while being under attack as an indicator that help is urgently needed.

SentiHealth-Cancer: A sentiment analysis tool to help detecting mood of patients in online social networks

BackgroundCancer is a critical disease that affects millions of people and families around the world. In 2012 about 14.1 million new cases of cancer occurred globally. Because of many reasons like the severity of some cases, the side effects of some treatments and death of other patients, cancer patients tend to be affected by serious emotional disorders, like depression, for instance. Thus, monitoring the mood of the patients is an important part of their treatment. Many cancer patients are users of online social networks and many of them take part in cancer virtual communities where they exchange messages commenting about their treatment or giving support to other patients in the community. Most of these communities are of public access and thus are useful sources of information about the mood of patients. Based on that, Sentiment Analysis methods can be useful to automatically detect positive or negative mood of cancer patients by analyzing their messages in these online communities. ObjectiveThe objective of this work is to present a Sentiment Analysis tool, named SentiHealth-Cancer (SHC-pt), that improves the detection of emotional state of patients in Brazilian online cancer communities, by inspecting their posts written in Portuguese language. The SHC-pt is a sentiment analysis tool which is tailored specifically to detect positive, negative or neutral messages of patients in online communities of cancer patients. We conducted a comparative study of the proposed method with a set of general-purpose sentiment analysis tools adapted to this context. MethodsDifferent collections of posts were obtained from two cancer communities in Facebook. Additionally, the posts were analyzed by sentiment analysis tools that support the Portuguese language (Semantria and SentiStrength) and by the tool SHC-pt, developed based on the method proposed in this paper called SentiHealth. Moreover, as a second alternative to analyze the texts in Portuguese, the collected texts were automatically translated into English, and submitted to sentiment analysis tools that do not support the Portuguese language (AlchemyAPI and Textalytics) and also to Semantria and SentiStrength, using the English option of these tools. Six experiments were conducted with some variations and different origins of the collected posts. The results were measured using the following metrics: precision, recall, F1-measure and accuracy ResultsThe proposed tool SHC-pt reached the best averages for accuracy and F1-measure (harmonic mean between recall and precision) in the three sentiment classes addressed (positive, negative and neutral) in all experimental settings. Moreover, the worst accuracy value (58%) achieved by SHC-pt in any experiment is 11.53% better than the greatest accuracy (52%) presented by other addressed tools. Finally, the worst average F1 (48.46%) reached by SHC-pt in any experiment is 4.14% better than the greatest average F1 (46.53%) achieved by other addressed tools. Thus, even when we compare the SHC-pt results in complex scenario versus others in easier scenario the SHC-pt is better. ConclusionsThis paper presents two contributions. First, it proposes the method SentiHealth to detect the mood of cancer patients that are also users of communities of patients in online social networks. Second, it presents an instantiated tool from the method, called SentiHealth-Cancer (SHC-pt), dedicated to automatically analyze posts in communities of cancer patients, based on SentiHealth. This context-tailored tool outperformed other general-purpose sentiment analysis tools at least in the cancer context. This suggests that the SentiHealth method could be instantiated as other disease-based tools during future works, for instance SentiHealth-HIV, SentiHealth-Stroke and SentiHealth-Sclerosis.

Machine learning classification of design team members' body language patterns for real time emotional state detection

Design team interactions are one of the least understood aspects of the engineering design process. Given the integral role that designers play in the engineering design process, understanding the emotional states of individual design team members will help us quantify interpersonal interactions and how those interactions affect resulting design solutions. The methodology presented in this paper enables automated detection of individual team member's emotional states using non-wearable sensors. The methodology uses the link between body language and emotions to detect emotional states with accuracies above 98%. A case study involving human participants, enacting eight body language poses relevant to design teams, is used to illustrate the effectiveness of the methodology. This will enable researchers to further understand design team interactions.

Experimental Approach on Affective Aware Systems for Disabled Users

In this paper, we present an experimental approach to design systems sensitive to user's emotions. We describe a system for the detection of emotional states based on physiological signals and an application use case utilizing the detected emotional state. The application is an emotion management system to be used for the support in the improvement of life conditions of users suffering from cerebral palsy (CP). The system presented here combines effectively biofeedback sensors and a set of software algorithms to detect the current emotional state of the user.

42. Detection of emotional states based on scalp and source maps of EEG

Recent methods of emotional states detection are mostly based on low-density EEG recording and analysis of the scalp activity. This disadvantage should result in the low classification accuracy of emotions. We extend the research in this area to high-density EEG recordings (118 electrodes) and source localization methods. Our experiment consisted of 100 affective pictures (mostly IAPS dataset) standing for three different emotional categories (positive, neutral and negative). They were presented to 30 participants and the EEG signal was recorded. The spectral bands were calculated for scalp projection and further ICA decomposition and dipole fitting was applied for source localization. We analyzed classification errors for specific emotional categories. The goal of our study was to identify differences between low-density and high-density density EEG recordings. We were interested in the difference between scalp maps and the localized sources of the EEG signal. These differences should clarify possible sources of classification errors. We also analyzed whether differences in the spectral activity of specific emotions on the scalp correlates with the source signals. The advantages and disadvantages of specific methods are discussed.

Automatic Emotional State Detection using Facial Expression Dynamic in Videos

In this paper, an automatic emotion detection system is built for a computer or machine to detect the emotional state from facial expressions in human computer communication. Firstly, dynamic motion features are extracted from facial expression videos and then advanced machine learning methods for classification and regression are used to predict the emotional states. The system is evaluated on two publicly available datasets, i.e. GEMEP_FERA and AVEC2013, and satisfied performances are achieved in comparison with the baseline results provided. With this emotional state detection capability, a machine can read the facial expression of its user automatically. This technique can be integrated into applications such as smart robots, interactive games and smart surveillance systems.