Classification Of Human Emotions Research Articles

High-Accuracy Classification of Multiple Distinct Human Emotions Using EEG Differential Entropy Features and ResNet18

The high-accuracy detection of multiple distinct human emotions is crucial for advancing affective computing, mental health diagnostics, and human–computer interaction. The integration of deep learning networks with entropy measures holds significant potential in neuroscience and medicine, especially for analyzing EEG-based emotion states. This study proposes a method combining ResNet18 with differential entropy to identify five types of human emotions (happiness, sadness, fear, disgust, and neutral) from EEG signals. Our approach first calculates the differential entropy of EEG signals to capture the complexity and variability of the emotional states. Then, the ResNet18 network is employed to learn feature representations from the differential entropy measures, which effectively captures the intricate spatiotemporal dynamics inherent in emotional EEG patterns using residual connections. To validate the efficacy of our method, we conducted experiments on the SEED-V dataset, achieving an average accuracy of 95.61%. Our findings demonstrate that the combination of ResNet18 with differential entropy is highly effective in classifying multiple distinct human emotions from EEG signals. This method shows robust generalization and broad applicability, indicating its potential for extension to various pattern recognition tasks across different domains.

Exploring Human Emotions: A Virtual Reality-Based Experimental Approach Integrating Physiological and Facial Analysis

This paper researches the classification of human emotions in a virtual reality (VR) context by analysing psychophysiological signals and facial expressions. Key objectives include exploring emotion categorisation models, identifying critical human signals for assessing emotions, and evaluating the accuracy of these signals in VR environments. A systematic literature review was performed through peer-reviewed articles, forming the basis for our methodologies. The integration of various emotion classifiers employs a ‘late fusion’ technique due to varying accuracies among classifiers. Notably, facial expression analysis faces challenges from VR equipment occluding crucial facial regions like the eyes, which significantly impacts emotion recognition accuracy. A weighted averaging system prioritises the psychophysiological classifier over the facial recognition classifiers due to its higher accuracy. Findings suggest that while combined techniques are promising, they struggle with mixed emotional states as well as with fear and trust emotions. The research underscores the potential and limitations of current technologies, recommending enhanced algorithms for effective interpretation of complex emotional expressions in VR. The study provides a groundwork for future advancements, aiming to refine emotion recognition systems through systematic data collection and algorithm optimisation.

FetchEEG: a hybrid approach combining feature extraction and temporal-channel joint attention for EEG-based emotion classification

Objective. Electroencephalogram (EEG) analysis has always been an important tool in neural engineering, and the recognition and classification of human emotions are one of the important tasks in neural engineering. EEG data, obtained from electrodes placed on the scalp, represent a valuable resource of information for brain activity analysis and emotion recognition. Feature extraction methods have shown promising results, but recent trends have shifted toward end-to-end methods based on deep learning. However, these approaches often overlook channel representations, and their complex structures pose certain challenges to model fitting. Approach. To address these challenges, this paper proposes a hybrid approach named FetchEEG that combines feature extraction and temporal-channel joint attention. Leveraging the advantages of both traditional feature extraction and deep learning, the FetchEEG adopts a multi-head self-attention mechanism to extract representations between different time moments and channels simultaneously. The joint representations are then concatenated and classified using fully-connected layers for emotion recognition. The performance of the FetchEEG is verified by comparison experiments on a self-developed dataset and two public datasets. Main results. In both subject-dependent and subject-independent experiments, the FetchEEG demonstrates better performance and stronger generalization ability than the state-of-the-art methods on all datasets. Moreover, the performance of the FetchEEG is analyzed for different sliding window sizes and overlap rates in the feature extraction module. The sensitivity of emotion recognition is investigated for three- and five-frequency-band scenarios. Significance. FetchEEG is a novel hybrid method based on EEG for emotion classification, which combines EEG feature extraction with Transformer neural networks. It has achieved state-of-the-art performance on both self-developed datasets and multiple public datasets, with significantly higher training efficiency compared to end-to-end methods, demonstrating its effectiveness and feasibility.

Electroencephalogram based human emotion classification for valence and arousal using machine learning approach

<span>Humans have unique ability to express emotions and electroencephalogram (EEG) signals are one of the sought-after ways to analyze a person’s emotional state. However, extracting proper emotion related features from EEG and finding corresponding emotion is challenging because of complex nature of emotions and underlying brain activities. The objective of this paper is to address this issue for more accurate emotion classification based on EEG. It also compares feature extraction methods namely fast fourier transform (FFT) and discrete wavelet transform (DWT). DEAP dataset is used for classification of human emotions through support vector machine (SVM) and K-nearest neighbor (KNN) algorithms by considering features such as standard deviation, mean, variance, power spectrum density (PSD) for FFT; and energy, entropy for DWT. It is observed that feature extraction from FFT yielded better results than DWT and KNN gave more accuracy of 96.61% for valence and 96.42% for arousal as compared to SVM. The proposed method based on PSD and FFT fared better than other existing ones in terms of accuracy when compared against different features and feature extraction techniques. This approach is expected to help researchers to understand feature extraction from EEG signals and decide proper features and techniques for better implementation.</span>

Automatic detection of artifacts and improved classification models for emotional activity detection from multimodal physiological data

This manuscript proposes an automated artifacts detection and multimodal classification system for human emotion analysis from human physiological signals. First, multimodal physiological data, including the Electrodermal Activity (EDA), electrocardiogram (ECG), Blood Volume Pulse (BVP) and respiration rate signals are collected. Second, a Modified Compressed Sensing-based Decomposition (MCSD) is used to extract the informative Skin Conductance Response (SCR) events of the EDA signal. Third, raw features (edge and sharp variations), statistical and wavelet coefficient features of EDA, ECG, BVP, respiration and SCR signals are obtained. Fourth, the extracted raw features, statistical and wavelet coefficient features from all physiological signals are fed into the parallel Deep Convolutional Neural Network (DCNN) to reduce the dimensionality of feature space by removing artifacts. Fifth, the fused artifact-free feature vector is obtained for neutral, stress and pleasure emotion classes. Sixth, an artifact-free feature vector is used to train the Random Forest Deep Neural Network (RFDNN) classifier. Then, a trained RFDNN classifier is applied to classify the test signals into different emotion classes. Thus, leveraging the strengths of both RF and DNN algorithms, more comprehensive feature learning using multimodal psychological data is achieved, resulting in robust and accurate classification of human emotional activities. Finally, an extensive experiment using the Wearable Stress and Affect Detection (WESAD) dataset shows that the proposed system outperforms other existing human emotion classification systems using physiological data.

Integration of MFCC Extraction and LSTM Algorithm on PYNQ-Z2 for Enhanced Audio Analysis

The need for Speech Emotion Recognition (SER) is growing since researchers have found it difficult to interpret human emotions from speech data. SER is very interesting yet very challenging task of human-computer interaction (HCI). The SER application can be benefitted depending on the type of feature extraction technique and model used for classification. Deep Learning has made a great impact in the field of audio, image, video, EEG and ECG classification. The speech signal characteristics and classification model affect how well the SER application performs. The paper briefs about deploying Deep Learning Algorithm on FPGA based board i.e., PYNQ-Z2. MFCC feature extraction technique and LSTM model used for classification of human emotion is implemented on the board. Emotion can be predicted using led buttons on the board.

EMSN: An Energy-Efficient Memristive Sequencer Network for Human Emotion Classification in Mental Health Monitoring

Mental health problems are an increasingly common social issue severely affecting health and well-being. Multimedia processing technologies via facial expression show appealing prospects in the consumer field for mental health monitoring, while still suffer from intensive computation and low energy efficiency. This paper proposes an energy-efficiency memristive sequencer network (EMSN) for human emotion classification, which offers an environmentally friendly approach for consumers with low cost and easily deployable hardware. Firstly, two-dimensional (2D) materials are employed to construct an eco-friendly memristor, the efficacy and reliability of which are confirmed through performance testing. Then, a sequencer block is proposed using memristive circuits. Notably, it is a core component of the EMSN, consisting of a bidirectional long short-term memory circuit, normalisation circuit module, and multi-layer perception module. After combining some necessary function modules, the EMSN can be achieved. Furthermore, the proposed EMSN is applied for human emotion classification. The experimental results demonstrate that the proposed EMSN has advantages in computational efficiency and classification accuracy compared to existing mainstream methods, indicating an advancement in consumer health monitoring.

Deep Emotions Recognition from Facial Expressions using Deep Learning

Deep emotion recognition has a wide range of applications, including human-robot communication, business, movies, services hotels, and even politics. Despite the use of various supervised and unsupervised methods in many different fields, there is still a lack of accurate analysis. Therefore, we have taken on this challenge as our research problem. We have proposed a mechanism for efficient and fine-grained classification of human deep emotions that can be applied to many other problems in daily life. This study aims to explore the best-suited algorithm along with optimal parameters to provide a solution for an efficient emotion detection machine learning system. In this study, we aimed to recognize emotions from facial expressions using deep learning techniques and the JAFFE dataset. The performance of three different models, a CNN (Convolutional Neural Network), an ANN (Artificial Neural Network), and an SVM (Support Vector Machine) were evaluated using precision, recall, F1-score, and accuracy as the evaluation metrics. The results of the experiments show that all three models performed well in recognizing emotions from facial expressions. The CNN model achieved a precision of 0.653, recall of 0.561, F1-score of 0.567, and accuracy of 0.62. The ANN model achieved a precision of 0.623, recall of 0.542, F1-score of 0.542, and accuracy of 0.59. The SVM model achieved a precision of 0.643, recall of 0.559, F1-score of 0.545, and accuracy of 0.6. Overall, the results of the study indicate that deep learning techniques can be effectively used for recognizing emotions from facial expressions using the JAFFE dataset.

Robust Human Face Emotion Classification Using Triplet-Loss-Based Deep CNN Features and SVM

Human facial emotion detection is one of the challenging tasks in computer vision. Owing to high inter-class variance, it is hard for machine learning models to predict facial emotions accurately. Moreover, a person with several facial emotions increases the diversity and complexity of classification problems. In this paper, we have proposed a novel and intelligent approach for the classification of human facial emotions. The proposed approach comprises customized ResNet18 by employing transfer learning with the integration of triplet loss function (TLF), followed by SVM classification model. Using deep features from a customized ResNet18 trained with triplet loss, the proposed pipeline consists of a face detector used to locate and refine the face bounding box and a classifier to identify the facial expression class of discovered faces. RetinaFace is used to extract the identified face areas from the source image, and a ResNet18 model is trained on cropped face images with triplet loss to retrieve those features. An SVM classifier is used to categorize the facial expression based on the acquired deep characteristics. In this paper, we have proposed a method that can achieve better performance than state-of-the-art (SoTA) methods on JAFFE and MMI datasets. The technique is based on the triplet loss function to generate deep input image features. The proposed method performed well on the JAFFE and MMI datasets with an accuracy of 98.44% and 99.02%, respectively, on seven emotions; meanwhile, the performance of the method needs to be fine-tuned for the FER2013 and AFFECTNET datasets.

Социально-психологический анализ основных проблем психологии правовых отношений

The social psychology of legal relations is a new scientific field. It develops at the meeting point of social and legal psychology. The main scientific problem in studying legal relations is associated with understanding the essence of law. Two leading directions have emerged in legal science: the doctrine of the natural and the social nature of law. In psychological terms, these doctrines reflect one of the main issues of psychology about the relationship between the biological and the social in the human psyche. The concept of the natural origin of law gives psychologists the opportunity to study law from the perspective of the biological foundations of the human psyche. The concept of the social nature of law (positive law) is linked to social psychology and orients psychologists towards the study of legal relations. In legal reality, there is a close relationship between natural law and positive law, which emphasises the unity of the biological and social essence of human beings.
 Theoretical analysis of the aspects of legal relations is determined by actual problems. Interest in the problems of legal relations is caused by the need to form a state governed by the rule of law and to establish a new system of legal relations in the global community. The Western legal and social psychology pays not much attention to the legal relations. The article analyses the concept of «legal relations» on the basis of national legal and socio-psychological literature. There is no consensus in jurisprudence regarding the definition of «legal relations». Legal relations are a form of social relations. Problems in the formation of consolidated legal relations are linked to insufficient knowledge in the field of law, especially among young people. When studying legal relations it is necessary to take into account psychological peculiarities of legal behavior. The article examines two socio-psychological aspects of the formation of legal relations. The first aspect is the interrelation of psyche and law, the second aspect is the connection of law with morality and ethics. The unity of law, morality and ethics is determined by the necessity of not only legal but also moral regulation of behavior. Philosophical and legal literature has comprehensively analysed the relationship between law and the human psyche. G. W. F. Hegel emphasised the psychological nature of law, since the phenomenon of law emerges as a result of the spiritual activity of a person. On this basis public legal relations originate. In Russian science, the doctrine of natural law and the relationship between psyche and law was developed by L. I. Petrazhitsky who focused his attention on the study of motives of legal behavior and developed the classification of human emotions. Positive law is psychologically expressed in the concepts of duty, responsibility and obligation. A socio-psychological analysis shows that the structure of legal relations is made up of cognitive, communicative and behavioral components. The social nature of law is determined by the moral foundations of society. Morality shapes the ideal of human behavior, while law determines the requirements for achieving that ideal.

Innovative Measures of Verhulst Diagram for Emotion Recognition using Eye-Blinking Variability

Background: The human body continuously reveals the status of several organs through biomedical signals. Over time, biomedical signal acquisition, monitoring, and analysis have captured the attention of many scientists for further prediction, diagnosis, decision-making, and recognition. Recently, building an intelligent emotion recognition system has become a challenging issue using the application of signal processing. Frequently, human emotion classification was proposed utilizing the internal body status in dealing with affective provocations. However, external states, such as eye movements, have been claimed to convey practical information about the participant’s emotions. In this study, we proposed an automatic emotion recognition scheme through the analysis of a single-modal eye-blinking variability. Methods: Initially, the signal was transformed into a 2D space using the Verhulst diagram, a simple analysis based on the signal’s dynamics. Next, some innovative features were introduced to characterize the maps. Then, the extracted measures were inputted to the support vector machine (SVM) and k-nearest neighbor (kNN). The former classifier was evaluated with three kernel functions, including RBF, linear, and polynomial. The latter performances were examined with different values for k. Moreover, the classification results were assessed in two feature-set partitioning modes: a 5-fold and 10-fold cross-validation. Results: The results showed a statistically significant difference between neutral/fear and neutral/sadness for all Verhulst indices. Also, the average values of these characteristics were higher for fear and sadness than those of other emotions. Our results indicated a maximum rate of 100% for the fear/neutral classification. Therefore, the suggested Verhulst-based approach was supremely talented in emotion classification and analysis using eye-blinking signals. Conclusion: The novel biomarkers set the scene for designing a simple accurate emotion recognition system. Additionally, this experiment could fortify the territory of ocular affective computing, and open a new horizon for diagnosing or treating various emotion deficiency disorders.

Human Emotion Classification based on EEG Signals Using Recurrent Neural Network And KNN

In human contact, emotion is very crucial. Attributes like words, voice intonation, facial expressions, and kinesics can all be used to portray one's feelings. However, brain-computer interface (BCI) devices have not yet reached the level required for emotion interpretation. With the rapid development of machine learning algorithms, dry electrode techniques, and different real-world applications of the brain-computer interface for normal individuals, emotion categorization from EEG data has recently gotten a lot of attention. Electroencephalogram (EEG) signals are a critical resource for these systems. The primary benefit of employing EEG signals is that they reflect true emotion and are easily resolved by computer systems. In this work, EEG signals associated with good, neutral, and negative emotions were identified using channel selection preprocessing. However, researchers had a limited grasp of the specifics of the link between various emotional states until now. To identify EEG signals, we used discrete wavelet transform and machine learning techniques such as recurrent neural network (RNN) and k-nearest neighbor (kNN) algorithm. Initially, the classifier methods were utilized for channel selection. As a result, final feature vectors were created by integrating the features of EEG segments from these channels. Using the RNN and kNN algorithms, the final feature vectors with connected positive, neutral, and negative emotions were categorized independently. The classification performance of both techniques is computed and compared. Using RNN and kNN, the average overall accuracies were 94.844 % and 93.438 %, respectively.

Human emotion detection and classification using modified viola-jones and convolution neural network

<span lang="EN-US">Facial expression is a kind of nonverbal communication that conveys information about a person's emotional state. Human emotion detection and recognition remains a major task in computer vision (CV) and artificial intelligence (AI). To recognize and identify the many sorts of emotions, several algorithms are proposed in the literature. In this paper, the modified Viola-Jones method is introduced to provide a robust approach capable of detecting and identifying human feelings such as angerness,sadness, desire, surprise, anxiety, disgust, and neutrality in real-time. This technique captures real-time pictures and then extracts the characteristics of the facial image to identify emotions very accurately. In this method, many feature extraction techniques like gray-level co-occurrence matrix (GLCM), linear binary pattern (LBP) and robust principal components analysis (RPCA) are applied to identify the distinct mood states and they are categorized using a convolution neural network (CNN) classifier. The obtained outcome demonstrates that the proposed method outperforms in terms of determining the rate of emotion recognition as compared to the current human emotion recognition techniques.</span><br /><div style="mso-element: comment-list;"><div style="mso-element: comment;"><div id="_com_1" class="msocomtxt"><!--[if !supportAnnotations]--></div><!--[endif]--></div></div>

Towards emotionally intelligent buildings: A Convolutional neural network based approach to classify human emotional experience in virtual built environments

Accurate classification of human emotions in designed spaces is essential for architects and engineers, who aim to maximize positive emotions by configuring architectural design features. Previous studies at the conjunction of neuroscience and architecture confirmed the impact of architectural design features on human emotions. Recent development of biometric sensors enabled researchers to identify emotions by measuring human physiological responses (e.g., the use of electroencephalogram (EEG) to measure brain activities). However, a gap in the knowledge exists in terms of an accurate classification model for human emotions in design variants. This study proposed a convolutional neural network (CNN) based approach to classify human emotions. The approach considered two types of CNN architectures as CNN ensemble and auto-encoders. The inputs of these CNN algorithms were 2D images generated by projecting the frequency band power of EEG onto the scalp graph in accordance with the electrode placements. This transformation from time-series EEG data to 2D frequency band power images retain the spatial, time and frequency domain features from participants’ brain dynamics. Performance of the proposed approach was validated using multiple metrics, including precision, recall, f-1 score, and Area Under Curve (AUC). Results showed that the auto-encoder based approach achieved the best performance with an AUC of 0.95.

Treatment of Imbalance Dataset for Human Emotion Classification

Treatment of Imbalance Dataset for Human Emotion Classification

Human emotion classification enabled by EEG signal analysis and machine learning

Human emotion classification enabled by EEG signal analysis and machine learning

An Efficient AP-ANN-Based Multimethod Fusion Model to Detect Stress through EEG Signal Analysis.

Stress is a universal emotion that every human experiences daily. Psychologists say stress may lead to heart attack, depression, hypertension, strokes, or even sudden death. Many technical explorations like stress detection through facial expression, speech, text, physical behaviors, etc., were explored, but no consensus has been reached on the best method. The advancement in biomedical engineering yielded a rapid development of electroencephalogram (EEG) signal analysis that has inspired the idea of a multimethod fusion approach for the first time which employs multiple techniques such as discrete wavelet transform (DWT) for de-noising, adaptive synthetic sampling (ADASYN) for class balancing, and affinity propagation (AP) as a stratified sampling model along with the artificial neural network (ANN) as the classifier model for human emotion classification. From the EEG recordings of the DEAP dataset, the artifacts are removed, the signal is decomposed using a DWT, and features are extracted and fused to form the feature vector. As the dataset is high-dimensional, feature selection is done and ADASYN is used to address the imbalance of classes resulting in large-scale data. The innovative idea of the proposed system is to perform sampling using affinity propagation as a stratified sampling-based clustering algorithm as it determines the number of representative samples automatically which makes it superior to the K-Means, K-Medoid, that requires the K-value. Those samples are used as inputs to various classification models, the comparison of the AP-ANN, AP-SVM, and AP-RF is done, and their most important five performance metrics such as accuracy, precision, recall, F1-score, and specificity were compared. From our experiment, the AP-ANN model provides better accuracy of 86.8% and greater precision of 85.7%, a higher F1 score of 84.9%, a recall rate of 84.1%, and a specificity value of 89.2% which altogether provides better results than the other existing algorithms.

Hybrid Model Recognition and Classification of Human Emotions in Thermal Images

Hybrid Model Recognition and Classification of Human Emotions in Thermal Images

Automated robust human emotion classification system using hybrid EEG features with ICBrainDB dataset.

Emotion identification is an essential task for human-computer interaction systems. Electroencephalogram (EEG) signals have been widely used in emotion recognition. So far, there have been several EEG-based emotion recognition datasets that the researchers have used to validate their developed models. Hence, we have used a new ICBrainDB EEG dataset to classify angry, neutral, happy, and sad emotions in this work. Signal processing-based wavelet transform (WT), tunable Q-factor wavelet transform (TQWT), and image processing-based histogram of oriented gradients (HOG), local binary pattern (LBP), and convolutional neural network (CNN) features have been used extracted from the EEG signals. The WT is used to extract the rhythms from each channel of the EEG signal. The instantaneous frequency and spectral entropy are computed from each EEG rhythm signal. The average, and standard deviation of instantaneous frequency, and spectral entropy of each rhythm of the signal are the final feature vectors. The spectral entropy in each channel of the EEG signal after performing the TQWT is used to create the feature vectors in the second signal side method. Each EEG channel is transformed into time-frequency plots using the synchrosqueezed wavelet transform. Then, the feature vectors are constructed individually using windowed HOG and LBP features. Also, each channel of the EEG data is fed to a pretrained CNN to extract the features. In the feature selection process, the ReliefF feature selector is employed. Various feature classification algorithms namely, k-nearest neighbor (KNN), support vector machines, and neural networks are used for the automated classification of angry, neutral, happy, and sad emotions. Our developed model obtained an average accuracy of 90.7% using HOG features and a KNN classifier with a tenfold cross-validation strategy.

Wavelet based machine learning models for classification of human emotions using EEG signal

Humans have the ability to portray different expressions contrary to the emotional state of mind. Therefore, it is difficult to judge the human's real emotional state simply by judging the physical appearance. Although researchers are working on facial expressions analysis, voice recognition, gesture recognition accuracy levels of such analysis are much less and the results are not reliable. Classifying the human emotions with machine learning models and extracting discrete wavelet features of Electroencephalogram (EEG) is proposed. The EEG data from Database for Emotion Analysis using Physiological signal (DEAP) online datasets is used for analysis and consists of peripheral biological signals as well as EEG recordings. EEG signal is collected from 32 subjects while watching 40 1-min-long music videos. Each video clip is rated by the participants in terms of the level of Valence, Arousal, Dominance. In the proposed work we have considered a significant band of EEG with a reduced frontal electrode (Fp1, F3, F4, Fp2) to get a comparable good result. The accuracy obtained from K- nearest neighbour (KNN), Fine KNN and Support Vector Machine (SVM) are 92.5%, 90% and 90% respectively for Valence, Arousal and Dominance.