Emotion Classification Model Research Articles

HunEmBERT: A Fine-Tuned BERT-Model for Classifying Sentiment and Emotion in Political Communication

The growing number of digitally accessible text corpora and the accelerating development of NLP tools and methods (particularly the emergence of powerful large-scale language models) have allowed their widespread use in various classification tasks, including the vast field of sentiment analysis. However, these models must often be fine-tuned to perform this task efficiently. Therefore, we aimed to create a transformer-based fine-tuned model for the emotion and sentiment analysis of Hungarian political texts. The training data for the model were the manually annotated parliamentary speech texts from 2014 to 2018, which have the advantage of being rich in various emotions. The compiled corpus can be freely used for research purposes. In our work, we describe in detail the process of fine-tuning the Hungarian BERT model for sentiment and emotion classification, the performance achieved, and the typical classification errors, mainly due to a lack of recognition of pragmatic and other language use features by the fine-tuned models.

EMOSENSE – Multi-Modal Emotion Recognition to Identify Emotions

An extended study has been done over the past years to better comprehend human emotions. The embracement of technology to recognize and react to human emotions has become a required component of society. We present a fully functional multi-modal emotion recognition system in this study that integrates data from text, voice, facial expressions, and body language. In this study, the automatic classification of anger, fear, joy, sadness, surprise, disgust, and neutral emotions from text, facial expressions, voice, and body movements have been studied on the TESS, MELD, FER2013, and EDNLP datasets. Random Forest Classifier has been used for the classification of emotions using body language, VGG16 pre-trained model for facial emotion classification, Logistic Resgression for text emotion classification, and CNN for voice emotion classification. The logistic regression model for text emotion prediction leverages natural language processing (NLP) techniques to extract emotions from textual data. The CNN-based voice model utilizes speech recognition and emotion recognition algorithms to analyze audio signals and detect emotional cues in the speaker's voice. The facial expression model employs a combination of CNN-based VGG16 pre-trained model and modified convolutional layers to detect emotions. Meanwhile, the Random Forest Clasifier model is used to capture and interpret non-verbal cues, such as gestures, posture, and overall body movements, to enrich the emotion detection process. The real strength of our proposed system lies in its ability to synergistically combine information from multiple modalities.

Parameter Tuned Machine Learning Based Emotion Recognition on Arabic Twitter Data

Arabic is one of the most spoken languages across the globe. However, there are fewer studies concerning Sentiment Analysis (SA) in Arabic. In recent years, the detected sentiments and emotions expressed in tweets have received significant interest. The substantial role played by the Arab region in international politics and the global economy has urged the need to examine the sentiments and emotions in the Arabic language. Two common models are available: Machine Learning and lexicon-based approaches to address emotion classification problems. With this motivation, the current research article develops a Teaching and Learning Optimization with Machine Learning Based Emotion Recognition and Classification (TLBOML-ERC) model for Sentiment Analysis on tweets made in the Arabic language. The presented TLBOML-ERC model focuses on recognising emotions and sentiments expressed in Arabic tweets. To attain this, the proposed TLBOML-ERC model initially carries out data pre-processing and a Continuous Bag Of Words (CBOW)-based word embedding process. In addition, Denoising Autoencoder (DAE) model is also exploited to categorise different emotions expressed in Arabic tweets. To improve the efficacy of the DAE model, the Teaching and Learning-based Optimization (TLBO) algorithm is utilized to optimize the parameters. The proposed TLBOML-ERC method was experimentally validated with the help of an Arabic tweets dataset. The obtained results show the promising performance of the proposed TLBOML-ERC model on Arabic emotion classification.

EMPOLITICON: NLP and ML Based Approach for Context and Emotion Classification of Political Speeches From Transcripts

Political speeches have played one of the most influential roles in shaping the world. Speeches of the written variety have been etched into history. These sorts of speeches have a great effect on the general people and their actions in the coming few days. Moreover, if left unchecked, political personnel or parties may cause major problems. In many cases, there may be a warning sign that the government needs to change its policies and also listen to the people. Understanding the emotion and context of a political speech is important, as they can be early indicators or warning signs for impending international crises, alignments, wars and future conflicts. In our research, we have focused on the presidents/prime ministers of China, Russia, the United Kingdom and the United States which are the permanent members of the United Nations Security Council and classified the speeches given by them based on the context and emotion of the speeches. The speeches were categorized into optimism, neutral, joy or upset in terms of emotion and five context categories, which are international affairs, nationalism, development, extremism and others. Here, optimism is a secondary emotion, whereas joy and upset are primary emotions. Apart from classifying the speeches based on context and emotion, one of the major works of our research is that we are introducing a dataset of political speeches that contains 2010 speeches labelled with emotion and context of the speech. The speeches we have worked on are large in word count. We propose EMPOLITICON-Context, a soft voting classifier ensemble learning model for context classification and EMPOLITICON-Emotion, a soft voting classifier ensemble learning model for emotion classification of political speeches. The proposed EMPOLITICON-Context model has achieved 73.13% accuracy in terms of context classification and the EMPOLITICON-Emotion model has achieved 53.07% accuracy in classifying the emotion of the political speeches.

Multi-Source Discriminant Subspace Alignment for Cross-Domain Speech Emotion Recognition

Cross-domain speech emotion recognition (SER) is an effective strategy to improve the generalization ability of emotion classification models, which is an important research direction in speech signal processing. However, since the speech signals are non-stationary, it is difficult to train a robust classifier from single-source emotional corpus. To solve this shortcoming, we propose a novel method named multi-source discriminant subspace alignment (MDSA) for cross-domain SER. In MDSA, we first conduct linear discriminant analysis (LDA) in the multi-source domain. Then, the instances in the multi-source discriminant subspace are used to linearly reconstruct the instances in the target subspace. At the same time, the reconstruction contribution of each source discriminant subspace is determined by adaptive weights. Furthermore, the multi-source discriminant subspace is aligned by reducing the loss between projections, which can make our model more robust. In this way, MDSA considers both the alignment of cross-domain data distribution and the structural information of cross-domain instances. Finally, extensive experiments are conducted on five standard emotional corpora, i.e., Berlin, IEMOCAP, CVE, EMOVO, and TESS, and the results demonstrate the proposed MDSA is superior to several state-of-the-art transfer learning algorithms in terms of performance. The codes are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/shaokai1209/MDSA.</uri>

The Application and Research of Double-layer Music Emotion Classification Model Based on Random Forest Algorithm in Digital Music

The Application and Research of Double-layer Music Emotion Classification Model Based on Random Forest Algorithm in Digital Music

The application and research of double-layer music emotion classification model based on random forest algorithm in digital music

The application and research of double-layer music emotion classification model based on random forest algorithm in digital music

Speech Emotion Classification using Ensemble Models with MFCC

Speech is one of the most promising features that reflects the underlying emotion of a human being. There are some measurable parameters in speech signals that reveal a persons affective state. Speech Emotion Recognition (SER) is a process of identifying the emotional elements in communication regardless of contextual relevance. Leveraging studies have taken place in this area. This paper proposes an ensemble model to automatically classify emotion from speech signals to one among the seven emotional classes neutral, calm, angry, sad, happy, fear, disgust, and surprised. In this work, speech spectral features have been extracted using Mel Frequency Cepstral Coefficient (MFCC). An emotion classification model based on 2-Dimensional Convolutional Neural Networks (2D-CNN) and eXtreme Gradient Boosting (XG-Boost) is proposed in this paper. This work also compares the performance of the proposed ensemble model with baseline models and other ensemble models. The accuracy of each model on the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) dataset is computed and the proposed model shows maximum accuracy in classifying emotions.

EEG Emotion Recognition Applied to the Effect Analysis of Music on Emotion Changes in Psychological Healthcare.

Music therapy is increasingly being used to promote physical health. Emotion semantic recognition is more objective and provides direct awareness of the real emotional state based on electroencephalogram (EEG) signals. Therefore, we proposed a music therapy method to carry out emotion semantic matching between the EEG signal and music audio signal, which can improve the reliability of emotional judgments, and, furthermore, deeply mine the potential influence correlations between music and emotions. Our proposed EER model (EEG-based Emotion Recognition Model) could identify 20 types of emotions based on 32 EEG channels, and the average recognition accuracy was above 90% and 80%, respectively. Our proposed music-based emotion classification model (MEC model) could classify eight typical emotion types of music based on nine music feature combinations, and the average classification accuracy was above 90%. In addition, the semantic mapping was analyzed according to the influence of different music types on emotional changes from different perspectives based on the two models, and the results showed that the joy type of music video could improve fear, disgust, mania, and trust emotions into surprise or intimacy emotions, while the sad type of music video could reduce intimacy to the fear emotion.

An Efficient AP-ANN-Based Multimethod Fusion Model to Detect Stress through EEG Signal Analysis.

Stress is a universal emotion that every human experiences daily. Psychologists say stress may lead to heart attack, depression, hypertension, strokes, or even sudden death. Many technical explorations like stress detection through facial expression, speech, text, physical behaviors, etc., were explored, but no consensus has been reached on the best method. The advancement in biomedical engineering yielded a rapid development of electroencephalogram (EEG) signal analysis that has inspired the idea of a multimethod fusion approach for the first time which employs multiple techniques such as discrete wavelet transform (DWT) for de-noising, adaptive synthetic sampling (ADASYN) for class balancing, and affinity propagation (AP) as a stratified sampling model along with the artificial neural network (ANN) as the classifier model for human emotion classification. From the EEG recordings of the DEAP dataset, the artifacts are removed, the signal is decomposed using a DWT, and features are extracted and fused to form the feature vector. As the dataset is high-dimensional, feature selection is done and ADASYN is used to address the imbalance of classes resulting in large-scale data. The innovative idea of the proposed system is to perform sampling using affinity propagation as a stratified sampling-based clustering algorithm as it determines the number of representative samples automatically which makes it superior to the K-Means, K-Medoid, that requires the K-value. Those samples are used as inputs to various classification models, the comparison of the AP-ANN, AP-SVM, and AP-RF is done, and their most important five performance metrics such as accuracy, precision, recall, F1-score, and specificity were compared. From our experiment, the AP-ANN model provides better accuracy of 86.8% and greater precision of 85.7%, a higher F1 score of 84.9%, a recall rate of 84.1%, and a specificity value of 89.2% which altogether provides better results than the other existing algorithms.

SENTIMENT CLASSIFICATION OF E-COMMERCE REVIEWS BASED ON BERT-CNN

Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) were used to handle natural language tasks in early days, but the Transformer model changed that. The Bidirectional Encoder Representations from Transformers (BERT) model is another optimization based on the Transformer model, which directly makes the performance of the NLP model reach an unprecedented height. In order to distinguish the emotion classification model with the best processing effect in the field of e-commerce reviews, the BERT model is fine-tuned based on the mobile e-commerce review data, and then input to another deep learning models(such as CNN,RNN) as embedding. Finally, we compare the training effect of several current deep learning models, such as BERT, BERT-RNN and BERT-CNN. Experimental results show that the BERT-CNN model performs best in the binary classification of e-commerce review text sentiment.

Multi-Kernel Temporal and Spatial Convolution for EEG-Based Emotion Classification.

Deep learning using an end-to-end convolutional neural network (ConvNet) has been applied to several electroencephalography (EEG)-based brain-computer interface tasks to extract feature maps and classify the target output. However, the EEG analysis remains challenging since it requires consideration of various architectural design components that influence the representational ability of extracted features. This study proposes an EEG-based emotion classification model called the multi-kernel temporal and spatial convolution network (MultiT-S ConvNet). The multi-scale kernel is used in the model to learn various time resolutions, and separable convolutions are applied to find related spatial patterns. In addition, we enhanced both the temporal and spatial filters with a lightweight gating mechanism. To validate the performance and classification accuracy of MultiT-S ConvNet, we conduct subject-dependent and subject-independent experiments on EEG-based emotion datasets: DEAP and SEED. Compared with existing methods, MultiT-S ConvNet outperforms with higher accuracy results and a few trainable parameters. Moreover, the proposed multi-scale module in temporal filtering enables extracting a wide range of EEG representations, covering short- to long-wavelength components. This module could be further implemented in any model of EEG-based convolution networks, and its ability potentially improves the model's learning capacity.

Computational Linguistics Based Emotion Detection and Classification Model on Social Networking Data

Computational linguistics (CL) is the application of computer science for analysing and comprehending written and spoken languages. Recently, emotion classification and sentiment analysis (SA) are the two techniques that are mostly utilized in the Natural Language Processing (NLP) field. Emotion analysis refers to the task of recognizing the attitude against a topic or target. The attitude may be polarity (negative or positive) or an emotional state such as sadness, joy, or anger. Therefore, classifying posts and opinion mining manually is a difficult task. Data subjectivity has made this issue an open problem in the domain. Therefore, this article develops a computational linguistics-based emotion detection and a classification model on social networking data (CLBEDC-SND) technique. The presented CLBEDC-SND technique investigates the recognition and classification of emotions in social networking data. To attain this, the presented CLBEDC-SND model performs different stages of data pre-processing to make it compatible for further processing. In addition, the CLBEDC-SND model undergoes vectorization and sentiment scoring process using fuzzy approach. For emotion classification, the presented CLBEDC-SND model employs extreme learning machine (ELM). Finally, the parameters of the ELM model are optimally modified by the use of the shuffled frog leaping optimization (SFLO) algorithm. The performance validation of the CLBEDC-SND model is tested using benchmark datasets. The experimental results demonstrate the better performance of the CLBEDC-SND model over other models.

Emotion Analysis Based on Deep Learning With Application to Research on Development of Western Culture.

Cultural development is often reflected in the emotional expression of various cultural carriers, such as literary works, movies, etc. Therefore, the cultural development can be analyzed through emotion analysis of the text, so as to sort out its context and obtain its development dynamics. This paper proposes a text emotion analysis method based on deep learning. The traditional neural network model mainly deals with the classification task of short texts in the form of word vectors, which causes the model to rely too much on the accuracy of word segmentation. In addition, the short texts have the characteristics of short corpus and divergent features. A text emotion classification model combing the Bidirectional Encoder Representations from Transformers (BERT) and Bi-directional Long Short-Term Memory (BiLSTM) is developed in this work. First, the BERT model is used to convert the trained text into a word-based vector representation. Then, the generated word vector is employed as the input of the BiLSTM to obtain the semantic representation of the context of the relevant word. By adding random dropout, the mechanism prevents the model from overfitting. Finally, the extracted feature vector is input to the fully connected layer, and the emotion category to which the text belongs is calculated through the Softmax function. Experiments show that in processing short texts, the proposed model based on BERT-BiLSTM is more accurate and reliable than the traditional neural network model using word vectors. The proposed method has a better analysis effect on the development of western culture.

Emotion Recognition and Classification of Film Reviews Based on Deep Learning and Multimodal Fusion

In terms of cross-cultural exchanges, the film is not only an important embodiment of a country’s cultural soft power but also the most direct and favorable way of communication. The advent of the all-around well-off era has propelled people’s demand for spiritual, cultural, and entertainment which promotes the vigorous development of the film culture industry. The expansion and development of China’s film market, domestic films, are a reflection and extension of China’s culture and ideas. It plays an extremely important role in enhancing cultural self-confidence and cultural output. In order to better grasp the emotional tendency of the audience, understand the viewing needs, and put forward suggestions on domestic film production, it is very necessary to analyze the emotion of film reviews and dig deep into semantics. Since the evaluation of film works considers many factors that are complex and changeable, the choice of model plays a significant role in the process of emotion analysis. The deep learning model represented by a deep neural network has high tolerance to sentence noise, has strong information discrimination, and features self-learning ability. It also has great advantages in emotion classification tasks. This study conducts an in-depth study and research on the traditional emotion analysis methods and finally puts forward an effective emotion analysis framework that combines the traditional emotion analysis method and deep learning network. This framework enhances the text vectorization representation and emotion classification model by performing emotion analysis. The effectiveness is verified by corresponding experiments which justify the superiority of the approach.

EEG-based cross-subject emotion recognition using Fourier-Bessel series expansion based empirical wavelet transform and NCA feature selection method

Automated emotion recognition using brain electroencephalogram (EEG) signals is predominantly used for the accurate assessment of human actions as compared to facial expression or speech signals. Various signal processing methods have been used for extracting representative features from EEG signals for emotion recognition. However, the EEG signals are non-stationary and vary across the subjects as well as in different sessions of the same subject; hence it exhibits poor generalizability and low classification accuracy for an emotion classification of cross subjects. In this paper, EEG signals-based automated cross-subject emotion recognition framework is proposed using the Fourier-Bessel series expansion-based empirical wavelet transform (FBSE-EWT) method. This method is used to decompose the EEG signals from each channel into four sub-band signals. Manually ten channels are selected from the frontal lobe, from which entropy and energy features are extracted from each sub-band signal. The subject variability is reduced using an average moving filter method on each channel to obtain the smoothened feature vector of size 80. The three feature selection techniques, such as neighborhood component analysis (NCA), relief-F, and mRMR, are used to obtain an optimal feature vector. The machine learning models, such as artificial neural network (ANN), k-nearest neighborhood (k-NN) with two (fine and weighted) functions, and ensemble bagged tree classifiers are trained by the obtained feature vectors. The experiments are performed on two publicly accessible databases, named SJTU emotion EEG dataset (SEED) and dataset for emotion analysis using physiological signals (DEAP). The training and testing of the models have been performed using 10-fold cross-validation and leave-one-subject-out-cross-validation (LOSOCV). The proposed framework based on FBSE-EWT and NCA feature selection approach shows superior results for classifying human emotions compared to other state-of-art emotion classification models.

EEG-Based Emotion Classification Using Improved Cross-Connected Convolutional Neural Network.

The use of electroencephalography to recognize human emotions is a key technology for advancing human–computer interactions. This study proposes an improved deep convolutional neural network model for emotion classification using a non-end-to-end training method that combines bottom-, middle-, and top-layer convolution features. Four sets of experiments using 4500 samples were conducted to verify model performance. Simultaneously, feature visualization technology was used to extract the three-layer features obtained by the model, and a scatterplot analysis was performed. The proposed model achieved a very high accuracy of 93.7%, and the extracted features exhibited the best separability among the tested models. We found that adding redundant layers did not improve model performance, and removing the data of specific channels did not significantly reduce the classification effect of the model. These results indicate that the proposed model allows for emotion recognition with a higher accuracy and speed than the previously reported models. We believe that our approach can be implemented in various applications that require the quick and accurate identification of human emotions.

Music aesthetic teaching and emotional visualization under emotional teaching theory and deep learning

The study aims to overcome the shortcomings of the traditional music teaching system, for it cannot analyze the emotions of music works and does not have the advantages in music aesthetic teaching. First, the relevant theories of emotional teaching are expounded and the important roles of emotional teaching and aesthetic teaching in shaping students’ personalities are described. Second, a music emotion classification model based on the deep neural network (DNN) is proposed, and it can accurately classify music emotions through model training. Finally, according to the emotional teaching theory and the model based on DNN, a visual system of music teaching is designed for visualizing the emotions, which is helpful to students’ understanding of music works and the improvement of teaching effect. The results show that: (1) the teaching system designed has five parts, namely the audio input layer, emotion classification layer, virtual role perception layer, emotion expression layer, and output layer. The system can classify the emotions of the current input audio and map it to the virtual characters for emotional expression. Finally, the emotions are displayed to the students through the display screen layer to realize the visualization of the emotions of music works, so that the students can intuitively feel the emotional elements in the works. (2) The accuracy of the music emotion classification model based on DNN is more than 3.4% higher than other models and has better performance. The study provides important technical support for the upgrading of the teaching system and improving the quality of music aesthetic teaching.

CONSTRUCTION OF INDIVIDUAL COGNITIVE MODEL IN THE CONTEXT OF EMOTION AND AFFECTIVE DISORDER

CONSTRUCTION OF INDIVIDUAL COGNITIVE MODEL IN THE CONTEXT OF EMOTION AND AFFECTIVE DISORDER

Needle in a Haystack: Spotting and recognising micro-expressions “in the wild”

Computational research on facial micro-expressions has long focused on videos captured under constrained laboratory conditions due to the challenging elicitation process and limited samples that are publicly available. Moreover, processing micro-expressions is extremely challenging under unconstrained scenarios. This paper introduces, for the first time, a completely automatic micro-expression “spot-and-recognize” framework that is performed on in-the-wild videos, such as in poker games and political interviews. The proposed method first spots the apex frame from a video by handling head movements and unconscious actions which are typically larger in motion intensity, with alignment employed to enforce a canonical face pose. Optical flow guided features play a central role in our method: they can robustly identify the location of the apex frame, and are used to learn a shallow neural network model for emotion classification. Experimental results demonstrate the feasibility of the proposed methodology, establishing good baselines for both spotting and recognition tasks – ASR of 0.33 and F1-score of 0.6758 respectively on the MEVIEW micro-expression database. In addition, we present comprehensive qualitative and quantitative analyses to further show the effectiveness of the proposed framework, with new suggestion for an appropriate evaluation protocol. In a nutshell, this paper provides a new benchmark for apex spotting and emotion recognition in an in-the-wild setting.