Emotion Classification System Research Articles

A federated learning method for real-time emotion state classification from multi-modal streaming.

Emotional and physical health are strongly connected and should be taken care of simultaneously to ensure completely healthy persons. A person's emotional health can be determined by detecting emotional states from various physiological measurements (EDA, RB, EEG, etc.). Affective Computing has become the field of interest, which uses software and hardware to detect emotional states. In the IoT era, wearable sensor-based real-time multi-modal emotion state classification has become one of the hottest topics. In such setting, a data stream is generated from wearable-sensor devices, data accessibility is restricted to those devices only and usually a high data generation rate should be processed to achieve real-time emotion state responses. Additionally, protecting the users' data privacy makes the processing of such data even more challenging. Traditional classifiers have limitations to achieve high accuracy of emotional state detection under demanding requirements of decentralized data and protecting users' privacy of sensitive information as such classifiers need to see all data. Here comes the federated learning, whose main idea is to create a global classifier without accessing the users' local data. Therefore, we have developed a federated learning framework for real-time emotion state classification using multi-modal physiological data streams from wearable sensors, called Fed-ReMECS. The main findings of our Fed-ReMECS framework are the development of an efficient and scalable real-time emotion classification system from distributed multimodal physiological data streams, where the global classifier is built without accessing (privacy protection) the users' data in an IoT environment. The experimental study is conducted using the popularly used multi-modal benchmark DEAP dataset for emotion classification. The results show the effectiveness of our developed approach in terms of accuracy, efficiency, scalability and users' data privacy protection.

Driver’s emotion and behavior classification system based on Internet of Things and deep learning for Advanced Driver Assistance System (ADAS)

Classification of driver’s emotion is an important issue that can be used to increase awareness of driving habits of drivers as many drivers are overconfident and are unaware of their bad driving habits. If the drivers driving behaviors are identified automatically, the drivers can be aware of their bad habits which can assist them to avoid potential car accidents. Many researches have suggested many methods or techniques that can be helpful in determining driver’s behavior but there is no comprehensive method existed that can cater almost all types of distractions which occur while driving. In this paper the emotional as well as behavioral distraction states of the driver are analyzed and classified using advance deep learning algorithms like convolutional neural networks (CNN) and visual geometry group (VGG16). The handling of both the emotional and behavioral states can be helpful in developing a comprehensive driver’s detection system in an efficient manner addressing the previous limitations or challenges that are needed to be solved. The results show that the accuracy of CNN was far better than that of VGG16 but the training time of VGG16 was far less than that of CNN This system can be further integrated with the in-vehicle driving systems or built-in safety systems.

Text emotion classification system based on multifractal methods

Emotion classification is an attractive research, emotion classification based on machine learning has broad application prospects in the field of computer. It is particularly important to construct an effective and accurate algorithm to extract emotional features from subjective texts, and classify them into corresponding emotional categories. In this paper, we implement a novel classification system for identifying emotions in a positive and negative corpus dataset. The corpus contains hotel reviews of Chinese emotion mining, and we choose 1000 positive and 1000 negative corpus as the dataset for analysis. We first preprocess the positive and negative texts, and after obtaining the characteristic word text of positive and negative corpus, we use Word2Vec model to transform the sentence into a numerical vector. We regard the obtained feature word vector as a time series, and adopt three multifractal analysis on the time series, such as multifractal detrended fluctuation analysis (MF-DFA), Multifractal detrended moving average (MF-DMA), and Multifractal detrended weighted average algorithm of historical volatility (MF-DHV). After extracting the Hurst exponents of the vector time series, we put the features as the input vector into SVM for classification analysis. We utilize four key values such as Accuracy (Acc), Precision (P), Recall (R) and F-score (F1) to evaluate the classification experiments. The results suggest that Word2Vec-MF-DHV-SVM model is more effective and competitive in extracting information from corpus numerical vector.

BEmoC: A Corpus for Identifying Emotion in Bengali Texts.

Emotion classification in text has growing interest among NLP experts due to the enormous availability of people’s emotions and its emergence on various Web 2.0 applications/services. Emotion classification in the Bengali texts is also gradually being considered as an important task for sports, e-commerce, entertainments, and security applications. However, It is a very critical task to develop an automatic emotion classification system for low-resource languages such as, Bengali. Scarcity of resources and deficiency of benchmark corpora make the task more complicated. Thus, the development of a benchmark corpus is the prerequisite to develop an emotion classifier for Bengali texts. This paper describes the development of an emotional corpus (hereafter called ‘BEmoC’) for classifying six emotions in Bengali texts. The corpus development process consists of four key steps: data crawling, pre-processing, labelling, and verification. A total of 7000 texts are labelled into six basic emotion categories such as anger, fear, surprise, sadness, joy, and disgust, respectively. Dataset evaluation with 0.969 Cohen’s κ score indicates the close agreement between the corpus annotators and the expert. The analysis of evaluation also represents that the distribution of emotion words obeys Zipf’s law. Moreover, the results of BEmoC analysis shown in terms of coding reliability, emotion density, and most frequent emotion words, respectively.

Human Emotions Classification Using EEG via Audiovisual Stimuli and AI

Electroencephalogram (EEG) is a medical imaging technology that can measure the electrical activity of the scalp produced by the brain, measured and recorded chronologically the surface of the scalp from the brain. The recorded signals from the brain are rich with useful information. The inference of this useful information is a challenging task. This paper aims to process the EEG signals for the recognition of human emotions specifically happiness, anger, fear, sadness, and surprise in response to audiovisual stimuli. The EEG signals are recorded by placing neurosky mindwave headset on the subject’s scalp, in response to audiovisual stimuli for the mentioned emotions. Using a bandpass filter with a bandwidth of 1–100 Hz, recorded raw EEG signals are preprocessed. The preprocessed signals then further analyzed and twelve selected features in different domains are extracted. The Random forest (RF) and multilayer perceptron (MLP) algorithms are then used for the classification of the emotions through extracted features. The proposed audiovisual stimuli based EEG emotion classification system shows an average classification accuracy of 80% and 88% using MLP and RF classifiers respectively on hybrid features for experimental signals of different subjects. The proposed model outperforms in terms of cost and accuracy.

데이터 표준화를 위한 패션 감성 분류 체계

Accumulation of high-quality data is crucial for AI learning. The goal of using AI in fashion service is to propose of a creative, personalized solution that is close to the know-how of a human operator. These customized solutions require an understanding of fashion products and emotions. Therefore, it is necessary to accumulate data on the attributes of fashion products and fashion emotion. The first step for accumulating fashion data is to standardize the attribute with coherent system. The purpose of this study is to propose a fashion emotional classification system. For this, images of fashion products were collected, and metadata was obtained by allowing consumers to describe their emotions about fashion images freely. An emotional classification system with a hierarchical structure, was then constructed by performing frequency and CONCOR analyses on metadata. A final classification system was proposed by supplementing attribute values with reference to findings from previous studies and SNS data.

Emotionally Intelligent Chatbot

An emotionally intelligent chatbot system aims to make an effective conversation between humans and machines in as natural and interactive a manner as possible. The chatbot agent has the pre-embedded knowledge base to identify the sentences, intents, entities, and context of the input query to be precise for making a valid, predictable decision itself as a self-generated response to answer the query. The present technical project consists of developing an intelligent system for college enquiry purposes using a web-based chatbot agent, through machine learning, query processing, and sentiment and emotion classification system to analyze the sentiment of the visitor towards the college. Emotionally Intelligent College Enquiry Chatbot System is nothing but a chatbot to understand the user queries and respond to them during a conversation. A chatbot can actively help humans to involve in a digital automated conversation with a machine or a system effective. In the following proposed system, feature extraction and data cleaning techniques are applied to the dataset, and classifiers such as multinomial naive Bayes, logistic regression and k nearest neighbours are used to train the model. The classifier with the highest accuracy is further used for the emotion classification of users.

Real-Time Emotion Classification Using EEG Data Stream in E-Learning Contexts.

In face-to-face and online learning, emotions and emotional intelligence have an influence and play an essential role. Learners’ emotions are crucial for e-learning system because they promote or restrain the learning. Many researchers have investigated the impacts of emotions in enhancing and maximizing e-learning outcomes. Several machine learning and deep learning approaches have also been proposed to achieve this goal. All such approaches are suitable for an offline mode, where the data for emotion classification are stored and can be accessed infinitely. However, these offline mode approaches are inappropriate for real-time emotion classification when the data are coming in a continuous stream and data can be seen to the model at once only. We also need real-time responses according to the emotional state. For this, we propose a real-time emotion classification system (RECS)-based Logistic Regression (LR) trained in an online fashion using the Stochastic Gradient Descent (SGD) algorithm. The proposed RECS is capable of classifying emotions in real-time by training the model in an online fashion using an EEG signal stream. To validate the performance of RECS, we have used the DEAP data set, which is the most widely used benchmark data set for emotion classification. The results show that the proposed approach can effectively classify emotions in real-time from the EEG data stream, which achieved a better accuracy and F1-score than other offline and online approaches. The developed real-time emotion classification system is analyzed in an e-learning context scenario.

DEEP LEARNING TECHNIQUE BASED TEXT EMOTION CLASSIFICATION SYSTEM USING GENETIC ALGORITHM TECHNIQUE

In today’s era the research area of emotion detection has becomes trendy field of research. The data in text form is the very easy way to communicate among interaction of human-machine through the social networking sites, is one of the schemes to share users views. Recognition of human’s emotion through analyzing textual documents is useful and essential, but sometimes difficult because of the fact that it is not necessary to use emotion words directly in textual expressions. In this research, a deep learning technique based Text Emotion Classification (TEC) System using Genetic Algorithm (GA) as an optimization technique is presented. Initially, a lexicon dictionary is prepared based on the emotional words and different processes such as pre-processing, feature extraction, optimization and classification has been applied to classify the textual emotion. The test data has been trained using deep learning scheme named as Deep Neural Network (DNN) with optimization technique based on GA with a novel fitness function. The emotion; happy, sad and angry are identified as per the shared data on the social media platform. Most of the state-of-the-art in the previous research on textual emotion mining is mainly focused on without utilizing the feature selection concept, so we introduce the concept of feature selection using the GA and passes an input to DNN. At the last, we compare the performance of the proposed TEC system with existing work proposed by Chatterjee et al. terms of precision, recall and F-measure and we observed that the system got better emotion classification accuracy.

An Ensemble Technique to Classify Multi-Class Textual Emotion

Classifying textual emotion plays a critical role in several HCI applications where the text is utilized as a central means of communication such as messages, reviews, blogs and other Web 2.0 platforms. The extensive usage of the Internet has emerged as an unprecedented means for people to express their feelings or emotion on blogs, social media, and e-commerce sites in recent years. Most of the emotions displayed on the online platforms are in textual forms (such as posts, tweets, comments and reviews), which are unorganized and time-consuming to structured due to their disordered forms. Although several emotion analysis tools are available in high-resource languages, it is critical to developing an automatic emotion classification system for low-resource languages, including Bengali, due to its constrained resources. This paper presents an ML-based ensemble method to classify six primary textual emotions (anger, fear, disgust, sadness, surprise and joy) from Bengali texts. An emotion corpus containing 8047 Bengali texts is developed to perform the textual emotion classification task. This work investigates eight standard ML-based techniques such as logistic regression (LR), multinomial naive Bayes (MNB), support vector machine (SVM), random forest (RF), decision tree (DT), K-nearest neighbour (KNN) and adaptive boosting (AdaBoost) and an ensemble method (a combination of LR, RF, SVM) with Bag of words (BoW) and tf-idf feature extraction techniques. The experimental result demonstrates that the ensemble with tf-idf achieved the highest weighted f1-score of 62.39% compared to other methods.

Digital Empirical Research of Influencing Factors of Musical Emotion Classification Based on Pleasure-Arousal Musical Emotion Fuzzy Model

In recent years, with the further breakthrough of artificial intelligence theory and technology, as well as the further expansion of the Internet scale, the recognition of human emotions and the necessity for satisfying human psychological needs in future artificial intelligence technology development tendencies have been highlighted, in addition to physical task accomplishment. Musical emotion classification is an important research topic in artificial intelligence. The key premise of realizing music emotion classification is to construct a musical emotion model that conforms to the characteristics of music emotion recognition. Currently, three types of music emotion classification models are available: discrete category, continuous dimensional, and music emotion-specific models. The pleasure-arousal music emotion fuzzy model, which includes a wide range of emotions compared with other models, is selected as the emotional classification system in this study to investigate the influencing factor for musical emotion classification. Two representative emotional attributes, i.e., speed and strength, are used as variables. Based on test experiments involving music and non-music majors combined with questionnaire results, the relationship between music properties and emotional changes under the pleasure-arousal model is revealed quantitatively.

Development of music emotion classification system using convolution neural network

Music emotion classification (MEC) is the multidisciplinary research area that is related to perceive the emotions from the songs and label the songs with particular emotion classes. MEC systems (MECS) extract the features from the songs and then the songs are categorized on the basis of emotions by comparing their features. In this paper an MECS has been proposed that makes use of Convolutional Neural Network (CNN) by converting the music to their visual representation known as spectrograms. By using CNN extraction of specific features of music signals is not necessarily required to classify the songs. In this work two MECS are trained and tested by using Hindi database by using CNN and third MECS system is developed by using SVM. In first MECS spectrograms are obtained by using hamming windows of size 2048 and noverlap factor of 1024 and in second MECS spectrograms are obtained by using hamming windows of size 1024 and noverlap factor of 512. The three combinations of CNN layers are used in order to classify the songs in four, eight and sixteen classes on the basis of emotional tags. The performance of MECS design is analyzed on the basis of training accuracy, validation accuracy, training loss and validation loss. Results show that the two MECS systems developed by CNN has better accuracy and less loss than the third MECS system modeled by SVM.

Multimodal Emotion Evaluation: A Physiological Model for Cost-Effective Emotion Classification.

Emotional responses are associated with distinct body alterations and are crucial to foster adaptive responses, well-being, and survival. Emotion identification may improve peoples’ emotion regulation strategies and interaction with multiple life contexts. Several studies have investigated emotion classification systems, but most of them are based on the analysis of only one, a few, or isolated physiological signals. Understanding how informative the individual signals are and how their combination works would allow to develop more cost-effective, informative, and objective systems for emotion detection, processing, and interpretation. In the present work, electrocardiogram, electromyogram, and electrodermal activity were processed in order to find a physiological model of emotions. Both a unimodal and a multimodal approach were used to analyze what signal, or combination of signals, may better describe an emotional response, using a sample of 55 healthy subjects. The method was divided in: (1) signal preprocessing; (2) feature extraction; (3) classification using random forest and neural networks. Results suggest that the electrocardiogram (ECG) signal is the most effective for emotion classification. Yet, the combination of all signals provides the best emotion identification performance, with all signals providing crucial information for the system. This physiological model of emotions has important research and clinical implications, by providing valuable information about the value and weight of physiological signals for emotional classification, which can critically drive effective evaluation, monitoring and intervention, regarding emotional processing and regulation, considering multiple contexts.

Emotion Classification System for Digital Music with a Cascaded Technique

Music selection is difficult without efficient organization based on metadata or tags, and one effective tag scheme is based on the emotion expressed by the music. However, manual annotation is labor intensive and unstable because the perception of music emotion varies from person to person. This paper presents an emotion classification system for digital music with a resolution of eight emotional classes. Russell’s emotion model was adopted as common ground for emotional annotation. The music information retrieval (MIR) toolbox was employed to extract acoustic features from audio files. The classification system utilized a supervised machine learning technique to recognize acoustic features and create predictive models. Four predictive models were proposed and compared. The models were composed by crossmatching two types of neural networks, i.e., Levenberg-Marquardt (LM) and resilient backpropagation (Rprop), with two types of structures: a traditional multiclass model and the cascaded structure of a binary-class model. The performance of each model was evaluated via the MediaEval Database for Emotional Analysis (DEAM) benchmark. The best result was achieved by the model trained with the cascaded Rprop neural network (accuracy of 89.5%). In addition, correlation coefficient analysis showed that timbre features were the most impactful for prediction. Our work offers an opportunity for a competitive advantage in music classification because only a few music providers currently tag music with emotional terms.

An analysis of environmental big data through the establishment of emotional classification system model based on machine learning: focus on multimedia contents for portal applications

With the advent of the Internet, there have been many changes. As existing means of off-line communication have transferred to the Internet, a wide range of multimedia services are emerging. And unlike in the past, these work as tremendous sources from which people’s opinions and attitudes can be obtained. Particularly, diverse opinions and reviews posted on the web in real time help corporations or the State establish the directions of policies. In this context, this study developed machine learning-based environment issue sentiment classifier in order to find out, from comments posted below news, how people recognize issues about climate change in terms of environment. Based on training data constructed by this study, a sentiment classification algorithm was constructed by applying SVM (support vector machine) and Naive Bayes, which are machine learning techniques, and a sentiment classifier was constructed by applying CNN (convolutional neural networks) and Bi-LSTM (bidirectional long-short term memory) techniques among deep learning techniques recently researched vigorously; and then their performance was compared.

Construction and Evaluation of Tamil Speech Emotion Corpus

Speech emotion-related analysis needs good emotion corpus. The construction and evaluation of two Tamil emotion corpora, one for children and the other for the adults, are described here. Children emotion speech samples are collected from 30 Tamil movies, and the length of the utterances varies from 5 to 40 s. Tamil audio plays are the resources for building the adult emotion corpus. The emotion prosodies are collected, segmented and annotated for the categories of anger, happy, sad and neutral emotions. Observers’ perception test result has been used to evaluate the annotation of emotion. Automatic emotion classification systems have been built using Gaussian Mixture Model and Support Vector Machine. The database is created with an objective of the acoustic investigation of emotion expression in Tamil, analyzing the influence of speaker's culture and age on emotion expression and investigating the requirement of the need of features unique to Tamil speech on various automatic analysis of speech like emotion recognition, speaker recognition, etc.

Deep Learning-Based Drivers Emotion Classification System in Time Series Data for Remote Applications

Aggressive driving emotions is indeed one of the major causes for traffic accidents throughout the world. Real-time classification in time series data of abnormal and normal driving is a keystone to avoiding road accidents. Existing work on driving behaviors in time series data have some limitations and discomforts for the users that need to be addressed. We proposed a multimodal based method to remotely detect driver aggressiveness in order to deal these issues. The proposed method is based on change in gaze and facial emotions of drivers while driving using near-infrared (NIR) camera sensors and an illuminator installed in vehicle. Driver’s aggressive and normal time series data are collected while playing car racing and truck driving computer games, respectively, while using driving game simulator. Dlib program is used to obtain driver’s image data to extract face, left and right eye images for finding change in gaze based on convolutional neural network (CNN). Similarly, facial emotions that are based on CNN are also obtained through lips, left and right eye images extracted from Dlib program. Finally, the score level fusion is applied to scores that were obtained from change in gaze and facial emotions to classify aggressive and normal driving. The proposed method accuracy is measured through experiments while using a self-constructed large-scale testing database that shows the classification accuracy of the driver’s change in gaze and facial emotions for aggressive and normal driving is high, and the performance is superior to that of previous methods.

Classification of Poetry Text Into the Emotional States Using Deep Learning Technique

The classification of emotional states from poetry or formal text has received less attention by the experts of computational intelligence in recent times as compared to informal textual content like SMS, email, chat, and online user reviews. In this study, an emotional state classification system for poetry text is proposed using the latest and cutting edge technology of Artificial Intelligence, called Deep Learning. For this purpose, an attention-based C-BiLSTM model is implemented on the poetry corpus. The proposed approach classifies the text of poetry into different emotional states, like love, joy, hope, sadness, anger, etc. Different experiments are conducted to evaluate the efficiency of the proposed system as compared to other state-of-art methods as well as machine learning and deep learning methods. Experimental results depict that the proposed model outperformed the baselines studies with 88% accuracy. Furthermore, the analysis of the statistical experiment also validates the performance of the proposed approach.

A Hybrid Physiological Approach of Emotional Reaction Detection Using Combined FCM and SVM Classifier.

Users' emotional reaction capturing is one of the primary issues for brain computer interface applications. Despite the intuitive feedback provided by the qualitative methods, emotional reactions are expected to be detected and classified quantitatively. Based on the human emotion representation on physiological signal, this paper offers an hybrid approach combining electroencephalogram (EEG) and facial expression together to classify the human emotion. Several advanced signal processing techniques are used to simplify the data and extract the features involving local binary patterns (LBP), Compressed Sensing (CS) and Wavelet Transform (WT). A novel machine learning algorithm, combined Fuzzy Cognitive Maps (FCM) and Support Vector Machine (SVM) are implemented to recognise the feature patterns. The result illustrates a stable emotion classification system with 75.64% accuracy. This design can provide fast and precise emotional feedback, which would further improve the communication between human and computer.

End-to-End Listening Agent for Audiovisual Emotional and Naturalistic Interactions

In this work, we established the foundations of a framework with the goal to build an end-to-end naturalistic expressive listening agent. The project was split into modules for recognition of the user’s paralinguistic and nonverbal expressions, prediction of the agent’s reactions, synthesis of the agent’s expressions and data recordings of nonverbal conversation expressions. First, a multimodal multitask deep learning-based emotion classification system was built along with a rule-based visual expression detection system. Then several sequence prediction systems for nonverbal expressions were implemented and compared. Also, an audiovisual concatenation-based synthesis system was implemented. Finally, a naturalistic, dyadic emotional conversation database was collected. We report here the work made for each of these modules and our planned future improvements.