Recognition Of Emotional States Research Articles

Stress Coping Strategies in Parents of Newborns and Infants with Congenital Cyanotic Heart Disease with Regard to Stress Levels and Negative Emotions.

Parents of children suffering from congenital heart disease experience high levels of stress and negative emotions. Therefore, recognition of parents' emotional states and their ways of coping with it is becoming more and more important. The study group consisted of 154 parents of newborns and infants with cyanotic congenital heart disease, before and after cardiac surgery (partial or full). To assess parental negative emotions, the level of stress, and strategies of coping with it, standardized questionnaires, such as HADS-M, PSS-10, and COPE, were used. Stress levels in parents were high and associated with negative emotions (anxiety, depression, irritability), as well as the choice of non-constructive coping strategies, which was observed especially in younger parents. Assessing parents' stress levels and ways of coping with stress can improve family functioning and provide better development conditions for the child.

Robust multi-frequency band joint dictionary learning with low-rank representation

Emotional state recognition is an important part of emotional research. Compared to non-physiological signals, the electroencephalogram (EEG) signals can truly and objectively reflect a person’s emotional state. To explore the multi-frequency band emotional information and address the noise problem of EEG signals, this paper proposes a robust multi-frequency band joint dictionary learning with low-rank representation (RMBDLL). Based on the dictionary learning, the technologies of sparse and low-rank representation are jointly integrated to reveal the intrinsic connections and discriminative information of EEG multi-frequency band. RMBDLL consists of robust dictionary learning and intra-class/inter-class local constraint learning. In robust dictionary learning part, RMBDLL separates complex noise in EEG signals and establishes clean sub-dictionaries on each frequency band to improve the robustness of the model. In this case, different frequency data obtains the same encoding coefficients according to the consistency of emotional state recognition. In intra-class/inter-class local constraint learning part, RMBDLL introduces a regularization term composed of intra-class and inter-class local constraints, which are constructed from the local structural information of dictionary atoms, resulting in intra-class similarity and inter-class difference of EEG multi-frequency bands. The effectiveness of RMBDLL is verified on the SEED dataset with different noises. The experimental results show that the RMBDLL algorithm can maintain the discriminative local structure in the training samples and achieve good recognition performance on noisy EEG emotion datasets.

TER-CA-WGNN: Trimodel Emotion Recognition Using Cumulative Attribute-Weighted Graph Neural Network

Affective computing is a multidisciplinary field encompassing artificial intelligence, natural language processing, linguistics, computer science, and social sciences. This field aims to deepen our comprehension and capabilities by deploying inventive algorithms. This article presents a groundbreaking approach, the Cumulative Attribute-Weighted Graph Neural Network, which is innovatively designed to integrate trimodal textual, audio, and visual data from the two multimodal datasets. This method exemplifies its effectiveness in performing comprehensive multimodal sentiment analysis. Our methodology employs vocal inputs to generate speaker embeddings trimodal analysis. Using a weighted graph structure, our model facilitates the efficient integration of these diverse modalities. This approach underscores the interrelated aspects of various emotional indicators. The paper’s significant contribution is underscored by its experimental results. Our novel algorithm achieved impressive performance metrics on the CMU-MOSI dataset, with an accuracy of 94% and precision, recall, and F1-scores above 92% for Negative, Neutral, and Positive emotion categories. Similarly, on the IEMOCAP dataset, the algorithm demonstrated its robustness with an overall accuracy of 93%, where exceptionally high precision and recall were noted in the Neutral and Positive categories. These results mark a notable advancement over existing state-of-the-art models, illustrating the potential of our approach in enhancing Sentiment Recognition through the synergistic use of trimodal data. This study’s comprehensive analysis and significant results demonstrate the proposed algorithm’s effectiveness in nuanced emotional state recognition and pave the way for future advancements in affective computing, emphasizing the value of integrating multimodal data for improved accuracy and robustness.

Alexithymia and facial emotion recognition in patients with functional neurological disorder

ObjectivesPatients with functional neurological disorder (FND) are known to have difficulties recognizing and processing emotions. Problems recognizing internal emotional states (alexithymia) are common in FND, but little is known about recognizing emotions expressed by other people. This study investigates whether patients with FND have higher levels of alexithymia and reduced facial emotion recognition compared to healthy controls. MethodsPatients with FND (n = 31, mean age=42.7 [SD=14.8] years, 54.8% women) were compared to healthy controls (n = 33, mean age=45.1 [SD=16.2] years, 63.6% women). The Bermond-Vorst Alexithymia Questionnaire (BVAQ) was used for the assessment of alexithymia and the Ekman 60 Faces Test (EFT) for facial emotion recognition. ResultsPatients with FND had higher levels of alexithymia than healthy controls (BVAQ=71.8 [SD=19.8] versus 59.3 [SD=20.3], p = .02, Cohen’s d=0.62). Facial emotion recognition did not significantly differ between FND patients and controls (EFT total score FND: 46.1 [SD=5.9], Controls: 47.5 [SD=5.5], p = .34, Cohen’s d=0.24). Only recognition of surprise differed between patients and controls (FND: 8.4 [SD=1.8], Controls: 9.2 [SD=1.0), p = .03, Cohen’s d= 0.56). Higher levels of alexithymia were associated with poorer facial emotion recognition, but this relationship was not statistically significant (FND: β= −0.20, p = .28; Controls: β=-0.03; p = .87). ConclusionsThe current data confirm prior observations that patients with FND have higher alexithymia levels than controls without FND. Difficulties recognizing emotions among patients with FND primarily involves recognition of internal emotional states rather than recognition of facially expressed emotions by others. These findings require replication in a larger and more divers sample.

Combining Deep Learning with Signal-image Encoding for Multi-Modal Mental Wellbeing Classification

The quantification of emotional states is an important step to understanding wellbeing. Time series data from multiple modalities such as physiological and motion sensor data have proven to be integral for measuring and quantifying emotions. Monitoring emotional trajectories over long periods of time inherits some critical limitations in relation to the size of the training data. This shortcoming may hinder the development of reliable and accurate machine learning models. To address this problem, this article proposes a framework to tackle the limitation in performing emotional state recognition: (1) encoding time series data into coloured images; (2) leveraging pre-trained object recognition models to apply a Transfer Learning (TL) approach using the images from step 1; (3) utilising a 1D Convolutional Neural Network (CNN) to perform emotion classification from physiological data; (4) concatenating the pre-trained TL model with the 1D CNN. We demonstrate that model performance when inferring real-world wellbeing rated on a 5-point Likert scale can be enhanced using our framework, resulting in up to 98.5% accuracy, outperforming a conventional CNN by 4.5%. Subject-independent models using the same approach resulted in an average of 72.3% accuracy (SD 0.038). The proposed methodology helps improve performance and overcome problems with small training datasets.

Psychological Emotion Recognition Of Students Based Chatbot

In the context of modern education, where students' emotional well-being plays a crucial role in academic success, the integration of technology, particularly machine learning-based chatbots, presents a promising avenue to support and recognize the psychological states of students. This paper delves into the development and evaluation of a chatbot system designed to recognize and respond to the emotions expressed by students. Leveraging natural language processing and sentiment analysis, the chatbot engages in conversations with students, allowing for the real-time recognition of emotional states. Our study not only focuses on the technical aspects of emotion recognition but also explores the implications and ethical considerations of deploying such technology in educational settings. By shedding light on the potential of machine learning-based chatbots to enhance emotional support and understanding within educational environments, this research contributes to the ongoing dialogue on student well-being and the role of technology in education.

EEG-based affective computing in virtual reality with a balancing of the computational efficiency and recognition accuracy

The field of VR-EEG affective computing is rapidly progressing. However, it faces challenges such as lacking a solid psychological theory foundation, limited classification accuracy, and high computational costs. This study established a standardized VR video library to elicit emotions. Participants viewed positive, negative, and neutral VR videos while EEG data was collected. Grounded in the Affective Style Theory, this research proposes an emotion valence recognition strategy in VR that balances computational efficiency and classification accuracy through multidimensional complementary feature extraction from EEG signals, feature selection or dimensionality reduction coupled with classifiers, and optimal frequency band selection. The research findings indicate that multidimensional complementary feature extraction in frequency and spatial domains can enhance recognition performance. Notably, the theta frequency band features are pivotal in emotion valence recognition within VR environments. Strategies like PCA-RF and RBFNN outperform existing methods, achieving an average classification accuracy of up to 95.6% while maintaining computational efficiency. In terms of theoretical contributions, the study enhances our understanding of emotional perception consistency and variability under the Affective Style Theory, offering insights into individual emotional state recognition. In practical terms, it emphasizes efficiency-accuracy balance, making integrating VR-EEG affective computation technology into a broader range of applications feasible.

AN EMOTION RECOGNITION MODEL BASED ON LONG SHORT-TERM MEMORY NETWORKS AND EEG SIGNALS AND ITS APPLICATION IN PARAMETRIC DESIGN

One of the design objectives of a product is to create a positive emotional user experience. Through careful design, the product can evoke emotional resonance in users and stimulate their pleasure and satisfaction. Therefore, emotion recognition is crucial for parameterized product design. Considering that emotion recognition based on electroencephalogram (EEG) signals is more objective and accurate compared to methods such as text and surveys, this paper proposes an emotion analysis model based on long short-term memory (LSTM) and EEG and applies it to parameterized design. The main contributions of this paper are as follows. (1) Constructing a high-accuracy emotion recognition model. First, EEG data reflecting the characteristic patterns of brain activities in different emotional states are collected through EEG electrodes. Then, the EEG data are input into the LSTM network for training, enabling it to learn and capture the features associated with emotional states. During the training process, the model learns to extract crucial emotional features from the EEG data for emotion state recognition. This model can automatically learn emotional features, handle long-term dependencies and provide a more accurate and reliable solution for emotion recognition tasks. (2) Creating an EEG dataset specifically for evaluating emotions related to a product and using the trained emotion recognition model to classify this dataset, obtaining emotion classification results. The emotion classification results can be used to determine which parameter designs in product development need to be retained or discarded. These parameter designs can involve aspects such as user experience, functionality, aesthetics, usability and user-friendliness. Decisions can be made based on the emotion classification results to improve the quality and user satisfaction of the product.

A multilayer stacking method base on RFE-SHAP feature selection strategy for recognition of driver’s mental load and emotional state

The driver state monitoring is becoming one of the research hotspots in the field of traffic and vehicle safety, which can ensure driving safety by monitoring the driver’s state. Therefore, this work makes an attempt to recognize driver's mental load and emotional states. However, the reliability and accuracy of driver status detection largely depend on the extracted features and the detection algorithm. The existing methods mainly improve accuracy by increasing the number of features, but for the problem with limited training samples, the increase in features may lead to an increase in model variance, and resulting in overfitting. In addition, single-model based methods may fall into local optimal solutions when performing local search. To alleviate these, we firstly propose an RFE-SHAP algorithm that improves the recursive feature elimination algorithm by using the Shapley Additive exPlanning value, and paired it with the eXtreme Gradient Boosting (XGBoost) algorithm to screen out the subset of features that best represent the driver's mental load and emotional state. Secondly, we proposed a multilayer stacking ensemble learning method based on different optimal feature subsets to further improve the accuracy of the driver’s mental load and emotional state recognition. For each base models used for integration learning, the corresponding optimal feature subsets are selected using the RFE-SHAP algorithm, and then the different optimal feature subsets are combined with multilayer stacking ensemble learning. The validation via experimental data demonstrates that the XGBoost-RFE-SHAP algorithm achieves superior model performance with fewer feature combinations than the classic algorithm. The proposed driver state detection model based on multiple optimum feature subsets and multilayer stacking integration methods would more accurately identify the driver's mental load and emotional state than the single model, commonly used machine learning and integrated learning algorithms.

Recognition of facial emotions training in autistic children - Clinical trial study

Introduction. Disability to recognize emotional and mental states from other people’s facial expressions is acknowledged as one of the major developmental-behavioral problems in autistic children. Therefore, this study aimed to investigate the effect of traditional and computer emotional facial training on the recognition of facial emotional states in children with autism and mother-child relation and parental competence in mothers of autistic children. Material and Methods. The present was clinical trial study. The children in the both groups were trained during 10 sessions. In traditional group children training using 40 images related to four situations as happiness, sadness, anger, and fear. The children in the computer group trained with the Let’s Face It software. These questionnaires were completed one day before and one day and one week after performing the intervention. The data were analyzed using SPSS software. Results. Findings showed that emotional state training has improved the identification of emotional states (p = 0.01), mother-child relation (p = 0.001) and parental competence (p = 0.001) in the both groups. There was also a statistically significant difference between the scores of identifying face emotional states (p = 0.01), mother-child relation (p = 0.001) and parental competence (p = 0.001) in the both groups. But training in traditional group was effectiveness than computer grope. Conclusion. Teaching emotional face to children with autism improved the recognition of emotional states by these children, and subsequently improved the mother-child relationship and motherly sense of competence in the mothers of these children.

Research on Emotion Recognition Method of Flight Training Based on Multimodal Fusion

The emotional activities of the human body are mainly regulated by the autonomic nervous system, the central nervous system, and the advanced cognition of the human brain. This paper proposes an emotional state recognition method in pilot training tasks based on multimodal information fusion. A set of emotion perception recognition systems, a two-dimensional valence-arousal emotional model, and a multimodal information intelligent perception model were established based on the human-computer interaction mode during flight training; an intelligent perception system was designed to collect and intelligently perceive four kinds of peripheral physiological signals of pilots in real-time. And based on traditional machine learning models, a binary tree support vector machine was designed to optimize and improve the multimodal information co-integration decision model, which increased the accuracy of emotional state recognition in flight training by 37.58% on average. The experimental result showed that it realizes accurate monitoring and identification of real-time emotional state, helps to improve the effect of flight training and flight safety, maintains the efficiency of operation, and has important research significance and application prospects in the field of pilot training.

Quantitative ECG based emotion state recognition using Detrended Fluctuation Analysis

Abstract Wearable emotion recogniton system is essential in identifying mental health disorders by early detection and continuous monitoring of human emotions to provide proper treatment care. Electrocardiogram (ECG) signals can be used for emotion recognition for its noninvasiveness and easy usability. In this study, Detrended Fluctuation Analysis (DFA) and Extreme Gradient Boost (XG Boost) classifier is used to classify the scary and boring emotion from the ECG signals. For this, ECG signal corresponding to these emotions are obtained from public database. The preprocessing is performed by adding the video IDs to the signal and annotating it. This preprocessed signal is subjected to DFA to understand the power-law correlations and similarity property. Further, from the power law correlations, features namely Hurst exponent and DFA intercept are extracted. These features are subjected to XG Boost classifier to differentiate the two emotions. Results shows that the log-log plot of power law correlation is linear in nature which indicates that ECG signals of both the emotions have long range correlations and self-similarity property. The extracted scaling exponent indicates variations between scary and boring with a mean and standard deviation of 0.81±0.13 and 0.68±0.07 respectively. Similarly, DFA intercept provides mean and standard deviation for scary and boring 0.15±0.06 and 0.17±0.07 respectively, showing less variability in the ECG signal. XG Boost classifier gives accuracy of 80.5% for classifying scary and boring emotion. Thus, the proposed approach can be used for wearable emotion recognition system to differentiate scary and boring emotion.

EMOTION RECOGNITION FOR MENTAL HEALTH PREDICTION USING AI TECHNIQUES: AN OVERVIEW

Human facial expressions are a mirror of human thoughts, feelings and human mental states. Facial Emotion Recognition (FER) can provide a social advantage. It's like a form of silent communication. Emotion recognition technology will help to automatically detect the patient's emotions during illness and avoid external acts such as suicide, mental disorders or mental health problems. If we understand all the signs of emotions, we can solve many problems for human beings. Emotion recognition and detection is also useful for healthcare. Through emotional state recognition, we can get information about patients. Recognizing a patient's emotions for a specific disease using artificial intelligence techniques is a challenging task. This article presents recognition, detection and methods for mental health patients. Using artificial intelligence techniques with an emotion detection library and matching emotions to mental health. This article uses an emotional scale to show that there is a link between negative emotions and mental health problems. In this paper, she provided a comprehensive review of AI-based FER methodology, including datasets, feature extraction techniques, algorithms, and recent breakthroughs with their applications in facial expression recognition. In the future, all aspects of FER for different ages would significantly influence the health research community.

Practical hyperparameters tuning of convolutional neural networks for EEG emotional features classification

EEG-based emotional states recognition is an active research direction, rushed up by recent emerging applications of intelligent systems and smart environments. However, accurate EEG-based emotion recognition is still challenging even with the use of the most efficient recent approaches like deep learning. Whereas many studies showed that convolutional neural networks can achieve good results in this field, they did not offer a general guide to subsequent researchers in designing dedicated networks. For practical deployment of deep neural networks in emotion-aware systems, an efficient design method, in terms of hyperparameters selection facilities, is needed. This paper investigates the simplification of the design process of a convolutional neural network applied to a binary and subject-dependent emotional valence classification. The proposed approach is based on two key solutions which are network architecture modularity and procedural tuning method. The proposed tuning approach estimates, in a given search space, the hyperparameter tunability defined as its main effect on the classification accuracy. The search space is depicted by selected hyperparameters levels and a fractional orthogonal array according to the Taguchi robust Design of Experiments method. In this investigation, we include twelve hyperparameters related to the neural network architecture and the training process. The significance of the estimated hyperparameter tunability is validated by statistical variance analysis. Experimental results allowed the determination of the best performance hyperparameters combination, confirmed the well-established effect of some training hyperparameters and revealed the importance of the pooling layer type in our case.

SMG: A Micro-gesture Dataset Towards Spontaneous Body Gestures for Emotional Stress State Analysis

We explore using body gestures for hidden emotional state analysis. As an important non-verbal communicative fashion, human body gestures are capable of conveying emotional information during social communication. In previous works, efforts have been made mainly on facial expressions, speech, or expressive body gestures to interpret classical expressive emotions. Differently, we focus on a specific group of body gestures, called micro-gestures (MGs), used in the psychology research field to interpret inner human feelings. MGs are subtle and spontaneous body movements that are proven, together with micro-expressions, to be more reliable than normal facial expressions for conveying hidden emotional information. In this work, a comprehensive study of MGs is presented from the computer vision aspect, including a novel spontaneous micro-gesture (SMG) dataset with two emotional stress states and a comprehensive statistical analysis indicating the correlations between MGs and emotional states. Novel frameworks are further presented together with various state-of-the-art methods as benchmarks for automatic classification, online recognition of MGs, and emotional stress state recognition. The dataset and methods presented could inspire a new way of utilizing body gestures for human emotion understanding and bring a new direction to the emotion AI community. The source code and dataset are made available: https://github.com/mikecheninoulu/SMG.

Preliminary data on the development of emotion vocabulary in typically developing children (5-13 years) using an experimental psycholinguistic measure.

Vocabulary of emotion is integral to emotional development and emotional intelligence is associated with improved mental health outcomes. Many language disordered groups experience emotional difficulties; Developmental Language Disorders, Attention Deficit Hyperactivity Disorder and autism. However, (as in the case of autism) research tends to focus on assessing recognition of emotional states, rather than exploring labeling skills. Where labeling is assessed, measures have focused on early-acquired vocabulary (happy, sad, angry) or self/parent reporting. To date, no objective assessment has been made of vocabulary of emotion across childhood. This study uses an experimental psycholinguistic measure, The Emotion Vocabulary: Expressive and Receptive ability measure (EVER) which includes two tasks (receptive vocabulary and word generation/expressive vocabulary). This measure has capacity to demonstrate vocabulary growth across age groups. 171 participants (5.0-13.11 years) completed The EVER Measure, alongside two closely matched standardized measures of basic language: BPVS (receptive vocabulary task) and CELF (word-association task). Assessments were completed online and en vivo (COVID testing restrictions dependent). As predicted, children's accuracy increased on both receptive and expressive emotion vocabulary tasks, in line with age at time of testing. EVER scores were significantly predicted by age and correlated with matched basic language scores. Secondary analysis provided preliminary findings on age of acquisition for specific emotion vocabulary items. The findings consequently demonstrate proof of concept for the use of The EVER Measure in assessing emotional vocabulary across childhood. This study provides important preliminary data on generating and recognizing emotion labels across typical child development. Critically, it extends current knowledge on emotion vocabulary acquisition into middle childhood, where linguistic ability is relatively mature. As such, findings have implications for research with potential clinical application in the assessment of older children, with either language or emotional differences or both. Findings demonstrate the need for a standardized tool, and its potential application in research and clinical practice is explored. A large-scale study offering proof of concept and reliability of The EVER Measure is indicated.

A Hybrid EEG-based Emotion Recognition Approach Using Wavelet Convolutional Neural Networks and Support Vector Machine.

Nowadays, deep learning and convolutional neural networks (CNNs) have become widespread tools in many biomedical engineering studies. CNN is an end-to-end tool, which makes the processing procedure integrated, but in some situations, this processing tool requires to be fused with machine learning methods to be more accurate. In this paper, a hybrid approach based on deep features extracted from wavelet CNNs (WCNNs) weighted layers and multiclass support vector machine (MSVM) was proposed to improve the recognition of emotional states from electroencephalogram (EEG) signals. First, EEG signals were preprocessed and converted to Time-Frequency (T-F) color representation or scalogram using the continuous wavelet transform (CWT) method. Then, scalograms were fed into four popular pre-trained CNNs, AlexNet, ResNet-18, VGG-19, and Inception-v3 to fine-tune them. Then, the best feature layer from each one was used as input to the MSVM method to classify four quarters of the valence-arousal model. Finally, the subject-independent leave-one-subject-out criterion was used to evaluate the proposed method on DEAP and MAHNOB-HCI databases. Results showed that extracting deep features from the earlier convolutional layer of ResNet-18 (Res2a) and classifying using the MSVM increased the average accuracy, precision, and recall by about 20% and 12% for MAHNOB-HCI and DEAP databases, respectively. Also, combining scalograms from four regions of pre-frontal, frontal, parietal, and parietal-occipital and two regions of frontal and parietal achieved the higher average accuracy of 77.47% and 87.45% for MAHNOB-HCI and DEAP databases, respectively. Combining CNN and MSVM increased the recognition of emotion from EEG signals and the results were comparable to state-of-the art studies.

A MODIFIED FUZZY SUPPORT VECTOR MACHINE CLASSIFICATION-BASED APPROACH FOR EMOTIONAL RECOGNITION USING PHYSIOLOGICAL SIGNALS

Emotional state recognition has become an essential topic for human–robot interaction researches that diverted and covers a wide range of topics. By specifying emotional expressions, robots can identify the significant variables of human behavior and apply them to communicate in a very human-like fashion and develop interaction possibilities. The multimodality and spontaneity nature of human emotions make them hard to be recognized by robots. Each modality has its advantages and limitations, which, along with the unstructured behavior of spontaneous facial expressions, make several challenges for the proposed approaches in the literature. The most important of these approaches is based on a combination of explicit feature extraction methods and manual modality. This paper proposes a modified fuzzy support vector machine (FSVM) classification-based approach for emotional recognition using physiological signals. The main contribution of this study includes applying various data extraction indices and proper kernels for the FSVM classification method and evaluating the signal's richness in experimental tests. The developed emotional recognition method is also compared with conventional SVM and other existing state-of-the-art emotional recognition algorithms. The comparison results show an improved accuracy of the developed method over other approaches.

Explainable automated recognition of emotional states from canine facial expressions: the case of positive anticipation and frustration

In animal research, automation of affective states recognition has so far mainly addressed pain in a few species. Emotional states remain uncharted territories, especially in dogs, due to the complexity of their facial morphology and expressions. This study contributes to fill this gap in two aspects. First, it is the first to address dog emotional states using a dataset obtained in a controlled experimental setting, including videos from (n = 29) Labrador Retrievers assumed to be in two experimentally induced emotional states: negative (frustration) and positive (anticipation). The dogs’ facial expressions were measured using the Dogs Facial Action Coding System (DogFACS). Two different approaches are compared in relation to our aim: (1) a DogFACS-based approach with a two-step pipeline consisting of (i) a DogFACS variable detector and (ii) a positive/negative state Decision Tree classifier; (2) An approach using deep learning techniques with no intermediate representation. The approaches reach accuracy of above 71% and 89%, respectively, with the deep learning approach performing better. Secondly, this study is also the first to study explainability of AI models in the context of emotion in animals. The DogFACS-based approach provides decision trees, that is a mathematical representation which reflects previous findings by human experts in relation to certain facial expressions (DogFACS variables) being correlates of specific emotional states. The deep learning approach offers a different, visual form of explainability in the form of heatmaps reflecting regions of focus of the network’s attention, which in some cases show focus clearly related to the nature of particular DogFACS variables. These heatmaps may hold the key to novel insights on the sensitivity of the network to nuanced pixel patterns reflecting information invisible to the human eye.

Single Emotion Recognition from Multiple Faces with Fuzzy System

Emotional inference by recognizing facial expressions in an image or video has been actively conducted for several years, for example, visual emotion recognition and emotional state recognition. The limitations of illumination conditions, variations of head poses, and partial occlusion are the most challenging problems in image processing. Furthermore, recognizing and inferring insignificant objects in the background is complex. These factors degrade performance. We collected evidence from human sensing through a psychological approach rather than a numerical approach, and converted it into a language through fuzzy theory. This can not only be intuitively explained, but it is also possible to evaluate and analyze numerically. The predicted inference accuracy increased by approximately 9.7% compared with the simple average method.