Automatic Emotion Research Articles

Impact of self adaptive-elephant herding optimization towards neural network for facial emotion recognition

FACIAL expression is one of the most efficient, universal and fundamental indicators to identify their emotions and intentions in humans. Various experiments have already been performed on automatic Facial Emotion Recognition (FER) owing to useful significance in medical diagnosis, stress monitoring for drivers, sociable robots, and other human-computer interface devices. Here, this proposed framework consists of two processes namely; “(i) proposed feature extraction and (ii) classification”. Here, a major novelty relies in the initial phase (i.e. feature extraction phase), where the Proposed Local Vector Pattern (Proposed- LVP) based features are extracted. In addition to the proposed-LVP, the other Discrete Wavelet Transform (DWT) and Gray Level Co-occurrence Matrix (GLCM) based features are also extracted. Besides, the Principal Component Analysis (PCA) method is used for reducing the dimension of the features. Further, they are subjected to classification process, where Optimized Neural Network (NN) is used. More particularly, a new Improved Elephant Herding Optimization (EHO) model termed as Self Adaptive-EHO (SA-EHO) is used to train the NN model via selecting the optimal weights. At last, the proposed work performance is computed over the other traditional systems with respect to the positive measures like “accuracy, sensitivity, specificity and precision”; negative measures like “False Positive Rate (FPR), False Negative Rate (FNR) and False Discovery Rate (FDR)”; other measures like “Negative Predictive Value (NPV), F1-score and Matthew’s Correlation Coefficient (MCC)”, respectively.

Facial Feature Extraction for Emotion Classification Using Fuzzy C-Mean Clustering

Background: Automatic human emotion recognition system is an active area of research due to its wide applications in the field of Human Computer Interaction(HCI) systems, driver fatigue monitoring systems, surveillance systems, human assistance systems, smile detectors etc. Objective: The study presents a fuzzy based approach to extract facial features from input image and builds different classification models to classify the image into two emotion classes i.e. happy and neutral. The system has potential implications in the field of smile detection systems, customer experience analysis and patient monitoring systems. Methods: The proposed system determines the dimensional attributes (l-attribute and w-attribute) of mouth region extracted from the facial image using viola-jones algortithm. The feature set is generated by using a total of 136 images from JAFFE, NimStim and MUG dataset. The differentiating power of the attribures is then evaluated using five different classification models. Results: The accuracy, precision and recall is determined for each classification model. The results show good accuracy of 70% for the grayscale JAFFE and NimStim databases and 95% for the coloured MUG database. Conclusion: The mouth features calculated in the study are based on the geometric coordinates which eliminates the possibility of false distance measurements due to presence of noise or shadows.

Automatic voice emotion recognition of child-parent conversations in natural settings

ABSTRACT While voice communication of emotion has been researched for decades, the accuracy of automatic voice emotion recognition (AVER) is yet to improve. In particular, the intergenerational communication has been under-researched, as indicated by the lack of an emotion corpus on child–parent conversations. In this paper, we presented our work of applying Support-Vector Machines (SVMs), established machine learning models, to analyze 20 pairs of child–parent dialogues on everyday life scenarios. Among many issues facing the emerging work of AVER, we explored two critical ones: the methodological issue of optimising its performance against computational costs, and the conceptual issue on the state of emotionally neutral. We used the minimalistic/extended acoustic feature set extracted with OpenSMILE and a small/large set of annotated utterances for building models, and analyzed the prevalence of the class neutral. Results indicated that the bigger the combined sets, the better the training outcomes. Regardless, the classification models yielded modest average recall when applied to the child–parent data, indicating their low generalizability. Implications for improving AVER and its potential uses are drawn.

Emotions in Online Reviews and Product Sales: Unifying Empirical and Theoretical Perspectives

Emotion artificial intelligence, the algorithm that recognizes and interprets various human emotions beyond valence (positive and negative polarity), is still in its infancy yet attracts much attention from both the industry and the academia. Based on the discrete emotion theory and statistical language modeling, this work proposes an algorithm to enable automatic domain-adaptive emotion lexicon construction and multi-dimensional emotion detection in texts. With a large-scale dataset of China’s movie market from 2012 to 2018, we construct and validate a domain-specific emotion lexicon and demonstrate the predictive power of eight discrete emotions (i.e., surprise, joy, anticipation, love, anxiety, sadness, anger, and disgust) in online reviews on box office. Further, we find that representing emotions using discrete emotions yields higher prediction accuracy than using valence or latent emotion variables generated by topic modeling. To understand the source of the predictive power from a theoretical perspective, and to test the cross-culture generalizability of our prediction study, we further conduct an experiment in the U.S. movie market, based on the “feelings as information” theory and theoretical research on emotion, judgement, and decision making. We find that mediated by processing fluency, discrete emotions significantly affect the perceived review helpfulness, which further influences purchase intention. Our work shows the economic value of emotions in online reviews, generates insight into the mechanism of their effects, and has managerial implications for online review platform design, movie marketing, and cinema operations.

Live human-robot interactive public demonstrations with automatic emotion and personality prediction.

Communication with humans is a multi-faceted phenomenon where the emotions, personality and non-verbal behaviours, as well as the verbal behaviours, play a significant role, and human-robot interaction (HRI) technologies should respect this complexity to achieve efficient and seamless communication. In this paper, we describe the design and execution of five public demonstrations made with two HRI systems that aimed at automatically sensing and analysing human participants' non-verbal behaviour and predicting their facial action units, facial expressions and personality in real time while they interacted with a small humanoid robot. We describe an overview of the challenges faced together with the lessons learned from those demonstrations in order to better inform the science and engineering fields to design and build better robots with more purposeful interaction capabilities. This article is part of the theme issue 'From social brains to social robots: applying neurocognitive insights to human-robot interaction'.

Automatic emotion and attention analysis of young children at home: a ResearchKit autism feasibility study

Current tools for objectively measuring young children’s observed behaviors are expensive, time-consuming, and require extensive training and professional administration. The lack of scalable, reliable, and validated tools impacts access to evidence-based knowledge and limits our capacity to collect population-level data in non-clinical settings. To address this gap, we developed mobile technology to collect videos of young children while they watched movies designed to elicit autism-related behaviors and then used automatic behavioral coding of these videos to quantify children’s emotions and behaviors. We present results from our iPhone study Autism & Beyond, built on ResearchKit’s open-source platform. The entire study—from an e-Consent process to stimuli presentation and data collection—was conducted within an iPhone-based app available in the Apple Store. Over 1 year, 1756 families with children aged 12–72 months old participated in the study, completing 5618 caregiver-reported surveys and uploading 4441 videos recorded in the child’s natural settings. Usable data were collected on 87.6% of the uploaded videos. Automatic coding identified significant differences in emotion and attention by age, sex, and autism risk status. This study demonstrates the acceptability of an app-based tool to caregivers, their willingness to upload videos of their children, the feasibility of caregiver-collected data in the home, and the application of automatic behavioral encoding to quantify emotions and attention variables that are clinically meaningful and may be refined to screen children for autism and developmental disorders outside of clinical settings. This technology has the potential to transform how we screen and monitor children’s development.

Secrets of a smile? Your gender and perhaps your biometric identity

With its numerous applications, automatic facial emotion recognition has recently become a very active area of research. Yet there has been little detailed study of the dynamic components of facial expressions. This article reviews research that shows gender is encoded in the dynamics of a smile, and how it may be possible to use the dynamic components of facial expressions as a form of biometric.

Discrete and continuous emotion recognition using sequence kernels

The field of automatic speech emotion recognition is a highly active and multi-diverse research area. The current state-of-the-art approach in machine analysis of human emotion has focused on recognition of discrete emotional states, such as the six basic emotion categories. However, emotion is deemed complex and is characterised in terms of latent dimensions. Accordingly, this paper aims at recognising discrete and continuous emotional states by adapting the emotional recognition system to the advanced kernel-based machine learning algorithms from the field of speaker recognition, we argue that it is more efficient in terms of recognition performance. The focus in this paper is to build a range of sequence kernel to recognise discrete and continuous emotions from the well-established real-life speech dataset (IEMOCAP) and the acted Berlin emotional speech dataset (Emo-DB).

The Mediating Role of Automatic Thought and Cognitive Emotion Regulation Strategies on Relationships among Childhood Abuse/Neglect, Behavioral Problems and Resilience of Adolescents

The purpose of this research is to examine the mediating role of automatic thoughts and cognitive emotion regulations strategies on the relationship among childhood maltreatment experiences, behavioral problems and resilience of adolescents. The sample of the study has consisted of 671 adolescents (338-girl, 322-boy). Childhood Trauma Questionnaire Short Form, Youth Self Report, Children’s Automatic Thoughts Scale, Resiliency Scale and Cognitive Emotion Regulation Questionnaire have been used. Research findings have shown that adaptive cognitive emotion regulation strategies and automatic thoughts have entire mediator role in relation between adolescents’ childhood abuse/neglect experience and resilience. However maladaptive cognitive emotion regulation strategies and automatic thoughts have partial mediating role in the relation between adolescents’ childhood abuse/neglect experience and behavioral problems. On the other hand maladaptive cognitive emotion regulation strategies have no mediating effect on relationship between adolescents’ childhood abuse/neglect experience and resilience levels and adaptive cognitive emotion regulation strategies have no mediating effect on relationship between adolescents’ childhood abuse/neglect experience and behavioral problems. In explaining the variance of behavioral problems, it has been found that the most effective variables are respectively the automatic thoughts, childhood abuse / neglect experiences and maladaptive cognitive emotion regulation strategies; adaptive cognitive emotion regulation strategies and automatic thoughts are respectively the most important variables to explain variance of resilience.

Emotional and Developmental Repair through Psychodrama: Floortime for Grown-Ups

Emotional and Developmental Repair through Psychodrama: Floortime for Grown-Ups

Automatic Speech Emotion and Speaker Recognition Based on Hybrid GMM and FFBNN

Automatic Speech Emotion and Speaker Recognition Based on Hybrid GMM and FFBNN

Affective-aware tutoring platform for interactive digital television

Interactive Digital TeleVision (IDTV) is emerging as a potentially important medium for learning at home. This paper presents a novel affective-aware tutoring platform for IDTV which makes use of automatic facial emotion recognition to improve the tutor-student relationship. The system goes further than simply broadcasting an interactive educational application by allowing the personalization of the course content. The tutor can easily access academic information relating to the students and also emotional information captured from learners' facial expressions. In this way, depending on their academic and affective progress, the tutor can send personal messages or extra educational contents for improving students' learning. In order to include these features it was necessary to address some important technical challenges derived from IDTV hardware and software restrictions. The system has been successfully tested with real students and tutors in a non-laboratory environment. Our system tries to advance in the challenge of providing to distance learning systems with the perceptual abilities of human teachers with the final aim of improving students learning experience and outcome. Nevertheless, there is still relatively little understanding of the impact of affect on students' behaviour and learning and of the dynamics of affect during learning with software. Systems like ours would make it possible to attack these relevant open questions.

Fear-Conditioning Mechanisms Associated with Trait Vulnerability to Anxiety in Humans

SummaryInvestigations of fear conditioning in rodents and humans have illuminated the neural mechanisms underlying cued and contextual fear. A critical question is how personality dimensions such as trait anxiety act through these mechanisms to confer vulnerability to anxiety disorders, and whether humans' ability to overcome acquired fears depends on regulatory skills not characterized in animal models. In a neuroimaging study of fear conditioning in humans, we found evidence for two independent dimensions of neurocognitive function associated with trait vulnerability to anxiety. The first entailed increased amygdala responsivity to phasic fear cues. The second involved impoverished ventral prefrontal cortical (vPFC) recruitment to downregulate both cued and contextual fear prior to omission (extinction) of the aversive unconditioned stimulus. These two dimensions may contribute to symptomatology differences across anxiety disorders; the amygdala mechanism affecting the development of phobic fear and the frontal mechanism influencing the maintenance of both specific fears and generalized anxiety.

An integrated music video browsing system for personalized television

In this paper, we propose an integrated music video browsing system for personalized digital television. The system has the functions of automatic music emotion classification, automatic theme-based music classification, salient region detection, and shot classification. From audio (music) tracks, highlight detection and emotion classification are performed on the basis of information on temporal energy, timbre and tempo. For video tracks, shot detection is fulfilled to classify shots into face shots and color-based shots. Lastly automatic grouping of themes is executed on music titles and their lyrics. With a database of international music videos, we evaluate the performance of each function implemented in this paper. The experimental results show that the music browsing system achieves remarkable performances. Thus, our system can be adopted in any digital television for providing personalized services.

Interaction between automatic attention and emotion in Attention Deficit Hyperactivity Disorder: an event-related potential study.

Interaction between automatic attention and emotion in Attention Deficit Hyperactivity Disorder: an event-related potential study.

Why Does Fear Override Hope in Societies Engulfed by Intractable Conflict, as It Does in the Israeli Society?

The question of why fear overrides hope in societies embarked on the road of peacemaking after years of intractable conflict is answered on the basis of accumulated knowledge in the psychology and sociology of emotions. This knowledge suggests that fear is an automatic emotion, grounded in the perceived present and often based on the memorized past (also processed unconsciously), that leads to freezing of beliefs, conservatism, and sometimes preemptive aggression. Hope, in contrast, involves mostly cognitive activity, which requires the search for new ideas and thus is based on creativity and flexibility. Because hope is based on thinking, it can be seriously impeded by the spontaneous and unconscious interference of fear. Both fear and hope can become collective emotional orientations that organize society’s views and direct its forms of action. It is assumed that societies involved in intractable conflict are dominated by a collective emotional orientation of fear, which is functional in their coping with the stressful and demanding situation. But such an orientation serves as a psychological obstacle to a peace process once it starts. The Israeli Jewish case of collective fear orientation is offered as an example. The presentation includes the roots of this orientation, the ways in which it is reflected and disseminated, and its expressions among the Israeli Jewish public; it ends optimistically with the suggestion that societies can determine to overcome their fear and establish a collective orientation of hope for peace.