Non-basic Emotions Research Articles

Are all emotions social? Embracing a pluralistic understanding of social emotions

While the importance of social emotions is widely recognized, the question whether all emotions are social and what this would mean for the category ‘social emotions’ is yet to be addressed systematically. Emotion theorists and researchers so far have proposed different candidates for social emotions. These include non-basic emotions, self-conscious emotions, higher-cognitive emotions, and defining social emotions via their social functions. This paper looks at these different candidates for social emotions and briefly discusses their issues. Discussing the candidates and their issues motivates embracing a pluralistic approach to social emotions. In a further step, the paper will look at approaches exploring how social factors impact emotions in general. This serves as a basis for explaining what it would mean to propose that all emotions are social. After reviewing different candidates for social emotions and looking at the impact of social factors on emotions in general, the paper will propose that a pluralistic understanding of social emotions is needed to embrace the different ways in which social emotions may function. Embracing a pluralistic understanding of social emotions does not mean, however, that all emotions are social emotions.

Combining GAN with reverse correlation to construct personalized facial expressions.

Recent deep-learning techniques have made it possible to manipulate facial expressions in digital photographs or videos, however, these techniques still lack fine and personalized ways to control their creation. Moreover, current technologies are highly dependent on large labeled databases, which limits the range and complexity of expressions that can be modeled. Thus, these technologies cannot deal with non-basic emotions. In this paper, we propose a novel interdisciplinary approach combining the Generative Adversarial Network (GAN) with a technique inspired by cognitive sciences, psychophysical reverse correlation. Reverse correlation is a data-driven method able to extract an observer's 'mental representation' of what a given facial expression should look like. Our approach can generate 1) personalized facial expression prototypes, 2) of basic emotions, and non-basic emotions that are not available in existing databases, and 3) without the need for expertise. Personalized prototypes obtained with reverse correlation can then be applied to manipulate facial expressions. In addition, our system challenges the universality of facial expression prototypes by proposing the concepts of dominant and complementary action units to describe facial expression prototypes. The evaluations we conducted on a limited number of emotions validate the effectiveness of our proposed method. The code is available at https://github.com/yansen0508/Mental-Deep-Reverse-Engineering.

The Taste of Emotion: Metaphoric Association Between Taste Words and Emotion/Emotion-Laden Words.

According to the Conceptual Metaphor Theory, abstract concepts can be metaphorically associated with more concrete, physically embodied concepts, such as gustatory experience. Studies on taste–emotion metaphoric association reported that people associate love with sweet, jealousy with sour and bitter, and sadness with bitter. However, few studies have systematically examined the metaphoric association between taste and words referred to emotion (e.g., “sad”) or emotion-laden concepts (e.g., “funeral”). In the current four studies (total N = 357), we examined this metaphoric association by having participants come up with a taste word when reading an emotion and emotion-laden word (Study 1—explicit association of taste words-to-emotion/emotion-laden words), come up with an emotion word when reading taste words (Study 2—explicit association of emotion words-to-taste words), rate the association between taste words and basic or non-basic emotion words (Study 3), and rate the association between taste words and a more expanded pool of emotion/emotion-laden words (Study 4). Results showed that sweet was mostly associated with positive emotion and emotion-laden words, whereas bitter, followed by sour and spicy, was mostly associated with negative emotion and emotion-laden words. The bidirectionality of taste–emotion metaphoric association was supported by our dataset. The implications of these findings on the Conceptual Metaphor Theory and embodied cognition are discussed.

Distributed affective space represents multiple emotion categories across the human brain.

The functional organization of human emotion systems as well as their neuroanatomical basis and segregation in the brain remains unresolved. Here, we used pattern classification and hierarchical clustering to characterize the organization of a wide array of emotion categories in the human brain. We induced 14 emotions (6 ‘basic’, e.g. fear and anger; and 8 ‘non-basic’, e.g. shame and gratitude) and a neutral state using guided mental imagery while participants' brain activity was measured with functional magnetic resonance imaging (fMRI). Twelve out of 14 emotions could be reliably classified from the haemodynamic signals. All emotions engaged a multitude of brain areas, primarily in midline cortices including anterior and posterior cingulate gyri and precuneus, in subcortical regions, and in motor regions including cerebellum and premotor cortex. Similarity of subjective emotional experiences was associated with similarity of the corresponding neural activation patterns. We conclude that different basic and non-basic emotions have distinguishable neural bases characterized by specific, distributed activation patterns in widespread cortical and subcortical circuits. Regionally differentiated engagement of these circuits defines the unique neural activity pattern and the corresponding subjective feeling associated with each emotion.

Direct Social Perception of Emotions in Close Relations

Drawing on a pluralist approach to mindreading, I explore Direct Social Perception with respect to perceiving the emotional states of people that we are close to, such as spouses, friends, and family. I argue that in general, emotions are embodied and can be perceived directly. I further claim that perceptual content includes concepts. That is, I argue against a non-conceptual view of emotion recognition, claiming instead that we learn emotional concepts by attending to certain expressive patterns of emotions. This view implicates that we can directly perceive both basic and non-basic emotions of people we are close to.

Commentary: Rethinking the Development of "Nonbasic" Emotions: A Critical Review of Existing Theories.

Commentary: Rethinking the Development of "Nonbasic" Emotions: A Critical Review of Existing Theories.

Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition.

Automatic affect analysis has attracted great interest in various contexts including the recognition of action units and basic or non-basic emotions. In spite of major efforts, there are several open questions on what the important cues to interpret facial expressions are and how to encode them. In this paper, we review the progress across a range of affect recognition applications to shed light on these fundamental questions. We analyse the state-of-the-art solutions by decomposing their pipelines into fundamental components, namely face registration, representation, dimensionality reduction and recognition. We discuss the role of these components and highlight the models and new trends that are followed in their design. Moreover, we provide a comprehensive analysis of facial representations by uncovering their advantages and limitations; we elaborate on the type of information they encode and discuss how they deal with the key challenges of illumination variations, registration errors, head-pose variations, occlusions, and identity bias. This survey allows us to identify open issues and to define future directions for designing real-world affect recognition systems.

A Review and Meta-Analysis of Multimodal Affect Detection Systems

Affect detection is an important pattern recognition problem that has inspired researchers from several areas. The field is in need of a systematic review due to the recent influx of Multimodal (MM) affect detection systems that differ in several respects and sometimes yield incompatible results. This article provides such a survey via a quantitative review and meta-analysis of 90 peer-reviewed MM systems. The review indicated that the state of the art mainly consists of person-dependent models (62.2% of systems) that fuse audio and visual (55.6%) information to detect acted (52.2%) expressions of basic emotions and simple dimensions of arousal and valence (64.5%) with feature- (38.9%) and decision-level (35.6%) fusion techniques. However, there were also person-independent systems that considered additional modalities to detect nonbasic emotions and complex dimensions using model-level fusion techniques. The meta-analysis revealed that MM systems were consistently (85% of systems) more accurate than their best unimodal counterparts, with an average improvement of 9.83% (median of 6.60%). However, improvements were three times lower when systems were trained on natural (4.59%) versus acted data (12.7%). Importantly, MM accuracy could be accurately predicted (cross-validated R 2 of 0.803) from unimodal accuracies and two system-level factors. Theoretical and applied implications and recommendations are discussed.

Emotion categorization of body expressions in narrative scenarios.

Humans can recognize emotions expressed through body motion with high accuracy even when the stimuli are impoverished. However, most of the research on body motion has relied on exaggerated displays of emotions. In this paper we present two experiments where we investigated whether emotional body expressions could be recognized when they were recorded during natural narration. Our actors were free to use their entire body, face, and voice to express emotions, but our resulting visual stimuli used only the upper body motion trajectories in the form of animated stick figures. Observers were asked to perform an emotion recognition task on short motion sequences using a large and balanced set of emotions (amusement, joy, pride, relief, surprise, anger, disgust, fear, sadness, shame, and neutral). Even with only upper body motion available, our results show recognition accuracy significantly above chance level and high consistency rates among observers. In our first experiment, that used more classic emotion induction setup, all emotions were well recognized. In the second study that employed narrations, four basic emotion categories (joy, anger, fear, and sadness), three non-basic emotion categories (amusement, pride, and shame) and the “neutral” category were recognized above chance. Interestingly, especially in the second experiment, observers showed a bias toward anger when recognizing the motion sequences for emotions. We discovered that similarities between motion sequences across the emotions along such properties as mean motion speed, number of peaks in the motion trajectory and mean motion span can explain a large percent of the variation in observers' responses. Overall, our results show that upper body motion is informative for emotion recognition in narrative scenarios.

Music, Affect, Method, Data: Reflections on the Carroll Versus Kivy Debate

The comprehensive exchange between Noel Carroll and Peter Kivy, which took place in 2007, addressed key issues in the relationship between music and affect. More than in any prior philosophical debate on this topic, experimental psychologists' methods and data played a significant role. However, to a nontrivial extent, the findings-perhaps especially the dubious-were misconstrued or misused, usually without acknowledging the existence of contrary data-based opinion within the psychology of music itself. Therefore, one objective of the present article is to identify the specific problematic features and shed light on the broader context shared by the two disciplines. A complementary goal is to examine contributions to philosophers' transgressions by music psychologists' insufficiently conscientious reporting, frequent overgeneralizations, and unawareness of philosophers' critical arguments. Another objective is to examine the current status of key concepts-the relevant music, basic emotions, mood, expression, induction, movement and dance, and methods (including introspection and experimental procedures)-thus perhaps enabling the discussion of music and affect to proceed with fewer misunderstandings. Finally, the article moves beyond the initial debate and builds on a remarkable agreement of philosophical and psychological opinion on a key issue (the induction of non-basic emotions by absolute music) to reach a new conceptual ground.

Cross-linguistic emotion recognition: Dutch, Korean, and American English

This study investigates the occurrence of asymmetries in cross-linguistic recognition of emotion in speech. Theories on emotion recognition do not address asymmetries in the cross-linguistic recognition of emotion. To study perceptual asymmetries, a fully crossed design was used, with speakers and listeners from two typologically unrelated languages, Dutch and Korean. Additionally, listeners of American English, typologically close to Dutch but not Korean, were tested. Eight emotions, balanced in valence (positive-negative), arousal (active-passive), and basic vs. non-basic emotions -properties that are known to affect emotion recognition- were recorded by eight Dutch and eight Korean professional actors, in a nonsense phrase that was phonologically legal in both languages (and English). Stimuli were selected on the basis of prior validation studies with Dutch and Korean listeners. 28 Dutch, 24 Korean, and 26 American participants were presented with all 256 Dutch and Korean stimuli, blocked by language. Participants indicated for each stimulus which emotion it expressed by clicking on one of the eight emotions or “neutral”. Results showed strong asymmetries across languages and listener groups that cannot be explained along previously described dimensions (valence, arousal, basic-non-basic). The present results call for the extension of theories of cross-linguistic emotion recognition to incorporate asymmetrical perception patterns.

Three Time Scales of Neural Self-Organization Underlying Basic and Nonbasic Emotions

Our model integrates the nativist assumption of prespecified neural structures underpinning basic emotions with the constructionist view that emotions are assembled from psychological constituents. From a dynamic systems perspective, the nervous system self-organizes in different ways at different time scales, in relation to functions served by emotions. At the evolutionary scale, brain parts and their connections are specified by selective pressures. At the scale of development, connectivity is revised through synaptic shaping. At the scale of real time, temporary networks of synchronized activity mediate responses to situations. To the degree that humans share common emotional functions, neural structuration is similar across scales, giving rise to “basic” emotions. However, unique developmental and situational factors select for neural configurations mediating emotional variants.

Classifying emotion: a developmental account

The aim of this paper is to propose a systematic classification of emotions which can also characterize their nature. The first challenge we address is the submission of clear criteria for a theory of emotions that determine which mental phenomena are emotions and which are not. We suggest that emotions as a subclass of mental states are determined by their functional roles. The second and main challenge is the presentation of a classification and theory of emotions that can account for all existing varieties. We argue that we must classify emotions according to four developmental stages: 1. pre-emotions as unfocussed expressive emotion states, 2. basic emotions, 3. primary cognitive emotions, and 4. secondary cognitive emotions. We suggest four types of basic emotions (fear, anger, joy and sadness) which are systematically differentiated into a diversity of more complex emotions during emotional development. The classification distinguishes between basic and non-basic emotions and our multi-factorial account considers cognitive, experiential, physiological and behavioral parameters as relevant for constituting an emotion. However, each emotion type is constituted by a typical pattern according to which some features may be more significant than others. Emotions differ strongly where these patterns of features are concerned, while their essential functional roles are the same. We argue that emotions form a unified ontological category that is coherent and can be well defined by their characteristic functional roles. Our account of emotions is supported by data from developmental psychology, neurobiology, evolutionary biology and sociology.

Représentation et détection des émotions dans des dialogues enregistrés dans un centre d'appel. Des émotions complexes dans des données réelles

Complex emotions in real-life context have not been studied much. We are exploring how to represent and automatically detect a subject's emotional state in human-human spoken interaction. In contrast to most previous studies, conducted on artificial data, this paper addresses some of the challenges faced when studying real-life non-basic emotions. Real-life spoken dialogs from call-center services have revealed the presence of many blended emotions. A soft emotion vector is used to represent emotion mixtures. This representation enables to obtain a much more reliable annotation and to select the part of the corpus without conflictual blended emotions for training models. A correct detection rate of about 80% was obtained with SVMs and decision trees models between Negative and Neutral emotions using paralinguistic cues on a corpus of 20 hours of recording in a Medical call-center.

The Role of Basic, Self-Conscious and Self-Conscious Evaluative Emotions in Children’s Memory and Understanding of Emotion

In past research, emotion has been classified as basic, self-conscious or self-conscious evaluative, with each type of emotion being progressively more difficult for children to understand (Lewis, M. (2000a). In M. Lewis & J. M. Haviland-Jones (Eds.), Handbook of emotions (2nd ed., pp. 265–280), New York, NY: The Guilford Press). Although researchers have examined children’s understanding of individual emotions (e.g., guilt), researchers have not assessed children’s understanding and memory for emotions based on this classification. In the present research, 6-, 8- and 10-year-old children’s memory and understanding of basic, self-conscious and self-conscious evaluative emotions were examined. Although a memory advantage was found for emotions, this effect was less so for the younger children and less so for non-basic emotions. In fact, 6-year-old children and, to a lesser extent 8-year-old children, were more likely than older children to recall self-conscious and self-conscious evaluative emotions with basic emotion labels, and were more likely to explain them using basic emotion labels. Overall, negative emotions (e.g., mad, guilt) were better recalled than positive emotions (e.g., happy, pride), regardless of type of emotion. Gender differences were found as girls were more likely to remember emotion than boys, especially when the emotion action was specifically labeled and a female character experienced it.

INTRACULTURAL VARIATION OF SEMANTIC AND EPISODIC EMOTION KNOWLEDGE IN ESTONIAN

This article compares gender- and age-based differences in the cognitive salience of emotion words elicited in two list tasks displaying the content of semantic and episodic emotion knowledge. The semantic emotion knowledge of women appeared as more accessible and different from their episodic knowledge; the difference in the cognitive salience of basic and non-basic emotion terms was higher. As for the age-related variation, the maximum difference between the basic term salience in the semantic and episodic emotion knowledge was revealed in the youngest group (age 14-26), whereas the middle-aged group (age 29-41) displayed the highest similarity.

Mixed feelings: expression of non-basic emotions in a muscle-based talking head

We present an algorithm for generating facial expressions for a continuum of pure and mixed emotions of varying intensity. Based on the observation that in natural interaction among humans, shades of emotion are much more frequently encountered than expressions of basic emotions, a method to generate more than Ekman’s six basic emotions (joy, anger, fear, sadness, disgust and surprise) is required. To this end, we have adapted the algorithm proposed by Tsapatsoulis et al. [1] to be applicable to a physics-based facial animation system and a single, integrated emotion model. A physics-based facial animation system was combined with an equally flexible and expressive text-to-speech synthesis system, based upon the same emotion model, to form a talking head capable of expressing non-basic emotions of varying intensities. With a variety of life-like intermediate facial expressions captured as snapshots from the system we demonstrate the appropriateness of our approach.

Mixed feelings: expression of non-basic emotions in a muscle-based talking head

Mixed feelings: expression of non-basic emotions in a muscle-based talking head

Challenges in real-life emotion annotation and machine learning based detection

Since the early studies of human behavior, emotion has attracted the interest of researchers in many disciplines of Neurosciences and Psychology. More recently, it is a growing field of research in computer science and machine learning. We are exploring how the expression of emotion is perceived by listeners and how to represent and automatically detect a subject's emotional state in speech. In contrast with most previous studies, conducted on artificial data with archetypal emotions, this paper addresses some of the challenges faced when studying real-life non-basic emotions. We present a new annotation scheme allowing the annotation of emotion mixtures. Our studies of real-life spoken dialogs from two call center services reveal the presence of many blended emotions, dependent on the dialog context. Several classification methods (SVM, decision trees) are compared to identify relevant emotional states from prosodic, disfluency and lexical cues extracted from the real-life spoken human-human interactions.

Is love a “basic” emotion?

AbstractThis article documents the neglect of love in many contemporary emotion theories, despite its prominence in the lay psychology of emotion. We argue that love should be considered a basic emotion, like anger, sadness, happiness, and fear. We discuss the criteria that various theorists use to distinguish basic from nonbasic emotions, and we marshal arguments and evidence from a variety of sources suggesting that love fits the criteria for basicness. We conclude that a number of controversies over the status of love can be resolved by distinguishing between the momentarysurgeform of love, a basic emotion having properties similar to joy, sadness, fear, etc., andrelationallove, a bond that develops between people, associated with states that include not only surge love, but many other emotions such as distress and anxiety. Finally, we suggest that “love” is the broad, everyday name for emotions related to three interrelated behavioral systems discussed by Bowlby (1979): attachment, caregiving, and sex.