Emotion Portrayals Research Articles

Categorical and Dimensional Ratings of Emotional Speech: Behavioral Findings From the Morgan Emotional Speech Set

Purpose Emotion classification for auditory stimuli typically employs 1 of 2 approaches (discrete categories or emotional dimensions). This work presents a new emotional speech set, compares these 2 classification methods for emotional speech stimuli, and emphasizes the need to consider the entire communication model (i.e., the talker, message, and listener) when studying auditory emotion portrayal and perception. Method Emotional speech from male and female talkers was evaluated using both categorical and dimensional rating methods. Ten young adult listeners (ages 19-28 years) evaluated stimuli recorded in 4 emotional speaking styles (Angry, Calm, Happy, and Sad). Talker and listener factors were examined for potential influences on emotional ratings using categorical and dimensional rating methods. Listeners rated stimuli by selecting an emotion category, rating the activation and pleasantness, and indicating goodness of category fit. Results Discrete ratings were generally consistent with dimensional ratings for speech, with accuracy for emotion recognition well above chance. As stimuli approached dimensional extremes of activation and pleasantness, listeners were more confident in their category selection, indicative of a hybrid approach to emotion classification. Female talkers were rated as more activated than male talkers, and female listeners gave higher ratings of activation compared to male listeners, confirming gender differences in emotion perception. Conclusion A hybrid model for auditory emotion classification is supported by the data. Talker and listener factors, such as gender, were found to impact the ratings of emotional speech and must be considered alongside stimulus factors in the design of future studies of emotion.

Development and validation of the Facial Expression Recognition Test (FERT).

Detecting the emotional state of others from facial expressions is a key ability in emotional competence and several instruments have been developed to assess it. Typical emotion recognition tests are assumed to be unidimensional, use pictures or videos of emotional portrayals as stimuli, and ask the participant which emotion is depicted in each stimulus. However, using actor portrayals adds a layer of difficulty in developing such a test: the portrayals may fail to be convincing and may convey a different emotion than intended. For this reason, evaluating and selecting stimuli is of crucial importance. Existing tests typically base item evaluation on consensus or expert judgment, but these methods could favor items with high agreement over items that better differentiate ability levels and they could not formally test the item pool for unidimensionality. To address these issues, the authors propose a new test, named Facial Expression Recognition Test (FERT), developed using an item response theory two-parameter logistic model. Data from 1,002 online participants were analyzed using both a unidimensional and a bifactor model, and showed that the item pool could be considered unidimensional. The selection was based on the items' discrimination parameters, retaining only the most informative items to investigate the latent ability. The resulting 36-item test was reliable and quick to administer. The authors found both a gender difference in the ability to recognize emotions and a decline of such ability with age. The PsychoPy implementation of the test and the scoring script are available on a Github repository. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

The wisdom of the body: Listeners' autonomic arousal distinguishes between spontaneous and posed vocal emotions.

It has been the matter of much debate whether perceivers are able to distinguish spontaneous vocal expressions of emotion from posed vocal expressions (e.g., emotion portrayals). In this experiment, we show that such discrimination can be manifested in the autonomic arousal of listeners during implicit processing of vocal emotions. Participants (N = 21, age: 20-55 years) listened to two consecutive blocks of brief voice clips and judged the gender of the speaker in each clip, while we recorded three measures of sympathetic arousal of the autonomic nervous system (skin conductance level, mean arterial blood pressure, pulse rate). Unbeknownst to the listeners, the blocks consisted of two types of emotional speech: spontaneous and posed clips. As predicted, spontaneous clips yielded higher arousal levels than posed clips, suggesting that listeners implicitly distinguished between the two kinds of expression, even in the absence of any requirement to retrieve emotional information from the voice. We discuss the results with regard to theories of emotional contagion and the use of posed stimuli in studies of emotions.

Motivational factors associated with drowsy driving behavior: a qualitative investigation of college students

ObjectivesThis qualitative investigation sought to identify the motivational factors that contribute to drowsy driving in college students and to discover important messaging strategies that may help prevent or reduce this behavior in this population. DesignFour focus groups of college students. SettingA large university in the Washington, DC, metropolitan area during the Fall 2016 term. ParticipantsTwenty-six undergraduate students between the ages of 18 and 25 years. MeasurementsNotes and transcripts from the focus group sessions were analyzed to identify recurring themes regarding attitudes, motivations, experiences, influences, and potential preventive messaging strategies related to drowsy driving. ResultsAlthough most participants had heard of drowsy driving and were concerned about it, they did not associate it with legal risks and were more concerned about alcohol-impaired and distracted driving as crash risks. Participants viewed drowsy driving as a normal and unavoidable part of their lives over which they had little control. For potential anti–drowsy driving messaging strategies, participants preferred messages delivered via audiovisual or social media that featured graphic and emotional portrayals of crashes and their consequences. Participants also voiced strong support for preventive messaging strategies equating various degrees of sleep deprivation to known impairing levels of alcohol, as well as messages providing cues to action to actual drowsy drivers on roadways. ConclusionsIncreased enforcement, education, and public messaging campaigns are needed to increase knowledge and influence attitudes and opinions among young drivers about the dangers and social unacceptability of drowsy driving.

The Mirror to Our Soul? Comparisons of Spontaneous and Posed Vocal Expression of Emotion

It has been the subject of much debate in the study of vocal expression of emotions whether posed expressions (e.g., actor portrayals) are different from spontaneous expressions. In the present investigation, we assembled a new database consisting of 1877 voice clips from 23 datasets, and used it to systematically compare spontaneous and posed expressions across 3 experiments. Results showed that (a) spontaneous expressions were generally rated as more genuinely emotional than were posed expressions, even when controlling for differences in emotion intensity, (b) there were differences between the two stimulus types with regard to their acoustic characteristics, and (c) spontaneous expressions with a high emotion intensity conveyed discrete emotions to listeners to a similar degree as has previously been found for posed expressions, supporting a dose–response relationship between intensity of expression and discreteness in perceived emotions. Our conclusion is that there are reliable differences between spontaneous and posed expressions, though not necessarily in the ways commonly assumed. Implications for emotion theories and the use of emotion portrayals in studies of vocal expression are discussed.

African pastors and the religious (re)production of a visual culture

The (re)production of visual material is an extended practice amongst Pentecostal-Charismatics worldwide and church leaders seem to play a decisive role in engendering such practice. While promoting spiritual missions and institutional visions, pastors reinforce their personae in accordance to the “spiritual gifts” they perform. This article explores the forging of a visual culture that legitimates such performances, through the usage of posters and flyers which, it is argued, feature a particular speech. The arguments emerge from the observed cases of African pastors located in Spain and South Africa and the “discovery” of these visual representations through ethnographic fieldwork carried out between 2012 and 2015. The ascription of meaning to such materials is achieved by focusing on the graphic portrayal of emotions, religious experiences, ethos and morality, aligned with a somewhat “African Pentecostal speech”. It is suggested that, by engendering a “speech community”, this graphic language reflects the global and local quests for belonging faced by worshippers in the Diaspora.

First experiences with neuropsychological effects of oxytocin administration in childhood-onset craniopharyngioma.

The hypothalamic hormone oxytocin plays a major role in regulation of behavior and body composition. Quality of survival is frequently impaired in childhood craniopharyngioma patients due to sequelae such as behavioral deficits and severe obesity caused by tumor or treatment-related hypothalamic lesions. In our pilot cross-sectional study, we analyzed emotion recognition abilities and oxytocin concentrations in saliva and urine before and after single nasal administration of 24 IU oxytocin in 10 craniopharyngioma patients. Four craniopharyngioma presented with grade I lesions (limited to anterior hypothalamic areas) and 6 craniopharyngioma with grade II lesions (involving mammillary bodies and posterior hypothalamic areas). Emotional tasks were assessed before and after administration of oxytocin using the Geneva multimodal emotion portrayals corpus and the Multidimensional Mood Questionnaire. All patients presented with detectable levels of oxytocin before administration. Nasal administration of oxytocin was well-tolerated and resulted in increased oxytocin concentrations in saliva and urine. After oxytocin administration, craniopharyngioma patients with postsurgical lesions limited to the anterior hypothalamus area showed improvements in emotional identifications compared to craniopharyngioma patients with lesions of anterior and posterior hypothalamic areas. Focusing on correct assignments to positive and negative emotion categories, craniopharyngioma patients improved assignment to negative emotions. Oxytocin might have positive effects on emotion perception in craniopharyngioma patients with specific lesions of the anterior hypothalamic area. Further studies on larger cohorts are warranted.

Jagd und Joch, Frau und Pferd: Marie von Ebner-Eschenbach’s Unsühnbar as a Reflection of Nineteenth-Century Human-Animal Bonds

The increased attention to animals during the nineteenth century often led to close human-animal bonds, an aspect of life depicted in the work of Austrian author Marie von Ebner-Eschenbach. Indeed, much of Ebner’s oeuvre is characterized by animal representations. Drawing on a developing conception of the animal’s capacity for feeling and emotion, Ebner observes and questions human ethical behavior, fulfilling her reputation as an important Tierfreundin of the nineteenth century. In so doing, she participates in a nineteenth-century tradition of animal representation that urges readers to examine their own humanity. In Unsuhnbar , Ebner’s novel of adultery, the depiction of Maria Dornach’s bonds with animals enriches Ebner’s criticisms of aristocratic social mores such as inheritance and arranged marriages and inspires reader empathy with the novel’s protagonist through her sympathetic identification with animal figures. Ebner’s portrayal of human emotions inspired by, influenced by, and directed at animals and her attribution of emotional capacity to animals themselves reflects a nineteenth-century preoccupation with the bonds between the species, moving readers to sympathy for both.

The Effect of Narrow-Band Transmission on Recognition of Paralinguistic Information From Human Vocalizations

Practically, no knowledge exists on the effects of speech coding and recognition for narrow-band transmission of speech signals within certain frequency ranges especially in relation to the recognition of paralinguistic cues in speech. We thus investigated the impact of narrow-band standard speech coders on the machine-based classification of affective vocalizations and clinical vocal recordings. In addition, we analyzed the effect of speech low-pass filtering by a set of different cut-off frequencies, either chosen as static values in the 0.5–5-kHz range or given dynamically by different upper limits from the first five speech formants (F1–F5). Speech coding and recognition were tested, first, according to short-term speaker states by using affective vocalizations as given by the Geneva Multimodal Emotion Portrayals. Second, in relation to long-term speaker traits, we tested vocal recording from clinical populations involving speech impairments as found in the Child Pathological Speech Database. We employ a large acoustic feature space derived from the Interspeech Computational Paralinguistics Challenge. Besides analysis of the sheer corruption outcome, we analyzed the potential of matched and multicondition training as opposed to miss-matched condition. In the results, first, multicondition and matched-condition training significantly increase performances as opposed to mismatched condition. Second, downgrades in classification accuracy occur, however, only at comparably severe levels of low-pass filtering. The downgrades especially appear for multi-categorical rather than for binary decisions. These can be dealt with reasonably by the alluded strategies.

Perception of temporal asymmetries in dynamic facial expressions.

In the current study we examined whether timeline-reversals and emotional direction of dynamic facial expressions affect subjective experience of human observers. We recorded natural movies of faces that increased or decreased their expressions of fear, and played them either in the natural frame order or reversed from last to first frame (reversed timeline). This led to four conditions of increasing or decreasing fear, either following the natural or reversed temporal trajectory of facial dynamics. This 2-by-2 factorial design controlled for visual low-level properties, static visual content, and motion energy across the different factors. It allowed us to examine perceptual consequences that would occur if the timeline trajectory of facial muscle movements during the increase of an emotion are not the exact mirror of the timeline during the decrease. It additionally allowed us to study perceptual differences between increasing and decreasing emotional expressions. Perception of these time-dependent asymmetries have not yet been quantified. We found that three emotional measures, emotional intensity, artificialness of facial movement, and convincingness or plausibility of emotion portrayal, were affected by timeline-reversals as well as by the emotional direction of the facial expressions. Our results imply that natural dynamic facial expressions contain temporal asymmetries, and show that deviations from the natural timeline lead to a reduction of perceived emotional intensity and convincingness, and to an increase of perceived artificialness of the dynamic facial expression. In addition, they show that decreasing facial expressions are judged as less plausible than increasing facial expressions. Our findings are of relevance for both, behavioral as well as neuroimaging studies, as processing and perception are influenced by temporal asymmetries.

On English Speakers’ Ability to Communicate Emotion in Mandarin

Abstract:The ability of Mandarin learners to express emotion in Mandarin has received little attention. This study examines how English L1 users express emotions in Mandarin and how this expression differs from that of Mandarin L1 users. Scenarios were adopted to elicit joy, anger, sadness, fear, and neutrality. Both groups articulated anger, joy, and fear with a high pitch. Both groups also employed high intensity for anger and joy and low intensity for sadness and fear. Learners generally employed larger F0 ranges than native speakers, particularly for anger and fear. Learners articulated level tones with lengthened duration and contour tones with shortened duration, affecting the correctness of the portrayal of emotions. Learners used a similar intensity range for all emotions, whereas native speakers tended to vary the intensity with different emotions. The results have implications for teaching Mandarin as a second language with special reference to prosodic naturalness in expressing emotions.

The Role of Perceived Voice and Speech Characteristics in Vocal Emotion Communication

Aiming at a more comprehensive assessment of nonverbal vocal emotion communication, this article presents the development and validation of a new rating instrument for the assessment of perceived voice and speech features. In two studies, using two different sets of emotion portrayals by German and French actors, ratings of perceived voice and speech characteristics (loudness, pitch, intonation, sharpness, articulation, roughness, instability, and speech rate) were obtained from non-expert (untrained) listeners. In addition, standard acoustic parameters were extracted from the voice samples. Overall, highly similar patterns of results were found in both studies. Rater agreement (reliability) reached highly satisfactory levels for most features. Multiple discriminant analysis results reveal that both perceived vocal features and acoustic parameters allow a high degree of differentiation of the actor-portrayed emotions. Positive emotions can be classified with a higher hit rate on the basis of perceived vocal features, confirming suggestions in the literature that it is difficult to find acoustic valence indicators. The results show that the suggested scales (Geneva Voice Perception Scales) can be reliably measured and make a substantial contribution to a more comprehensive assessment of the process of emotion inferences from vocal expression.

Paralingua – A New Speech Corpus for the Studies of Paralinguistic Features

This paper introduces “Paralingua” - a new speech corpus created within a larger ongoing project whose primary aim was to develop a speaker recognition and identification system for forensics. The present corpus was designed for the purpose of analysis of selected paralinguistic features in continuous speech and for preliminary examination of the vocal display of affective states. The recorded (and annotated) data include conversational speech in the form of task-oriented dialogues, emotional utterances (realized as emotion portrayals), and an acted court scene. As a reference material, a short read text was provided by each of the speakers.

Child Murder and Child’s Play: The Emotions of Children in Jakob Bidermann’s epic on the Massacre of the Innocents (Herodiados libri iii, 1622)

This article explores the portrayal of emotions by non-heroic characters (women and children) in a Latin epic on the massacre of the innocents by renowned seventeenth-century Jesuit playwright, Jakob Bidermann. Bidermann’s poem recreates the horrors of the biblical story in grisly detail, and yet it also contains vignettes of exquisitely observed psychological veracity, for example on the instinctive connection between mother and baby, on the passions of very young children, and on sibling rivalry. It is suggested that, notwithstanding its graphic treatment of a terrifying subject, the poem will have been considered suitable reading matter for young readers. Bidermann’s representation of the emotions of children and youth is considered in the light of Jesuit ideology, education, and missionary activity. The style of the poem is discussed in its relation both to Jesuit and classical models, especially Virgil.

Effort-Shape Characteristics of Emotion-Related Body Movement

The purpose of this study was to characterize the movement qualities of 5 target emotions during walking. We used an autobiographical memories paradigm for elicitation and observer judgments for emotion recognition. For each of the felt and recognized emotion portrayals, 6 Effort-Shape qualities were judged on a continuum between opposite qualities at the anchor points. Three general categories of movement style emerged, so that anger and joy shared anchor qualities at one end of the continuum, sadness had qualities at the opposite anchor, and content and neutral had qualities between the anchor extremes. The Effort-Shape profiles were unique for each target emotion, however, and mean scores were different between emotions even when emotions shared similar qualities. Emotions were classified using the Effort-Shape scores with accuracies ranging from 74–32 % for sad, anger, content and joy, respectively. For most of the target emotions, decoding accuracy was related to at least 4 Effort-Shape qualities, suggesting that decoding accuracy may be associated with a profile of movement qualities. This study highlights the importance of movement quality in bodily expression of emotion and demonstrates the effectiveness of Effort-Shape analysis in distinguishing among emotion-related movement styles.

An exploration on possible correlations among perception and physical characteristics of EMOVO emotional portrayals

This article presents the first attempt to investigate the existence of possible quantitative correlations among the physical characteristics of emotional portrayals and the emotions perceived by humans during their listening. Our aim was: a) to design and develop a new investigation procedure/protocol; b) to obtain information useful to recognition and synthesis of emotions conveyed by the human voice. Our results, obtained on a subset of the emotional portrayals contained in the corpus EMOVO, show that, apart from well known qualitative indications, it is also possible to observe clear quantitative trends for some couples of ‘emotionsignal characteristics’ as function of the recognition rate of the emotion.

Acoustical features in Mandarin emotional speech by native speakers of English

This study examines (1) whether native speakers of English (NS-E) can express emotions successfully in Mandarin speech, and (2) how their emotional expressions differ from native speakers of Mandarin (NS-C) when the emotional portrayals are recognizable. The acoustic features analyzed included F0, duration, and intensity. The scenario approach was adopted to elicit emotions joy, anger, sadness and fear, with neutral as a control. The data gathered (Sixteen NS-E and NS-C) were rated. F0 range at sentential level, mean F0 of each syllable, sentential and syllabic duration, and intensity signal of each segment were contrasted across groups within each emotional expression. The findings indicated that emotions by NS-C were recognized well, but joy, anger and fear by NS-E had low recognition rates because of accents, vocal cues and culture-specific components. Both groups adopted similar F0 range at the sentence level but joy in both and fear by NS-C showed small range. Only NS-C showed fast speech rate in anger. Emotions with high activations by NS-E were shorter. Anger and joy showed high intensity, while sadness and fear low intensity in both groups. NS-C tended to use different intensity range to indicate different emotions, while NS-E used similar range for all emotions.

Recognizing Emotions From an Ensemble of Features

This paper details the authors' efforts to push the baseline of emotion recognition performance on the Geneva Multimodal Emotion Portrayals (GEMEP) Facial Expression Recognition and Analysis database. Both subject-dependent and subject-independent emotion recognition scenarios are addressed in this paper. The approach toward solving this problem involves face detection, followed by key-point identification, then feature generation, and then, finally, classification. An ensemble of features consisting of hierarchical Gaussianization, scale-invariant feature transform, and some coarse motion features have been used. In the classification stage, we used support vector machines. The classification task has been divided into person-specific and person-independent emotion recognitions using face recognition with either manual labels or automatic algorithms. We achieve 100% performance for the person-specific one, 66% performance for the person-independent one, and 80% performance for overall results, in terms of classification rate, for emotion recognition with manual identification of subjects.

The Body Action and Posture Coding System (BAP): Development and Reliability

Several methods are available for coding body movement in nonverbal behavior research, but there is no consensus on a reliable coding system that can be used for the study of emotion expression. Adopting an integrative approach, we developed a new method, the body action and posture coding system, for the time-aligned micro description of body movement on an anatomical level (different articulations of body parts), a form level (direction and orientation of movement), and a functional level (communicative and self-regulatory functions). We applied the system to a new corpus of acted emotion portrayals, examined its comprehensiveness and demonstrated intercoder reliability at three levels: (a) occurrence, (b) temporal precision, and (c) segmentation. We discuss issues for further validation and propose some research applications.

Introducing the Geneva Multimodal expression corpus for experimental research on emotion perception.

Research on the perception of emotional expressions in faces and voices is exploding in psychology, the neurosciences, and affective computing. This article provides an overview of some of the major emotion expression (EE) corpora currently available for empirical research and introduces a new, dynamic, multimodal corpus of emotion expressions, the Geneva Multimodal Emotion Portrayals Core Set (GEMEP-CS). The design features of the corpus are outlined and justified, and detailed validation data for the core set selection are presented and discussed. Finally, an associated database with microcoded facial, vocal, and body action elements, as well as observer ratings, is introduced.