Nonverbal Emotional Vocalizations Research Articles

Automatic Brain Categorization of Discrete Auditory Emotion Expressions.

Seamlessly extracting emotional information from voices is crucial for efficient interpersonal communication. However, it remains unclear how the brain categorizes vocal expressions of emotion beyond the processing of their acoustic features. In our study, we developed a new approach combining electroencephalographic recordings (EEG) in humans with a frequency-tagging paradigm to 'tag' automatic neural responses to specificcategories of emotion expressions. Participants were presented with a periodic stream of heterogeneous non-verbal emotional vocalizations belonging to five emotion categories: anger, disgust, fear, happiness and sadness at 2.5Hz (stimuli length of 350ms with a 50ms silent gap between stimuli). Importantly, unknown to the participant, a specific emotion category appeared at a target presentation rate of 0.83Hz that would elicit an additional response in the EEG spectrum only if the brain discriminates the target emotion category from other emotion categories and generalizes across heterogeneous exemplars of the target emotion category. Stimuli were matched across emotion categories for harmonicity-to-noise ratio, spectral center of gravity and pitch. Additionally, participants were presented with a scrambled version of the stimuli with identical spectral content and periodicity but disrupted intelligibility. Both types of sequences had comparable envelopes and early auditory peripheral processing computed via the simulation of the cochlear response. We observed that in addition to the responses at the general presentation frequency (2.5Hz) in both intact and scrambled sequences, a greater peak in the EEG spectrum at the target emotion presentation rate (0.83Hz) and its harmonics emerged in the intact sequence in comparison to the scrambled sequence. The greater response at the target frequency in the intact sequence, together with our stimuli matching procedure, suggest that the categorical brain response elicited by a specific emotion is at least partially independent from the low-level acoustic features of the sounds. Moreover, responses at the fearful and happy vocalizations presentation rates elicited different topographies and differenttemporal dynamics, suggesting that different discrete emotions are represented differently in the brain. Our paradigm revealed the brain's ability to automatically categorize non-verbal vocal emotion expressions objectively (at a predefined frequency of interest), behavior-free, rapidly (in few minutes of recording time) and robustly (with a high signal-to-noise ratio), making it a useful tool to study vocal emotion processing and auditory categorization in generaland in populations where behavioral assessments are more challenging.

Decoding emotions from nonverbal vocalizations: How much voice signal is enough?

How much acoustic signal is enough for an accurate recognition of nonverbal emotional vocalizations? Using a gating paradigm (7 gates from 100 to 700 ms), the current study probed the effect of stimulus duration on recognition accuracy of emotional vocalizations expressing anger, disgust, fear, amusement, sadness and neutral states. Participants (n = 52) judged the emotional meaning of vocalizations presented at each gate. Increased recognition accuracy was observed from gates 2 to 3 for all types of vocalizations. Neutral vocalizations were identified with the shortest amount of acoustic information relative to all other types of vocalizations. A shorter acoustic signal was required to decode amusement compared to fear, anger and sadness, whereas anger and fear required equivalent amounts of acoustic information to be accurately recognized. These findings confirm that the time course of successful recognition of discrete vocal emotions varies by emotion type. Compared to prior studies, they additionally indicate that the type of auditory signal (speech prosody vs. nonverbal vocalizations) determines how quickly listeners recognize emotions from a speaker’s voice.

Automaticity in the recognition of nonverbal emotional vocalizations.

The ability to perceive the emotions of others is crucial for everyday social interactions. Important aspects of visual socioemotional processing, such as the recognition of facial expressions, are known to depend on largely automatic mechanisms. However, whether and how properties of automaticity extend to the auditory domain remains poorly understood. Here we ask if nonverbal auditory emotion recognition is a controlled deliberate or an automatic efficient process, using vocalizations such as laughter, crying, and screams. In a between-subjects design (N = 112), and covering eight emotions (four positive), we determined whether emotion recognition accuracy (a) is improved when participants actively deliberate about their responses (compared with when they respond as fast as possible) and (b) is impaired when they respond under low and high levels of cognitive load (concurrent task involving memorizing sequences of six or eight digits, respectively). Response latencies were also measured. Mixed-effects models revealed that recognition accuracy was high across emotions, and only minimally affected by deliberation and cognitive load; the benefits of deliberation and costs of cognitive load were significant mostly for positive emotions, notably amusement/laughter, and smaller or absent for negative ones; response latencies did not suffer under low or high cognitive load; and high recognition accuracy (approximately 90%) could be reached within 500 ms after the stimulus onset, with performance exceeding chance-level already between 300 and 360 ms. These findings indicate that key features of automaticity, namely fast and efficient/effortless processing, might be a modality-independent component of emotion recognition. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Auditory affective processing requires awareness.

Recent work has challenged the previously widely accepted belief that affective processing does not require awareness and can be carried out with more limited resources than semantic processing. This debate has focused exclusively on visual perception, even though evidence from both human and animal studies suggests that existence for nonconscious affective processing would be physiologically more feasible in the auditory system. Here we contrast affective and semantic processing of nonverbal emotional vocalizations under different levels of awareness in three experiments, using explicit (two-alternative forced choice masked affective and semantic categorization tasks, Experiments 1 and 2) and implicit (masked affective and semantic priming, Experiment 3) measures. Identical stimuli and design were used in the semantic and affective tasks. Awareness was manipulated by altering stimulus-mask signal-to-noise ratio during continuous auditory masking. Stimulus awareness was measured on each trial using a four-point perceptual awareness scale. In explicit tasks, neither affective nor semantic categorization could be performed in the complete absence of awareness, while both tasks could be performed above chance level when stimuli were consciously perceived. Semantic categorization was faster than affective evaluation. When the stimuli were partially perceived, semantic categorization accuracy exceeded affective evaluation accuracy. In implicit tasks neither affective nor semantic priming occurred in the complete absence of awareness, whereas both affective and semantic priming emerged when participants were aware of the primes. We conclude that auditory semantic processing is faster than affective processing, and that both affective and semantic auditory processing are dependent on awareness. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Supplemental Material for Automaticity in the Recognition of Nonverbal Emotional Vocalizations

Supplemental Material for Automaticity in the Recognition of Nonverbal Emotional Vocalizations

Perceptual and acoustic differences between authentic and acted nonverbal emotional vocalizations.

Most research on nonverbal emotional vocalizations is based on actor portrayals, but how similar are they to the vocalizations produced spontaneously in everyday life? Perceptual and acoustic differences have been discovered between spontaneous and volitional laughs, but little is known about other emotions. We compared 362 acted vocalizations from seven corpora with 427 authentic vocalizations using acoustic analysis, and 278 vocalizations (139 authentic and 139 acted) were also tested in a forced-choice authenticity detection task ( N = 154 listeners). Target emotions were: achievement, amusement, anger, disgust, fear, pain, pleasure, and sadness. Listeners distinguished between authentic and acted vocalizations with accuracy levels above chance across all emotions (overall accuracy 65%). Accuracy was highest for vocalizations of achievement, anger, fear, and pleasure, which also displayed the largest differences in acoustic characteristics. In contrast, both perceptual and acoustic differences between authentic and acted vocalizations of amusement, disgust, and sadness were relatively small. Acoustic predictors of authenticity included higher and more variable pitch, lower harmonicity, and less regular temporal structure. The existence of perceptual and acoustic differences between authentic and acted vocalizations for all analysed emotions suggests that it may be useful to include spontaneous expressions in datasets for psychological research and affective computing.

Mimicry of ingroup and outgroup emotional expressions

ABSTRACTResearch into emotional mimicry has found that the extent to which we mimic others depends on group membership and the emotion at hand. Particularly, negative emotions are mimicked more when expressed by the perceiver’s ingroup. It is, however, still debated what process underlies emotional mimicry and whether previous findings of enhanced mimicry of negative emotions expressed by ingroup members are robust. We therefore first aimed to replicate Study 2 of van der Schalk, Fischer et al. (2011), specifically testing the finding of differences in emotional mimicry for models from different ethnic groups. Moreover, we extended the study by (1) including nonverbal emotional vocalizations and (2) including all negative emotions that were previously studied in a group mimicry context, that is, anger, fear, and sadness, in addition to happiness. We test two alternative explanations of emotional mimicry: whether emotional mimicry is a matched-motor response or whether emotional mimicry is influenced by meaning and context as proposed by the Emotion Mimicry in Context view. The results do not replicate the findings of van der Schalk, Fischer et al. (2011). For the facial and vocalization stimuli, we did not find emotional mimicry effects for anger, fear, or sadness, neither did we find effects of group membership. We only found emotional mimicry effects for happiness (action units 6 and 12) in the facial study. We discuss various explanations for the lack of findings, with the within-subjects design as most likely explanation.

The voice conveys emotion in ten globalized cultures and one remote village in Bhutan.

With data from 10 different globalized cultures and 1 remote, isolated village in Bhutan, we examined universals and cultural variations in the recognition of 16 nonverbal emotional vocalizations. College students in 10 nations (Study 1) and villagers in remote Bhutan (Study 2) were asked to match emotional vocalizations to 1-sentence stories of the same valence. Guided by previous conceptualizations of recognition accuracy, across both studies, 7 of the 16 vocal burst stimuli were found to have strong or very strong recognition in all 11 cultures, 6 vocal bursts were found to have moderate recognition, and 4 were not universally recognized. All vocal burst stimuli varied significantly in terms of the degree to which they were recognized across the 11 cultures. Our discussion focuses on the implications of these results for current debates concerning the emotion conveyed in the voice.

Laugh Like You Mean It: Authenticity Modulates Acoustic, Physiological and Perceptual Properties of Laughter

Several authors have recently presented evidence for perceptual and neural distinctions between genuine and acted expressions of emotion. Here, we describe how differences in authenticity affect the acoustic and perceptual properties of laughter. In an acoustic analysis, we contrasted spontaneous, authentic laughter with volitional, fake laughter, finding that spontaneous laughter was higher in pitch, longer in duration, and had different spectral characteristics from volitional laughter that was produced under full voluntary control. In a behavioral experiment, listeners perceived spontaneous and volitional laughter as distinct in arousal, valence, and authenticity. Multiple regression analyses further revealed that acoustic measures could significantly predict these affective and authenticity judgements, with the notable exception of authenticity ratings for spontaneous laughter. The combination of acoustic predictors differed according to the laughter type, where volitional laughter ratings were uniquely predicted by harmonics-to-noise ratio (HNR). To better understand the role of HNR in terms of the physiological effects on vocal tract configuration as a function of authenticity during laughter production, we ran an additional experiment in which phonetically trained listeners rated each laugh for breathiness, nasality, and mouth opening. Volitional laughter was found to be significantly more nasal than spontaneous laughter, and the item-wise physiological ratings also significantly predicted affective judgements obtained in the first experiment. Our findings suggest that as an alternative to traditional acoustic measures, ratings of phonatory and articulatory features can be useful descriptors of the acoustic qualities of nonverbal emotional vocalizations, and of their perceptual implications.

In the ear of the beholder: How age shapes emotion processing in nonverbal vocalizations.

It is well established that emotion recognition of facial expressions declines with age, but evidence for age-related differences in vocal emotions is more limited. This is especially true for nonverbal vocalizations such as laughter, sobs, or sighs. In this study, 43 younger adults (M = 22 years) and 43 older ones (M = 61.4 years) provided multiple emotion ratings of nonverbal emotional vocalizations. Contrasting with previous research, which often includes only one positive emotion (happiness) versus several negative ones, we examined 4 positive and 4 negative emotions: achievement/triumph, amusement, pleasure, relief, anger, disgust, fear, and sadness. We controlled for hearing loss and assessed general cognitive decline, cognitive control, verbal intelligence, working memory, current affect, emotion regulation, and personality. Older adults were less sensitive than younger ones to the intended vocal emotions, as indicated by decrements in ratings on the intended emotion scales and accuracy. These effects were similar for positive and negative emotions, and they were independent of age-related differences in cognitive, affective, and personality measures. Regression analyses revealed that younger and older participants' responses could be predicted from the acoustic properties of the temporal, intensity, fundamental frequency, and spectral profile of the vocalizations. The two groups were similarly efficient in using the acoustic cues, but there were differences in the patterns of emotion-specific predictors. This study suggests that ageing produces specific changes on the processing of nonverbal vocalizations. That decrements were not attenuated for positive emotions indicates that they cannot be explained by a positivity effect in older adults.

Joint effect of alexithymia and mood on the categorization of nonverbal emotional vocalizations

The role of stable factors, such as alexithymia (i.e., difficulties identifying and expressing feelings, externally oriented cognitive style), or temporary factors, such as affective states (mood), on emotion perception has been widely investigated in the literature. However, little is known about the separate or joint effect of the alexithymia level and affective states (positive affectivity, negative affectivity) on the recognition of nonverbal emotional vocalizations (NEV) (e.g., laughs, cries, or sighs). In this study, participants had to categorize NEV communicating 10 emotions by selecting the correct verbal emotional label. Results show that the level of alexithymia is negatively correlated to the capacity to accurately categorize negative vocalizations, and more particularly sad NEV. On the other hand, negative affectivity appeared negatively correlated with the ability to accurately categorize NEV in general, and negative vocalizations in particular. After splitting the results by the alexithymia level (high vs. low scorers), significant associations between mood and accuracy rates were found in the group of high alexithymia scorers only. These findings support the idea that alexithymic features act across sensory modalities and suggest a mood-interference effect that would be stronger in those individuals.

Common and differential brain responses in men and women to nonverbal emotional vocalizations by the same and opposite sex

Nonverbal emotional vocalizations are one of the most elementary ways of communicating in humans. We examined the impact of sex differences on neural responses to laughter and crying produced by the same and opposite sex. Thirty subjects (15 women) underwent functional magnetic resonance imaging during a sex identification task for laughter, crying, and neutral voices. The parahippocampal gyrus was involved in both men and women while hearing laughter of the same sex, suggesting greater positive emotional processing and greater attention toward emotional context in response to laughter of the same sex than of the opposite sex. The posterior cingulate was involved in both men and women while hearing crying of the opposite sex, suggesting that empathic processing may occur more in response to crying of the opposite sex than of the same sex. Furthermore, brain responses to crying of the opposite sex seem to reflect upon men's efforts to perform emotional regulation and women's empathic concerns.

Electrophysiological insights into processing nonverbal emotional vocalizations

The ability to identify emotions from the human voice is a crucial aspect of social cognition. Currently, very little is known about the neural correlates of nonverbal emotional vocalizations processing. We used electrophysiological measures to examine the processing of emotional versus neutral vocalizations. Participants listened to nonverbal angry, happy, and neutral vocalizations, as well as to monkey voices, which served as a response target. Angry sounds were processed differently than happy and neutral ones starting at 50 ms, whereas both vocal emotions were associated with decreased N100 and increased P200 components relative to neutral sounds. These findings indicate a rapid and automatic differentiation of emotional as compared with neutral vocalizations and suggest that this differentiation is not dependent on valence.

The impairment of emotion recognition in Huntington’s disease extends to positive emotions

Patients with Huntington’s Disease (HD) are impaired in the recognition of emotional signals. However, the nature and extent of the impairment is controversial: it has variously been argued to disproportionately affect disgust (e.g., Sprengelmeyer et al., 1996), to be general for negative emotions ( Snowden et al., 2008), or to be a consequence of item difficulty ( Milders et al., 2003). Yet no study to date has included more than one positive stimulus category in emotion recognition tasks, and most studies have focused on the recognition of emotions from facial stimuli. In this study, we test the hypothesis that patients with HD may be impaired in their recognition of positive as well as negative emotional signals, by examining the recognition of a range of positive emotions from vocal cues. We present a study of 14 Huntington’s patients and 15 controls performing a forced-choice task with a previously validated set of negative and positive non-verbal emotional vocalizations ( Sauter and Scott, 2007). Although HD patients performed above chance for each emotion, they were found to be impaired in both positive and negative emotions, including pleasure, fear and anger. These findings complement previous work by demonstrating that impairments in emotion recognition in HD extend to positive and negative emotions, which may imply a general deficit.

Cross-cultural recognition of basic emotions through nonverbal emotional vocalizations

Emotional signals are crucial for sharing important information, with conspecifics, for example, to warn humans of danger. Humans use a range of different cues to communicate to others how they feel, including facial, vocal, and gestural signals. We examined the recognition of nonverbal emotional vocalizations, such as screams and laughs, across two dramatically different cultural groups. Western participants were compared to individuals from remote, culturally isolated Namibian villages. Vocalizations communicating the so-called "basic emotions" (anger, disgust, fear, joy, sadness, and surprise) were bidirectionally recognized. In contrast, a set of additional emotions was only recognized within, but not across, cultural boundaries. Our findings indicate that a number of primarily negative emotions have vocalizations that can be recognized across cultures, while most positive emotions are communicated with culture-specific signals.