Vocal Affect Perception Research Articles

Disentangling emotional signals in the brain: an ALE meta-analysis of vocal affect perception.

Recent advances in neuroimaging research on vocal emotion perception have revealed voice-sensitive areas specialized in processing affect. Experimental data on this subject is varied, investigating a wide range of emotions through different vocal signals and task demands. The present meta-analysis was designed to disentangle this diversity of results by summarizing neuroimaging data in the vocal emotion perception literature. Data from 44 experiments contrasting emotional and neutral voices was analyzed to assess brain areas involved in vocal affect perception in general, as well as depending on the type of voice signal (speech prosody or vocalizations), the task demands (implicit or explicit attention to emotions), and the specific emotion perceived. Results reassessed a consistent bilateral network of Emotional Voices Areas consisting of the superior temporal cortex and primary auditory regions. Specific activations and lateralization of these regions, as well as additional areas (insula, middle temporal gyrus) were further modulated by signal type and task demands. Exploring the sparser data on single emotions also suggested the recruitment of other regions (insula, inferior frontal gyrus, frontal operculum) for specific aspects of each emotion. These novel meta-analytic results suggest that while the bulk of vocal affect processing is localized in the STC, the complexity and variety of such vocal signals entails functional specificities in complex and varied cortical (and potentially subcortical) response pathways.

The paradoxical role of emotional intensity in the perception of vocal affect

Vocalizations including laughter, cries, moans, or screams constitute a potent source of information about the affective states of others. It is typically conjectured that the higher the intensity of the expressed emotion, the better the classification of affective information. However, attempts to map the relation between affective intensity and inferred meaning are controversial. Based on a newly developed stimulus database of carefully validated non-speech expressions ranging across the entire intensity spectrum from low to peak, we show that the intuition is false. Based on three experiments (N = 90), we demonstrate that intensity in fact has a paradoxical role. Participants were asked to rate and classify the authenticity, intensity and emotion, as well as valence and arousal of the wide range of vocalizations. Listeners are clearly able to infer expressed intensity and arousal; in contrast, and surprisingly, emotion category and valence have a perceptual sweet spot: moderate and strong emotions are clearly categorized, but peak emotions are maximally ambiguous. This finding, which converges with related observations from visual experiments, raises interesting theoretical challenges for the emotion communication literature.

The perception of caricatured emotion in voice

Affective vocalisations such as screams and laughs can convey strong emotional content without verbal information. Previous research using morphed vocalisations (e.g. 25% fear/75% anger) has revealed categorical perception of emotion in voices, showing sudden shifts at emotion category boundaries. However, it is currently unknown how further modulation of vocalisations beyond the veridical emotion (e.g. 125% fear) affects perception. Caricatured facial expressions produce emotions that are perceived as more intense and distinctive, with faster recognition relative to the original and anti-caricatured (e.g. 75% fear) emotions, but a similar effect using vocal caricatures has not been previously examined. Furthermore, caricatures can play a key role in assessing how distinctiveness is identified, in particular by evaluating accounts of emotion perception with reference to prototypes (distance from the central stimulus) and exemplars (density of the stimulus space). Stimuli consisted of four emotions (anger, disgust, fear, and pleasure) morphed at 25% intervals between a neutral expression and each emotion from 25% to 125%, and between each pair of emotions. Emotion perception was assessed using emotion intensity ratings, valence and arousal ratings, speeded categorisation and paired similarity ratings. We report two key findings: 1) across tasks, there was a strongly linear effect of caricaturing, with caricatured emotions (125%) perceived as higher in emotion intensity and arousal, and recognised faster compared to the original emotion (100%) and anti-caricatures (25%–75%); 2) our results reveal evidence for a unique contribution of a prototype-based account in emotion recognition. We show for the first time that vocal caricature effects are comparable to those found previously with facial caricatures. The set of caricatured vocalisations provided open a promising line of research for investigating vocal affect perception and emotion processing deficits in clinical populations.

Recalibration of vocal affect by a dynamic face

Perception of vocal affect is influenced by the concurrent sight of an emotional face. We demonstrate that the sight of an emotional face also can induce recalibration of vocal affect. Participants were exposed to videos of a ‘happy’ or ‘fearful’ face in combination with a slightly incongruous sentence with ambiguous prosody. After this exposure, ambiguous test sentences were rated as more ‘happy’ when the exposure phase contained ‘happy’ instead of ‘fearful’ faces. This auditory shift likely reflects recalibration that is induced by error minimization of the inter-sensory discrepancy. In line with this view, when the prosody of the exposure sentence was non-ambiguous and congruent with the face (without audiovisual discrepancy), aftereffects went in the opposite direction, likely reflecting adaptation. Our results demonstrate, for the first time, that perception of vocal affect is flexible and can be recalibrated by slightly discrepant visual information.

Processing emotions in sounds: cross-domain aftereffects of vocal utterances and musical sounds

ABSTRACTNonlinguistic signals in the voice and musical instruments play a critical role in communicating emotion. Although previous research suggests a common mechanism for emotion processing in music and speech, the precise relationship between the two domains is unclear due to the paucity of direct evidence. By applying the adaptation paradigm developed by Bestelmeyer, Rouger, DeBruine, and Belin [2010. Auditory adaptation in vocal affect perception. Cognition, 117(2), 217–223. doi:10.1016/j.cognition.2010.08.008], this study shows cross-domain aftereffects from vocal to musical sounds. Participants heard an angry or fearful sound four times, followed by a test sound and judged whether the test sound was angry or fearful. Results show cross-domain aftereffects in one direction – vocal utterances to musical sounds, not vice-versa. This effect occurred primarily for angry vocal sounds. It is argued that there is a unidirectional relationship between vocal and musical sounds where emotion processing of vocal sounds encompasses musical sounds but not vice-versa.

Facial and vocal affect perception in people at ultra‐high risk of psychosis, first‐episode schizophrenia and healthy controls

The study aims to investigate affect recognition in young people at different stages of psychotic illness. Seventy-nine ultra-high risk patients, 30 first-episode schizophrenia patients and 30 healthy control subjects completed a facial affect labelling test and an affective prosody recognition test. Psychiatric symptoms were assessed using the Positive and Negative Syndrome Scale (PANSS). We observed significant impairments in facial and vocal emotion recognition in both of the clinical groups compared with the control group. These group differences remained significant when age, sex and education were taken into account. The findings suggest that emotion recognition impairments may be independent of the stage of illness in schizophrenia. Deficits in emotion recognition may be present before the full expression of psychotic illness, and may contribute to the social cognition and social functioning deficits apparent in emerging psychotic disorders.

Not Pitch Perfect: Sensory Contributions to Affective Communication Impairment in Schizophrenia

Schizophrenia patients have vocal affect (prosody) deficits that are treatment resistant and associated with negative symptoms and poor outcome. The neural correlates of this dysfunction are unclear. Prior study has suggested that schizophrenia vocal affect perception deficits stem from an inability to use acoustic cues, notably pitch, in decoding emotion. Functional magnetic resonance imaging was performed in 24 schizophrenia patients and 28 healthy control subjects, during the performance of a four-choice (happiness, fear, anger, neutral) vocal affect identification task in which items for each emotion varied parametrically in affective salient acoustic cue levels. We observed that parametric increases in cue levels in schizophrenia failed to produce the same identification rate increases as in control subjects. These deficits correlated with diminished reciprocal activation changes in superior temporal and inferior frontal gyri and reduced temporo-frontal connectivity. Task activation also correlated with independent measures of pitch perception and negative symptom severity. These findings illustrate the interplay between sensory and higher-order cognitive dysfunction in schizophrenia. Sensory contributions to vocal affect deficits also suggest that this neurobehavioral marker could be targeted by pharmacological or behavioral remediation of acoustic feature discrimination.

Auditory adaptation in vocal affect perception

Previous research has demonstrated perceptual aftereffects for emotionally expressive faces, but the extent to which they can also be obtained in a different modality is unknown. In two experiments we show for the first time that adaptation to affective, non-linguistic vocalisations elicits significant auditory aftereffects. Adaptation to angry vocalisations caused voices drawn from an anger–fear morphed continuum to be perceived as less angry and more fearful, while adaptation to fearful vocalisations elicited opposite aftereffects (Experiment 1). We then tested the link between these aftereffects and the underlying acoustics by using caricatured adaptors. Although caricatures exaggerated the acoustical and affective properties of the vocalisations, the caricatured adaptors resulted in aftereffects which were comparable to those obtained with natural vocalisations (Experiment 2). Our findings suggest that these aftereffects cannot be solely explained by low-level adaptation to acoustical characteristics of the adaptors but are likely to depend on higher-level adaptation of neural representations of vocal affect.

Perception of emotions from faces and voices following unilateral brain damage

The importance of the right hemisphere in emotion perception in general has been well documented but its precise role is disputed. We compared the performance of 30 right hemisphere damaged (RHD) patients, 30 left hemisphere damaged (LHD) patients, and 50 healthy controls on both facial and vocal affect perception tasks of specific emotions. Brain damaged subjects had a single episode cerebrovascular accident localised to one hemisphere. The results showed that right hemisphere patients were markedly impaired relative to left hemisphere and healthy controls on test performance: labelling and recognition of facial expressions and recognition of emotions conveyed by prosody. This pertained at the level of individual basic emotions, positive versus negative, and emotional expressions in general. The impairment remained highly significant despite covarying for the group’s poorer accuracy on a neutral facial perception test and identification of neutral vocal expressions. The LHD group were only impaired relative to controls on facial emotion tasks when their performance was summed over all the emotion categories and before age and other cognitive factors were taken into account. However, on the prosody test the LHD patients showed significant impairment, performing mid-way between the right hemisphere patients and healthy comparison group. Recognition of positive emotional expressions was better than negative in all subjects, and was not relatively poorer in the LHD patients. Recognition of individual emotions in one modality correlated weakly with recognition in another, in all three groups. These data confirm the primacy of the right hemisphere in processing all emotional expressions across modalities—both positive and negative—but suggest that left hemisphere emotion processing is modality specific. It is possible that the left hemisphere has a particular role in the perception of emotion conveyed through meaningful speech.

Right cerebral hemisphere advantage in perception of vocal affect in schizophrenia

Right cerebral hemisphere advantage in perception of vocal affect in schizophrenia

Gender stereotypes in the expression and perception of vocal affect

This study examined the perception and expression of vocal affect for behavioral evidence of gender stereotypes. Subjects were 97% Caucasian and were asked to either identify or rate the effectiveness of vocal affect portrayals for 5 emotions (fear, anger, happiness, sadness, and neutral) as portrayed by 3 female and 3 male actors reading 2 stories with nonemotional word content. The results revealed that female judges identified fear, happiness, and sadness better than males. Male actors' portrayals were identified better than female actors' portrayals of anger and fear. Female actors received higher identification rates than male actors for portrayals of happiness. Partial support for the general hypothesis that there is behavioral evidence of stereotypic gender differences for vocal affect was obtained.