Neutral Prosody Research Articles

Developmental trends of perceiving happy voice in preverbal infants

Perception of emotional prosody is essential for children to identify emotions of communication partners from utterances. Newborns prefer listening to happy talks but children start to weight emotional prosody over lexical meanings to judge emotions in speech around middle childhood. Happy voice is expressed with higher fundamental frequency (F0) and F0 variations in syllables manifest lexical tones in Mandarin. Would lexical tones affect happy prosody perception development in preverbal infants? The goal of this study was to explore whether the ability of perceiving happy prosody varies with lexical tones. Mandarin-learning children master the production of Tone 2 earlier than Tone 3, and development trend of lexical tone production might reflect perceptual trends of lexical tones. Thirty-nine Mandarin-learning 7- and 11-month-old infants were tested with two prosody (happy vs. neutral prosody) contrasts in a speech discrimination task. For each contrast, speech stimuli were monosyllabic words /ji/, varied with emotional prosody. Tonal information (Tone 2 or Tone 3) varied between contrasts and remained the same within a contrast. Results showed that older infants were more accurate than younger infants to distinguish happy prosody and the pitch contours of lexical tones did not greatly shift the developmental trend.

Loud and angry: sound intensity modulates amygdala activation to angry voices in social anxiety disorder.

Angry expressions of both voices and faces represent disorder-relevant stimuli in social anxiety disorder (SAD). Although individuals with SAD show greater amygdala activation to angry faces, previous work has failed to find comparable effects for angry voices. Here, we investigated whether voice sound-intensity, a modulator of a voice’s threat-relevance, affects brain responses to angry prosody in SAD. We used event-related functional magnetic resonance imaging to explore brain responses to voices varying in sound intensity and emotional prosody in SAD patients and healthy controls (HCs). Angry and neutral voices were presented either with normal or high sound amplitude, while participants had to decide upon the speaker’s gender. Loud vs normal voices induced greater insula activation, and angry vs neutral prosody greater orbitofrontal cortex activation in SAD as compared with HC subjects. Importantly, an interaction of sound intensity, prosody and group was found in the insula and the amygdala. In particular, the amygdala showed greater activation to loud angry voices in SAD as compared with HC subjects. This finding demonstrates a modulating role of voice sound-intensity on amygdalar hyperresponsivity to angry prosody in SAD and suggests that abnormal processing of interpersonal threat signals in amygdala extends beyond facial expressions in SAD.

Emotional prosody effects on verbal memory in older and younger adults

ABSTRACTProsody, or the way things are said, can modify the meaning of utterances making qualitatively different affective prosodies useful for understanding how auditory affective information is processed and remembered. In this study, we collected behavioral data from 225 younger (M age = 20.8 years, SD = 2.5 years; 119 males) and 225 older adults (M age = 71.6 years, SD = 6.5 years; 119 males) in order to examine age differences in emotional prosody effects on verbal memory. Participants were randomly divided into three subgroups according to different prosody listening conditions (positive, negative, and neutral) and prosody effects on a yes–no recognition memory task were investigated. The results showed how older adults who listened to the story read with a neutral prosody remembered more words than those who listened to the same story with a positive or negative prosody. Younger adults showed no valence effects. Our findings highlighted an age and affective prosody interaction that affects remembering in older adults alone.

The Voice of Anger: Oscillatory EEG Responses to Emotional Prosody.

Emotionally relevant stimuli and in particular anger are, due to their evolutionary relevance, often processed automatically and able to modulate attention independent of conscious access. Here, we tested whether attention allocation is enhanced when auditory stimuli are uttered by an angry voice. We recorded EEG and presented healthy individuals with a passive condition where unfamiliar names as well as the subject’s own name were spoken both with an angry and neutral prosody. The active condition instead, required participants to actively count one of the presented (angry) names. Results revealed that in the passive condition the angry prosody only elicited slightly stronger delta synchronization as compared to a neutral voice. In the active condition the attended (angry) target was related to enhanced delta/theta synchronization as well as alpha desynchronization suggesting enhanced allocation of attention and utilization of working memory resources. Altogether, the current results are in line with previous findings and highlight that attention orientation can be systematically related to specific oscillatory brain responses. Potential applications include assessment of non-communicative clinical groups such as post-comatose patients.

Neural basis of processing threatening voices in a crowded auditory world.

In real world situations, we typically listen to voice prosody against a background crowded with auditory stimuli. Voices and background can both contain behaviorally relevant features and both can be selectively in the focus of attention. Adequate responses to threat-related voices under such conditions require that the brain unmixes reciprocally masked features depending on variable cognitive resources. It is unknown which brain systems instantiate the extraction of behaviorally relevant prosodic features under varying combinations of prosody valence, auditory background complexity and attentional focus. Here, we used event-related functional magnetic resonance imaging to investigate the effects of high background sound complexity and attentional focus on brain activation to angry and neutral prosody in humans. Results show that prosody effects in mid superior temporal cortex were gated by background complexity but not attention, while prosody effects in the amygdala and anterior superior temporal cortex were gated by attention but not background complexity, suggesting distinct emotional prosody processing limitations in different regions. Crucially, if attention was focused on the highly complex background, the differential processing of emotional prosody was prevented in all brain regions, suggesting that in a distracting, complex auditory world even threatening voices may go unnoticed.

Emotional language is all right: Emotional prosody reduces hemispheric asymmetry for linguistic processing

ABSTRACTIn an influential paper, Bryden and MacRae [(1989). Dichotic laterality effects obtained with emotional words. Neuropsychiatry, Neuropsychology, and Behavioral Neurology, 1, 171–176] introduced a dichotic listening task that allowed for the assessment of linguistic and prosodic processing asymmetries using the same stimuli. The task produces a robust right ear advantage (REA) for linguistic processing (identifying a target word), and a left ear advantage for emotional processing (identifying a target prosody). Here, we adapted this paradigm to determine whether and how the presence of emotional prosody might modulate hemispheric asymmetry for linguistic processing. Participants monitored for a target word among dichotic stimuli consisting of two different words, but spoken in the same emotional prosody—neutral, angry, happy, sad, or fearful. A strong REA was observed when the words were spoken in neutral prosody, which was attenuated for all the emotional prosodies. There were no differences in the ear advantage as a function of valence or discrete emotion, indicating that all emotions had similar effects. Findings are consistent with the hypothesis that the right hemisphere is better able to process speech when it carries emotional prosody. Implications for understanding of right hemisphere language functions are discussed.

The effects of sad prosody on hemispheric specialization for words processing

This study examined the effect of sad prosody on hemispheric specialization for word processing using behavioral and electrophysiological measures. A dichotic listening task combining focused attention and signal-detection methods was conducted to evaluate the detection of a word spoken in neutral or sad prosody. An overall right ear advantage together with leftward lateralization in early (150–170ms) and late (240–260ms) processing stages was found for word detection, regardless of prosody. Furthermore, the early stage was most pronounced for words spoken in neutral prosody, showing greater negative activation over the left than the right hemisphere. In contrast, the later stage was most pronounced for words spoken with sad prosody, showing greater positive activation over the left than the right hemisphere. The findings suggest that sad prosody alone was not sufficient to modulate hemispheric asymmetry in word-level processing. We posit that lateralized effects of sad prosody on word processing are largely dependent on the psychoacoustic features of the stimuli as well as on task demands.

Neural Processing of Emotional Prosody across the Adult Lifespan.

Emotion recognition deficits emerge with the increasing age, in particular, a decline in the identification of sadness. However, little is known about the age-related changes of emotion processing in sensory, affective, and executive brain areas. This functional magnetic resonance imaging (fMRI) study investigated neural correlates of auditory processing of prosody across adult lifespan. Unattended detection of emotional prosody changes was assessed in 21 young (age range: 18–35 years), 19 middle-aged (age range: 36–55 years), and 15 older (age range: 56–75 years) adults. Pseudowords uttered with neutral prosody were standards in an oddball paradigm with angry, sad, happy, and gender deviants (total 20% deviants). Changes in emotional prosody and voice gender elicited bilateral superior temporal gyri (STG) responses reflecting automatic encoding of prosody. At the right STG, responses to sad deviants decreased linearly with age, whereas happy events exhibited a nonlinear relationship. In contrast to behavioral data, no age by sex interaction emerged on the neural networks. The aging decline of emotion processing of prosodic cues emerges already at an early automatic stage of information processing at the level of the auditory cortex. However, top-down modulation may lead to an additional perceptional bias, for example, towards positive stimuli, and may depend on context factors such as the listener's sex.

Carrot and stick 2.0: The benefits of natural and motivational prosody in computer-assisted learning

For acquiring new skills or knowledge, contemporary learners frequently rely on the help of educational technologies supplementing human teachers as a learning aid. In the interaction with such systems, speech-based communication between the human user and the technical system has increasingly gained importance. Since spoken computer output can take on a variety of forms depending on the method of speech generation and the employment of prosodic modulations, the effects of such auditory variations on the user’s learning achievement require systematic investigation. The experiment reported here examined the specific effects of speech generation method and prosody of spoken system feedback in a computer-supported learning environment, and may serve as validational tool for future investigations of spoken computer feedback effects on learning. Learning performance in a basic cognitive task was compared between users receiving pre-recorded, naturally spoken system feedback with neutral prosody, pre-recorded feedback with motivating (praising or blaming) prosody, or computer-synthesized feedback. The observed results provide empirical evidence that users of technical tutoring systems benefit from pre-recorded, naturally spoken feedback, and do even more so from feedback with motivational prosodic modulations matching their performance success. Theoretical implications and considerations for future implementations of spoken feedback in computer-based educational systems are discussed.

Affective prosody perception in symptomatically remitted patients with schizophrenia and bipolar disorder

Affect perception has frequently been shown to be impaired in patients suffering from schizophrenia or bipolar disorder (BD), but it remains unclear whether these impairments exist during symptomatic remission and whether the two disorders differ from each other in this regard. Most previous studies have investigated facial affect recognition, but not the ability to decode mental states from emotional tone of voice, i.e. affective prosody perception (APP). Accordingly, the present study directly compared APP in symptomatically remitted patients with schizophrenia or BD and healthy control subjects and investigated its relationship with residual symptomatology in patients.Patients with schizophrenia and BD showed comparable APP impairments despite being symptomatically remitted. In comparison to healthy control subjects, overall APP deficits were found in BD but not in schizophrenia patients. Both patient groups were particularly impaired in the identification of anger and confounded it with neutral prosody. In addition, schizophrenia patients frequently confused sadness with happiness, anger, or fright. There was an inverse association between the degree of residual positive symptoms and the ability to correctly recognize happiness in schizophrenia patients.Overall, these data indicate that impairments in APP represent an enduring deficit and a trait marker of both schizophrenia and BD and that the level of impairment is comparable between disorders.

Auditory temporal structure processing in dyslexia: processing of prosodic phrase boundaries is not impaired in children with dyslexia

Reading disability in children with dyslexia has been proposed to reflect impairment in auditory timing perception. We investigated one aspect of timing perception--temporal grouping--as present in prosodic phrase boundaries of natural speech, in age-matched groups of children, ages 6-8 years, with and without dyslexia. Prosodic phrase boundaries are characterized by temporal grouping of functionally related speech elements and can facilitate syntactic processing of speech. For example, temporary syntactic ambiguities, such as early-closure structures, are processed faster when prosodic phrase boundaries are present. We examined children's prosodic facilitation by measuring their efficiency of sentence processing for temporary syntactic ambiguities spoken with (facilitating) versus without (neutral) prosodic phrase boundaries. Both groups of children benefited similarly from prosodic facilitation, displaying faster reaction times in facilitating compared to neutral prosody. These findings indicate that the use of prosodic phrase boundaries for speech processing is not impaired in children with dyslexia.

Unimpaired discrimination of fearful prosody after amygdala lesion

Prosody (i.e. speech melody) is an important cue to infer an interlocutor's emotional state, complementing information from face expression and body posture. Inferring fear from face expression is reported as impaired after amygdala lesions. It remains unclear whether this deficit is specific to face expression, or is a more global fear recognition deficit. Here, we report data from two twins with bilateral amygdala lesions due to Urbach-Wiethe syndrome and show they are unimpaired in a multinomial emotional prosody classification task. In a two-alternative forced choice task, they demonstrate increased ability to discriminate fearful and neutral prosody, the opposite of what would be expected under an hypothesis of a global role for the amygdala in fear recognition. Hence, we provide evidence that the amygdala is not required for recognition of fearful prosody.

Blunted feelings: Alexithymia is associated with a diminished neural response to speech prosody

How we perceive emotional signals from our environment depends on our personality. Alexithymia, a personality trait characterized by difficulties in emotion regulation has been linked to aberrant brain activity for visual emotional processing. Whether alexithymia also affects the brain's perception of emotional speech prosody is currently unknown. We used functional magnetic resonance imaging to investigate the impact of alexithymia on hemodynamic activity of three a priori regions of the prosody network: the superior temporal gyrus (STG), the inferior frontal gyrus and the amygdala. Twenty-two subjects performed an explicit task (emotional prosody categorization) and an implicit task (metrical stress evaluation) on the same prosodic stimuli. Irrespective of task, alexithymia was associated with a blunted response of the right STG and the bilateral amygdalae to angry, surprised and neutral prosody. Individuals with difficulty describing feelings deactivated the left STG and the bilateral amygdalae to a lesser extent in response to angry compared with neutral prosody, suggesting that they perceived angry prosody as relatively more salient than neutral prosody. In conclusion, alexithymia may be associated with a generally blunted neural response to speech prosody. Such restricted prosodic processing may contribute to problems in social communication associated with this personality trait.

GMM-Based Emotional Voice Conversion Using Spectrum and Prosody Features

We propose Gaussian Mixture Model (GMM)-based emotional voice conversion using spectrum and prosody features. In recent years, speech recognition and synthesis techniques have been developed, and an emotional voice conversion technique is required for synthesizing more expressive voices. The common emotional conversion was based on transformation of neutral prosody to emotional prosody by using huge speech corpus. In this paper, we convert a neutral voice to an emotional voice using GMMs. GMM-based spectrum conversion is widely used to modify non linguistic information such as voice characteristics while keeping linguistic information unchanged. Because the conventional method converts either prosody or voice quality (spectrum), some emotions are not converted well. In our method, both prosody and voice quality are used for converting a neutral voice to an emotional voice, and it is able to obtain more expressive voices in comparison with conventional methods, such as prosody or spectrum conversion.

Processing of angry voices is modulated by visual load

Visual perceptual load has been shown to modulate brain activation to emotional facial expressions. However, it is unknown whether cross-modal effects of visual perceptual load on brain activation to threat-related auditory stimuli also exist. The current fMRI study investigated brain responses to angry and neutral voices while subjects had to solve an easy or a demanding visual task. Although the easy visual condition was associated with increased activation in the right superior temporal region to angry vs. neutral prosody, this effect was absent during the demanding task. Thus, our results show that cross-modal perceptual load modulates the activation to emotional voices in the auditory cortex and that high visual load prevents the increased processing of emotional prosody.

Isolating N400 as neural marker of vocal anger processing in 6–11-year old children

IntroductionVocal anger is a salient social signal serving adaptive functions in typical child development. Despite recent advances in the developmental neuroscience of emotion processing with regard to visual stimuli, little remains known about the neural correlates of vocal anger processing in childhood. This study represents the first attempt to isolate a neural marker of vocal anger processing in children using electrophysiological methods. MethodsWe compared ERP wave forms during the processing of non-word emotional vocal stimuli in a population sample of 55 6–11-year-old typically developing children. Children listened to three types of stimuli expressing angry, happy, and neutral prosody and completed an emotion identification task with three response options (angry, happy and neutral/‘ok’). ResultsA distinctive N400 component which was modulated by emotional content of vocal stimulus was observed in children over parietal and occipital scalp regions—amplitudes were significantly attenuated to angry compared to happy and neutral voices. DiscussionFindings of the present study regarding the N400 are compatible with adult studies showing reduced N400 amplitudes to negative compared to neutral emotional stimuli. Implications for studies of the neural basis of vocal anger processing in children are discussed.

Relations entre prosodie et syntaxe

This paper addresses the question of the interface between prosody and syntax through the analysis of a few non canonical syntactic structures taken from corpora of spoken English. It-clefts, extrapositions, right noun-phrase dislocations and the insertion of auxiliary do in an assertive context are closely looked at. The prosody of these structures is compared to marked prosodic forms of emphatic utterances with a neutral syntax. The prosodic analysis is mainly based on the number of tone units, the place of the nuclear syllable and the pitch movement. The context and the information structure are also taken into account for the utterances analysed here. We show that a syntactically non canonical utterance can be pronounced with a neutral prosody or on the contrary that the prosody can be marked. Prosody and syntax are complementary but play a role at different levels. It is argued that a non neutral syntax is rather used for the organization of discourse, for thematization or focalisation, while prosody is used to emphasize an element or highlight a contrast when it is non neutral, and plays a role on the pragmatic or semantic level when it is neutral.

Specific Brain Networks during Explicit and Implicit Decoding of Emotional Prosody

To better define the underlying brain network for the decoding of emotional prosody, we recorded high-resolution brain scans during an implicit and explicit decoding task of angry and neutral prosody. Several subregions in the right superior temporal gyrus (STG) and bilateral in the inferior frontal gyrus (IFG) were sensitive to emotional prosody. Implicit processing of emotional prosody engaged regions in the posterior superior temporal gyrus (pSTG) and bilateral IFG subregions, whereas explicit processing relied more on mid STG, left IFG, amygdala, and subgenual anterior cingulate cortex. Furthermore, whereas some bilateral pSTG regions and the amygdala showed general sensitivity to prosody-specific acoustical features during implicit processing, activity in inferior frontal brain regions was insensitive to these features. Together, the data suggest a differentiated STG, IFG, and subcortical network of brain regions, which varies with the levels of processing and shows a higher specificity during explicit decoding of emotional prosody.

Visual attention modulates brain activation to angry voices.

In accordance with influential models proposing prioritized processing of threat, previous studies have shown automatic brain responses to angry prosody in the amygdala and the auditory cortex under auditory distraction conditions. However, it is unknown whether the automatic processing of angry prosody is also observed during cross-modal distraction. The current fMRI study investigated brain responses to angry versus neutral prosodic stimuli during visual distraction. During scanning, participants were exposed to angry or neutral prosodic stimuli while visual symbols were displayed simultaneously. By means of task requirements, participants either attended to the voices or to the visual stimuli. While the auditory task revealed pronounced activation in the auditory cortex and amygdala to angry versus neutral prosody, this effect was absent during the visual task. Thus, our results show a limitation of the automaticity of the activation of the amygdala and auditory cortex to angry prosody. The activation of these areas to threat-related voices depends on modality-specific attention.

Emotional Voice Areas: Anatomic Location, Functional Properties, and Structural Connections Revealed by Combined fMRI/DTI

We determined the location, functional response profile, and structural fiber connections of auditory areas with voice- and emotion-sensitive activity using functional magnetic resonance imaging (fMRI) and diffusion tensor imaging. Bilateral regions responding to emotional voices were consistently found in the superior temporal gyrus, posterolateral to the primary auditory cortex. Event-related fMRI showed stronger responses in these areas to voices-expressing anger, sadness, joy, and relief, relative to voices with neutral prosody. Their neural responses were primarily driven by prosodic arousal, irrespective of valence. Probabilistic fiber tracking revealed direct structural connections of these "emotional voice areas" (EVA) with ipsilateral medial geniculate body, which is the major input source of early auditory cortex, as well as with the ipsilateral inferior frontal gyrus (IFG) and inferior parietal lobe (IPL). In addition, vocal emotions (compared with neutral prosody) increased the functional coupling of EVA with the ipsilateral IFG but not IPL. These results provide new insights into the neural architecture of the human voice processing system and support a crucial involvement of IFG in the recognition of vocal emotions, whereas IPL may subserve distinct auditory spatial functions, consistent with distinct anatomical substrates for the processing of "how" and "where" information within the auditory pathways.