Prosody Processing Research Articles

Functional alterations of lateral temporal cortex for processing voice prosody in adults with autism spectrum disorder.

The human auditory system includes discrete cortical patches and selective regions for processing voice information, including emotional prosody. Although behavioral evidence indicates individuals with autism spectrum disorder (ASD) have difficulties in recognizing emotional prosody, it remains understudied whether and how localized voice patches (VPs) and other voice-sensitive regions are functionally altered in processing prosody. This fMRI study investigated neural responses to prosodic voices in 25 adult males with ASD and 33 controls using voices of anger, sadness, and happiness with varying degrees of emotion. We used a functional region-of-interest analysis with an independent voice localizer to identify multiple VPs from combined ASD and control data. We observed a general response reduction to prosodic voices in specific VPs of left posterior temporal VP (TVP) and right middle TVP. Reduced cortical responses in right middle TVP were consistently correlated with the severity of autistic symptoms for all examined emotional prosodies. Moreover, representation similarity analysis revealed the reduced effect of emotional intensity in multivoxel activation patterns in left anterior superior temporal cortex only for sad prosody. These results indicate reduced response magnitudes to voice prosodies in specific TVPs and altered emotion intensity-dependent multivoxel activation patterns in adult ASDs, potentially underlying their socio-communicative difficulties.

A simple psychophysical procedure separates representational and noise components in impairments of speech prosody perception after right-hemisphere stroke

After a right hemisphere stroke, more than half of the patients are impaired in their capacity to produce or comprehend speech prosody. Yet, and despite its social-cognitive consequences for patients, aprosodia following stroke has received scant attention. In this report, we introduce a novel, simple psychophysical procedure which, by combining systematic digital manipulations of speech stimuli and reverse-correlation analysis, allows estimating the internal sensory representations that subtend how individual patients perceive speech prosody, and the level of internal noise that govern behavioral variability in how patients apply these representations. Tested on a sample of N = 22 right-hemisphere stroke survivors and N = 21 age-matched controls, the representation + noise model provides a promising alternative to the clinical gold standard for evaluating aprosodia (MEC): both parameters strongly associate with receptive, and not expressive, aprosodia measured by MEC within the patient group; they have better sensitivity than MEC for separating high-functioning patients from controls; and have good specificity with respect to non-prosody-related impairments of auditory attention and processing. Taken together, individual differences in either internal representation, internal noise, or both, paint a potent portrait of the variety of sensory/cognitive mechanisms that can explain impairments of prosody processing after stroke.

Aging affects auditory contributions to focus perception in Jianghuai Mandarina).

Speakers can place their prosodic prominence on any locations within a sentence, generating focus prosody for listeners to perceive new information. This study aimed to investigate age-related changes in the bottom-up processing of focus perception in Jianghuai Mandarin by clarifying the perceptual cues and the auditory processing abilities involved in the identification of focus locations. Young, middle-aged, and older speakers of Jianghuai Mandarin completed a focus identification task and an auditory perception task. The results showed that increasing age led to a decrease in listeners' accuracy rate in identifying focus locations, with all participants performing the worst when dynamic pitch cues were inaccessible. Auditory processing abilities did not predict focus perception performance in young and middle-aged listeners but accounted significantly for the variance in older adults' performance. These findings suggest that age-related deteriorations in focus perception can be largely attributed to declined auditory processing of perceptual cues. Poor ability to extract frequency modulation cues may be the most important underlying psychoacoustic factor for older adults' difficulties in perceiving focus prosody in Jianghuai Mandarin. The results contribute to our understanding of the bottom-up mechanisms involved in linguistic prosody processing in aging adults, particularly in tonal languages.

Age-related decline in hearing and emotional prosody processing: A multi-feature oddball study

This study investigated how age and hearing loss contributed to the challenges in perceiving emotional prosody in speech. A multi-feature oddball paradigm was employed to assess mismatch negativity (MMN) and P3a responses to three deviant prosodies (happy, angry and sad) against the standard neutral prosody in spoken words. The participants were 22 adults in the age range of 18 to 70 with various degrees of hearing loss as assessed with pure tone audiometry. Linear mixed effects models revealed heightened MMN response to angry voice and strongest P3a response to happy voice among the three deviants. There was a significant positive correlation between age and MMN amplitude for the angry prosody. Significant correlations were also observed between hearing loss and MMN amplitude in response to both angry and happy stimuli. In addition, negative correlations were found between P3a amplitude and hearing loss as well as aging for happy stimuli. However, aging and hearing loss did not significantly affect the processing of sadness in this paradigm. These findings showed category-sensitive age-related decline in emotional prosody processing. The intricate correlations patterns among age, hearing loss, MMN, and P3a responses called for further exploration of broader implications in cognitive aging.

Infant-directed speech facilitates word learning through attentional mechanisms: An fNIRS study of toddlers.

The speech register that adults especially caregivers use when interacting with infants and toddlers, that is, infant-directed speech (IDS) or baby talk, has been reported to facilitate language development throughout the early years. However, the neural mechanisms as well as why IDS results in such a developmental faciliatory effect remain to be investigated. The current study uses functional near-infrared spectroscopy (fNIRS) to evaluate two alternative hypotheses of such a facilitative effect, that IDS serves to enhance linguistic contrastiveness or to attract the child's attention. Behavioral and fNIRS data were acquired from twenty-seven Cantonese-learning toddlers 15-20 months of age when their parents spoke to them in either an IDS or adult-directed speech (ADS) register in a naturalistic task in which the child learned four disyllabic pseudowords. fNIRS results showed significantly greater neural responses to IDS than ADS register in the left dorsolateral prefrontal cortex (L-dlPFC), but opposite response patterns in the bilateral inferior frontal gyrus (IFG). The differences in fNIRS responses to IDS and to ADS in the L-dlPFC and the left parietal cortex (L-PC) showed significantly positive correlations with the differences in the behavioral word-learning performance of toddlers. The same fNIRS measures in the L-dlPFC and right PC (R-PC) of toddlers were significantly correlated with pitch range differences of parents between the two speech conditions. Together, our results suggest that the dynamic prosody in IDS increased toddlers' attention through greater involvement of the left frontoparietal network that facilitated word learning, compared to ADS. RESEARCH HIGHLIGHTS: This study for the first time examined the neural mechanisms of how infant-directed speech (IDS) facilitates word learning in toddlers. Using fNIRS, we identified the cortical regions that were directly involved in IDS processing. Our results suggest that IDS facilitates word learning by engaging a right-lateralized prosody processing and top-down attentional mechanisms in the left frontoparietal networks. The language network including the inferior frontal gyrus and temporal cortex was not directly involved in IDS processing to support word learning.

Toward processing of prosody in spontaneous Japanese

This paper considers how prosody in spontaneous Japanese is processed. We have conducted Rapid Prosody Transcription (RPT) perception experiments on the Corpus of Spontaneous Japanese (CSJ) and investigated how boundaries and prominences are perceived. We recruited three groups of participants from different Japanese dialects and found that (i) F0 is not a strong prominence cue in Japanese, contra Japanese literature on focus prominence (Pierrehumbert & Beckman (P&B) 1988; Kori 1989; Ishihara 2016) and (ii) Japanese allows multi-headed and headless intonation phrases, and P&B’s reset theory, i.e. focal prominence resets boundary phrases, faces empirical difficulties. We also found that (iii) both content words and function morphemes get highlighted in Japanese, and (iv) perception strategies vary cross-dialectally and listeners from different dialects perceive boundaries and prominences differently.

The Development of Left Hemisphere Lateralization for Sentence-Level Prosodic Processing.

The fine-tuning of linguistic prosody in later childhood is poorly understood, and its neurological processing is even less well studied. In particular, it is unknown if grammatical processing of prosody is left- or right-lateralized in childhood versus adulthood and how phonological working memory might modulate such lateralization. Furthermore, it is virtually unknown how prosody develops neurologically among children with cochlear implants (CIs). Normal-hearing (NH) children ages 6-12 years and NH adults ages 18-28 years completed a functional near-infrared spectroscopy neuroimaging task, during which they heard sentence pairs and judged whether the sentences did or did not differ in their overall prosody (declarative, question, with or without narrow focus). Children also completed standard measures of expressive and receptive language. Age group differences emerged; children exhibited stronger bilateral temporoparietal activity but reduced left frontal activation. Furthermore, children's performance on a nonword repetition test was significantly associated with activation in the left inferior frontal gyrus-an area that was generally more activated in adults than in children. The prosody-related findings are generally consistent with prior neurodevelopmental works on sentence comprehension, especially those involving syntax and semantics, which have also noted a developmental shift from bilateral temporal to left inferior frontal regions typically associated with increased sensitivity to sentence structure. The findings thus inform theoretical perspectives on brain and language development and have implications for studying the effects of CIs on neurodevelopmental processes for sentence prosody. https://doi.org/10.23641/asha.22255996.

The disruptive effects of changing-state sound and emotional prosody on verbal short-term memory in blind, visually impaired, and sighted listeners

ABSTRACT Previous findings suggest that blind listeners are less susceptible to auditory distraction in a verbal serial recall task, compared to sighted individuals. However, it is unclear whether this is due to more selective encoding / filtering of auditory information (e.g. efficient perceptual streaming) or to enhanced attentional control. To test these alternatives, the interference induced by changing-state sound was compared with the disruptive effect of emotional speech prosody (happy, angry, fearful intonations) in blind (n = 17), visually impaired (n = 23), and sighted (n = 89) individuals. While a reduced changing-state effect suggests more efficient perceptual filtering, a reduced emotional prosody effect indicates enhanced attentional control. Blind participants were able to recall more items compared to sighted and visually impaired individuals. Moreover, in sighted and visually impaired participants, the changing-state effect was found to be enhanced with angry prosody, whereas blind individuals were less susceptible to changing-state sound regardless of the prosody. The results also suggest group differences in prosody processing, as both visually impaired and blind participants were better able to ignore fearful speech, compared to other prosodies. The findings suggest that profound visual sensory deprivation leads not only to a capacity increment in verbal short-term memory, but enables listeners to shield memory against auditory distracters, suggesting more efficient perceptual streaming.

The influence of musical abilities on the processing of second language prosody: An eye-tracking study

The perception of speech prosody in a second language (L2) remains challenging even for proficient L2 users. Eye-tracking evidence indicates that Dutch listeners show difficulty in the processing of pitch accents that signal a contrast (i.e., contrastive focus) in English, whereas native English listeners use this cue in perception to anticipate upcoming information [Ge et al., Appl. Psycholinguist. 42, 1057–1088 (2021)]. Prosody perception abilities in foreign languages have been associated with individual differences in musical abilities [Jansen et al., Speech Prosody, 713–717 (2022)]. We investigated whether musical abilities influenced the processing of prosodic cues by 45 Dutch adult L2 English users, using a visual-world eye-tracking paradigm. Participants listened to sentences with pitch accent cues to contrastive focus on different words, while viewing pictures showing objects and characters mentioned. We investigated to what extent participants showed anticipatory fixations on the image reflecting the alternative of the contrast, after hearing the accented word. We measured participants’ music perception abilities with a standardised test and analysed its influence on anticipatory fixations. Initial analyses indicate that individuals with stronger musical abilities show more anticipation. This suggests that having stronger perceptual resources underlying both music and speech processing [Patel, Front. Psychol. 2, 142 (2011)] may benefit L2 prosody perception.

Perception of Prosody in Hearing-Impaired Individuals and Users of Hearing Assistive Devices: An Overview of Recent Advances.

Prosody perception is an essential component of speech communication and social interaction through which both linguistic and emotional information are conveyed. Considering the importance of the auditory system in processing prosody-related acoustic features, the aim of this review article is to review the effects of hearing impairment on prosody perception in children and adults. It also assesses the performance of hearing assistive devices in restoring prosodic perception. Following a comprehensive online database search, two lines of inquiry were targeted. The first summarizes recent attempts toward determining the effects of hearing loss and interacting factors such as age and cognitive resources on prosody perception. The second analyzes studies reporting beneficial or detrimental impacts of hearing aids, cochlear implants, and bimodal stimulation on prosodic abilities in people with hearing loss. The reviewed studies indicate that hearing-impaired individuals vary widely in perceiving affective and linguistic prosody, depending on factors such as hearing loss severity, chronological age, and cognitive status. In addition, most of the emerging information points to limitations of hearing assistive devices in processing and transmitting the acoustic features of prosody. The existing literature is incomplete in several respects, including the lack of a consensus on how and to what extent hearing prostheses affect prosody perception, especially the linguistic function of prosody, and a gap in assessing prosody under challenging listening situations such as noise. This review article proposes directions that future research could follow to provide a better understanding of prosody processing in those with hearing impairment, which may help health care professionals and designers of assistive technology to develop innovative diagnostic and rehabilitation tools. https://doi.org/10.23641/asha.21809772.

Influence of bodily resonances on emotional prosody perception.

Emotional prosody is defined as suprasegmental and segmental changes in the human voice and related acoustic parameters that can inform the listener about the emotional state of the speaker. While the processing of emotional prosody is well represented in the literature, the mechanism of embodied cognition in emotional voice perception is very little studied. This study aimed to investigate the influence of induced bodily vibrations-through a vibrator placed close to the vocal cords-in the perception of emotional vocalizations. The main hypothesis was that induced body vibrations would constitute a potential interoceptive feedback that can influence the auditory perception of emotions. It was also expected that these effects would be greater for stimuli that are more ambiguous. Participants were presented with emotional vocalizations expressing joy or anger which varied from low-intensity vocalizations, considered as ambiguous, to high-intensity ones, considered as non-ambiguous. Vibrations were induced simultaneously in half of the trials and expressed joy or anger congruently with the voice stimuli. Participants had to evaluate each voice stimulus using four visual analog scales (joy, anger, and surprise, sadness as control scales). A significant effect of the vibrations was observed on the three behavioral indexes-discrimination, confusion and accuracy-with vibrations confusing rather than facilitating vocal emotion processing. Over all, this study brings new light on a poorly documented topic, namely the potential use of vocal cords vibrations as an interoceptive feedback allowing humans to modulate voice production and perception during social interactions.

Production and Comprehension of Prosodic Boundary Marking in Persons With Unilateral Brain Lesions.

Persons with unilateral brain damage in the right hemisphere (RH) or left hemisphere (LH) show limitations in processing linguistic prosody, with yet inconclusive results on their ability to process prosodically marked structural boundaries for syntactic ambiguity resolution. We aimed at systematically investigating production and comprehension of three prosodic cues (f 0 range, final lengthening, and pause) at structural boundaries in coordinate sequences in participants with right hemisphere brain damage (RHDP) and participants with left hemisphere brain damage (LHDP). Twenty RHDP and 15 LHDP participated in our study. Comprehension experiment: Participants and a control group listened to coordinate name sequences with internal grouping by a prosodically marked structural boundary (grouped condition, e.g., "(Gabi und Leni) # und Nina") or without internal grouping (ungrouped condition, e.g., "Gabi und Leni und Nina") and had to identify the target condition. The strength and combinations of prosodic cues in the stimuli were manipulated. Production experiment: Participants were asked to produce coordinate sequences in the two conditions (grouped, ungrouped) in two different tasks: a Reading Aloud and a Repetition experiment. Accuracy of participants' productions was subsequently assessed in a rating study and productions were analyzed with respect to use of prosodic cues. In the Comprehension experiment, RHDP and LHDP had overall lower identification accuracies than unimpaired control participants and LHDP were found to have particular problems with boundary identification when the pause cue was reduced. In production, LHDP and RHDP employed all three prosodic cues for boundary marking, but struggled to clearly mark prosodic boundaries in 28% of all productions. Both groups showed better performance in reading aloud than in repetition. LHDP relied more on using f 0 range and pause duration to prosodically mark structural boundaries, whereas RHDP employed final lengthening more vigorously than LHDP in reading aloud. We conclude that processing of linguistic prosody is affected in RHDP and LHDP, but not completely impaired. Therefore, prosody can serve as a relevant communicative resource. However, it should also be considered as a target area for assessment and treatment in both groups. https://doi.org/10.23641/asha.21637505.

Aberrant Emotional Prosody Circuitry Predicts Social Communication Impairments in Children With Autism

BackgroundEmotional prosody provides acoustical cues that reflect a communication partner’s emotional state and is crucial for successful social interactions. Many children with autism have deficits in recognizing emotions from voices; however, the neural basis for these impairments is unknown. We examined brain circuit features underlying emotional prosody processing deficits and their relationship to clinical symptoms of autism. MethodsWe used an event-related functional magnetic resonance imaging task to measure neural activity and connectivity during processing of sad and happy emotional prosody and neutral speech in 22 children with autism and 21 matched control children (7–12 years old). We employed functional connectivity analyses to test competing theoretical accounts that attribute emotional prosody impairments to either sensory processing deficits in auditory cortex or theory of mind deficits instantiated in the temporoparietal junction (TPJ). ResultsChildren with autism showed specific behavioral impairments for recognizing emotions from voices. They also showed aberrant functional connectivity between voice-sensitive auditory cortex and the bilateral TPJ during emotional prosody processing. Neural activity in the bilateral TPJ during processing of both sad and happy emotional prosody stimuli was associated with social communication impairments in children with autism. In contrast, activity and decoding of emotional prosody in auditory cortex was comparable between autism and control groups and did not predict social communication impairments. ConclusionsOur findings support a social-cognitive deficit model of autism by identifying a role for TPJ dysfunction during emotional prosody processing. Our study underscores the importance of tuning in to vocal-emotional cues for building social connections in children with autism.

Heritability of Sound Processing Deficits in Autism: Neural Insights

Heritability of Sound Processing Deficits in Autism: Neural Insights

Overt and implicit prosody contribute to neurophysiological responses previously attributed to grammatical processing

Recent neurophysiological research suggests that slow cortical activity tracks hierarchical syntactic structure during online sentence processing. Here we tested an alternative hypothesis: electrophysiological activity peaks at constituent phrase as well as sentence frequencies reflect cortical tracking of overt or covert (implicit) prosodic grouping. Participants listened to series of sentences presented in three conditions while electroencephalography (EEG) was recorded. First, prosodic cues in the sentence materials were neutralized. We found an EEG spectral power peak elicited at a frequency that only ‘tagged’ covert, implicit prosodic change, but not any major syntactic constituents. In the second condition, participants listened to a series of sentences with overt prosodic grouping cues that either aligned or misaligned with the syntactic phrasing in the sentences (initial overt prosody trials). Following each overt prosody trial, participants were presented with a second series of sentences lacking overt prosodic cues (instructed prosody trial) and were instructed to imagine the prosodic contour present in the previous, overt prosody trial. The EEG responses reflected an interactive relationship between syntactic processing and prosodic tracking at the frequencies of syntactic constituents (sentences and phrases): alignment of syntax and prosody boosted EEG responses, whereas their misalignment had an opposite effect. This was true for both overt and imagined prosody conditions. We conclude that processing of both overt and covert prosody is reflected in the frequency-tagged neural responses at sentence constituent frequencies. These findings need to be incorporated in any account that aims to identify neural markers reflecting syntactic processing.

Listen to my feelings! How prosody and accent drive the empathic relevance of complaining speech

Interpersonal communication often involves sharing our feelings with others; complaining, for example, aims to elicit empathy in listeners by vocally expressing a speaker's suffering. Despite the growing neuroscientific interest in the phenomenon of empathy, few have investigated how it is elicited in real time by vocal signals (prosody), and how this might be affected by interpersonal factors, such as a speaker's cultural background (based on their accent). To investigate the neural processes at play when hearing spoken complaints, twenty-six French participants listened to complaining and neutral utterances produced by in-group French and out-group Québécois (i.e., French-Canadian) speakers. Participants rated how hurt the speaker felt while their cerebral activity was monitored with electroencephalography (EEG). Principal Component Analysis of Event-Related Potentials (ERPs) taken at utterance onset showed culture-dependent time courses of emotive prosody processing. The high motivational relevance of ingroup complaints increased the P200 response compared to all other utterance types; in contrast, outgroup complaints selectively elicited an early posterior negativity in the same time window, followed by an increased N400 (due to ongoing effort to derive affective meaning from outgroup voices). Ingroup neutral utterances evoked a late negativity which may reflect re-analysis of emotively less salient, but culturally relevant ingroup speech. Results highlight the time-course of neurocognitive responses that contribute to emotive speech processing for complaints, establishing the critical role of prosody as well as social-relational factors (i.e., cultural identity) on how listeners are likely to “empathize” with a speaker.

Different stages of emotional prosody processing in healthy ageing-evidence from behavioural responses, ERPs, tDCS, and tRNS.

Past research suggests that the ability to recognise the emotional intent of a speaker decreases as a function of age. Yet, few studies have looked at the underlying cause for this effect in a systematic way. This paper builds on the view that emotional prosody perception is a multi-stage process and explores which step of the recognition processing line is impaired in healthy ageing using time-sensitive event-related brain potentials (ERPs). Results suggest that early processes linked to salience detection as reflected in the P200 component and initial build-up of emotional representation as linked to a subsequent negative ERP component are largely unaffected in healthy ageing. The two groups show, however, emotional prosody recognition differences: older participants recognise emotional intentions of speakers less well than younger participants do. These findings were followed up by two neuro-stimulation studies specifically targeting the inferior frontal cortex to test if recognition improves during active stimulation relative to sham. Overall, results suggests that neither tDCS nor high-frequency tRNS stimulation at 2mA for 30 minutes facilitates emotional prosody recognition rates in healthy older adults.

Social Communication Predictors in Autism Spectrum Disorder. Theoretical Review

social communication represents one of the main areas of impairment in the case of individuals with autism spectrum disorder (ASD) (APA, 2013). Several studies have investigated predictors in this domain; however, the results are mostly inconsistent or lack clarity. The purpose of this theoretical review is to analyze studies that address vocabulary, prosody, and cognitive flexibility in relation to deficits in pragmatic language, in the case of individuals with ASD. This synthesis was conducted on 18 studies, based on a systematic search of ScienceDirect and NCBI (PubMed and PubMed Central) databases. A number of 1523 participants were included in the analyzed studies. Results indicate that prosody has a major impact on social communication. However, difficulties in processing prosody do not account as the only predictors in the general abilities of language and communication. Regarding the executive functioning, future research is needed to fully understand the relationship between social communication and cognitive flexibility.

THE TREATMENT OF VOCAL STEREOTYPY IN CHILDREN WITH AUTISM SPECTRUM DISORDER

People with autism spectrum disorder (ASD) exhibit different impairments in the domain of auditory processing of pitch, altitude and prosody of sounds and voices from the environment. It is believed that altered sensory processing of these individuals originates from insufficient stimulation from the persons’ surroundings, therefore they have the need to stimulate themselves in a way of emitting vocal stereotypes of different forms. Having that in mind, the aim of this literature review is to present interventions used in reducing vocal stereotypy in children with ASD, as well as to examine their efficiency. For literature search engines Google Scholar, SCIndex, ProQuest and Serbian Library Consortium for Coordinated Acquisition – KoBson were used. Original research articles were searched in Serbian and English language. Literature review focused on interventions whose primary aim was reducing vocal stereotypy with the self-stimulatory function in participants diagnosed with ASD. A total number of participants in all articles was nine, whose mean age was 7.1 years old. Different procedures were implemented and in 34.8 sessions on average these procedures and their combination led to an 86.5% of success in reducing vocal stereotypy on average. Having the success of mentioned interventions in mind, it is necessary to educate special educators from our region on how to implement them and introduce these procedures in curriculum of all the Faculties which educate future special educators who will work with children diagnosed with ASD. Key words: Vocal stereotypy, auto stimulation, self-stimulation, autism, treatment.

Brain oscillatory patterns of affective prosody perception in children with autism spectrum disorder

BackgroundParalinguistic features, such as prosody (tempo, loudness, and timbre), are an essential marker of a speaker’s emotional state. Abnormal processing of emotional prosody may result in the deficient social behavior associated with autism spectrum disorder (ASD). MethodTwo groups of children participated in our study: the ASD group consisted of 30 preschoolers from 4 to 6 years of age and 24 typically developing (TD) peers. An electroencephalogram (EEG) was acquired in response to a combination of syllables uttered with the following types of emotional prosody: joy, anger, sadness, fear, and calmness. ResultsChildren with ASD and TD showed a similar EEG oscillatory response to fear and anger prosodies. Significant group differences in power spectral density (PSD) were detected for sad and joy intonations. The PSD differences between pairs of intonations, such as joyful and sad, sad and neutral, or joyful and neutral, were significantly higher in the control group than in the ASD group. EEG responses to affective prosody also demonstrated less hemispheric asymmetry in the ASD than in the TD group. ConclusionsOur results suggest that difficulties in emotional prosody recognition in autistic children could be based on the atypical processing of specific acoustic features coding differences between sad, neutral, and joyful intonations and could underlie emotional perception deficits in individuals with ASD.