Speech Prosody Research Articles

Successful lexical tone production of Mandarin Chinese autistic children with intellectual impairment.

Atypical speech prosody has been commonly found among autistic children. Yet it remains unknown whether prosody impairment originates from poor pitch ability in general or whether it is the result of the difficulty in understanding and using prosody for communicative purposes. To investigate whether native Mandarin Chinese-speaking autistic children with intellectual impairment were able to accurately produce native lexical tones, which are pitch patterns that distinguish word meaning lexically and serve little social purpose. Using a picture-naming task, thirteen 8-13-year-old Mandarin Chinese-speaking autistic children with intellectual impairment were tested on their production of Chinese lexical tones. Chronical age-matched typically developing (TD) children were included as the control group. Perceptual assessment and phonetic analyses were conducted with the produced lexical tones. The majority of the lexical tones produced by the autistic children were perceived as accurate by adult judges. Phonetic analysis of the pitch contours found no significant difference between the two groups, and the autistic children and TD children used the phonetic features in comparable ways when differentiating the lexical tones. However, the lexical tone accuracy rate was lower among the autistic children than among the TDs, and the larger individual difference was observed among the autistic children than the TD children. These results indicate that autistic children are able to produce the global contours of the lexical tones, and pitch deficits do not seem to qualify as a core feature of autism. What is already known on the subject Atypical prosody has been considered a maker of the speech of autistic children, and meta-analysis found a significant difference in mean pitch and pitch range between TD children and autistic children. Yet it remains unknown whether the pitch deficits are the result of impaired perceptual-motoric ability or if they reflect failure in learning sentential prosody, which requires an understanding of the interlocutors' mind. In addition, research on pitch ability of autistic children with intellectual disabilities has been scarce, and whether these children are able to produce pitch variation is largely unknown. What this paper adds to existing knowledge We tested native Mandarin Chinese autistic children with intellectual impairment on their production of native lexical tones. The lexical tones in Chinese are pitch variations realized on individual syllables that distinguish lexical meaning, but they do not serve social pragmatic purposes. We found that although these autistic children had only developed limited spoken language, the majority of their lexical tones were perceived as accurate. They were able to use the phonetic features in comparable ways with the TD children when distinguishing the lexical tones. What are the potential or actual clinical implications of this work? It seems unlikely that pitch processing at the lexical level is fundamentally impaired in autistic children, and pitch deficits do not seem to qualify for a core feature of their speech. Practitioners should be cautious when using pitch production as a clinical marker for autistic children.

The disruptive effects of changing-state sound and emotional prosody on verbal short-term memory in blind, visually impaired, and sighted listeners

ABSTRACT Previous findings suggest that blind listeners are less susceptible to auditory distraction in a verbal serial recall task, compared to sighted individuals. However, it is unclear whether this is due to more selective encoding / filtering of auditory information (e.g. efficient perceptual streaming) or to enhanced attentional control. To test these alternatives, the interference induced by changing-state sound was compared with the disruptive effect of emotional speech prosody (happy, angry, fearful intonations) in blind (n = 17), visually impaired (n = 23), and sighted (n = 89) individuals. While a reduced changing-state effect suggests more efficient perceptual filtering, a reduced emotional prosody effect indicates enhanced attentional control. Blind participants were able to recall more items compared to sighted and visually impaired individuals. Moreover, in sighted and visually impaired participants, the changing-state effect was found to be enhanced with angry prosody, whereas blind individuals were less susceptible to changing-state sound regardless of the prosody. The results also suggest group differences in prosody processing, as both visually impaired and blind participants were better able to ignore fearful speech, compared to other prosodies. The findings suggest that profound visual sensory deprivation leads not only to a capacity increment in verbal short-term memory, but enables listeners to shield memory against auditory distracters, suggesting more efficient perceptual streaming.

There might be more to syntactic bootstrapping than being pragmatic: A look at grammatical person and prosody in naturalistic child-directed speech.

In 'Being pragmatic about syntactic bootstrapping', Hacquard (2022) argues that abstract syntax is useful for word learning, but that an additional cue, pragmatics, is both necessary and available for young children during the first steps of language acquisition. She focuses on modals and attitude verbs, where the physical context seems particularly impoverished as the sole basis for deriving meanings, and thus where linguistic cues may be particularly helpful. She convincingly shows how pragmatic and syntactic cues could be combined to help young language learners learn and infer the possible meanings of attitude verbs such as "think", "know" or "want". She also argues that, in some circumstances, syntax and pragmatics would need to be supplemented by semantic information from context - for instance, in the case of modals such as "might", "can", or "must". We agree with Hacquard on the importance of the synergies between these different cues to meaning, and wish to add two other aspects of the input that might also be used by young children in these contexts. The aspects we describe can only be noticed when one analyzes concrete examples of what children hear in their everyday lives, something which Hacquard does very often in her work (e.g., Dieuleveut, van Dooren, Cournane & Hacquard, 2022; Huang, White, Liao, Hacquard & Lidz, 2022; Yang, 2022). Taking into account different cues for meaning would help the field go beyond current models of syntactic bootstrapping, and create an integrated picture of the synergies between different levels of linguistic information.

The influence of musical abilities on the processing of second language prosody: An eye-tracking study

The perception of speech prosody in a second language (L2) remains challenging even for proficient L2 users. Eye-tracking evidence indicates that Dutch listeners show difficulty in the processing of pitch accents that signal a contrast (i.e., contrastive focus) in English, whereas native English listeners use this cue in perception to anticipate upcoming information [Ge et al., Appl. Psycholinguist. 42, 1057–1088 (2021)]. Prosody perception abilities in foreign languages have been associated with individual differences in musical abilities [Jansen et al., Speech Prosody, 713–717 (2022)]. We investigated whether musical abilities influenced the processing of prosodic cues by 45 Dutch adult L2 English users, using a visual-world eye-tracking paradigm. Participants listened to sentences with pitch accent cues to contrastive focus on different words, while viewing pictures showing objects and characters mentioned. We investigated to what extent participants showed anticipatory fixations on the image reflecting the alternative of the contrast, after hearing the accented word. We measured participants’ music perception abilities with a standardised test and analysed its influence on anticipatory fixations. Initial analyses indicate that individuals with stronger musical abilities show more anticipation. This suggests that having stronger perceptual resources underlying both music and speech processing [Patel, Front. Psychol. 2, 142 (2011)] may benefit L2 prosody perception.

Exploring Prosodic Pragmatics at Affective Level

This research work presents a conceptual framework to explore the interplay between intonation and emotional expressions that conveys pragmatic meaning of linguistic utterances. Theory of Affective Pragmatics (TAP), an innovative theory that functions parallel to speech act theory, is applied to introduce a conceptual framework to study prosodic pragmatics at a profound level. The affective component of attitude is introduced in the framework using Ostrom’s ABC model to comprehend affective pragmatics. For further study on this framework, quantitative research design is desirable to analyze data and the data is suggested to be obtained through emotional recordings from emotional prosody speech and transcripts (EPST). Praat as an apt tool recommended for the prosodic analysis of the data. This framework explains how intonation patterns signal unabridged emotions and feelings to understand speaker’s implied meaning. It elaborates the emotional expressions that convey pragmatic meaning to intellectualize affective pragmatics. It highlights some other facets that render affective pragmatics a new-fangled field of research. It has also been envisioned to broaden the scope of prosodic pragmatics as a distinctive research domain in linguistics. The framework is an application of TAP and provides grounds to researchers to understand prosodic pragmatics at a deeper level and to perform future researches in more effective ways. The research is unique in its essence as it offers a conceptual framework to explore affective component of attitude in prosodic pragmatics that has not yet been explored.

Relating the prosody of infant-directed speech to children's vocabulary size.

This study examines correlations between the prosody of infant-directed speech (IDS) and children's vocabulary size. We collected longitudinal speech data and vocabulary information from Dutch mother-child dyads with children aged 18 (N = 49) and 24 (N = 27) months old. We took speech context into consideration and distinguished between prosody when mothers introduce familiar vs. unfamiliar words to their children. The results show that IDS mean pitch predicts children's vocabulary growth between 18 and 24 months. In addition, the degree of prosodic modification when mothers introduce unfamiliar words to their children correlates with children's vocabulary growth during this period. These findings suggest that the prosody of IDS, especially in word-learning contexts, may serve linguistic purposes.

ZSE-VITS: A Zero-Shot Expressive Voice Cloning Method Based on VITS

Voice cloning aims to synthesize the voice with a new speaker’s timbre from a small amount of the new speaker’s speech. Current voice cloning methods, which focus on modeling speaker timbre, can synthesize speech with similar speaker timbres. However, the prosody of these methods is flat, lacking expressiveness and the ability to control the expressiveness of cloned speech. To solve this problem, we propose a novel method ZSE-VITS (zero-shot expressive VITS) based on the end-to-end speech synthesis model VITS. Specifically, we use VITS as the backbone network and add the speaker recognition model TitaNet as the speaker encoder to realize zero-shot voice cloning. We use explicit prosody information to avoid effects from the speaker information and adjust speech prosody using the prosody information prediction and prosody fusion methods directly. We widen the pitch distribution of the train datasets using pitch augmentation to improve the generalization ability of the prosody model, and we fine-tune the prosody predictor alone in the emotion corpus to learn prosody prediction of various styles. The objective and subjective evaluations of the open datasets show that our method can generate more expressive speech and adjust prosody information artificially without affecting the similarity of speaker timbre.

Emotion recognition in autism spectrum disorder across age groups: A cross-sectional investigation of various visual and auditory communicative domains.

Previous research on emotion processing in autism spectrum disorder (ASD) has predominantly focused on human faces and speech prosody, with little attention paid to other domains such as nonhuman faces and music. In addition, emotion processing in different domains was often examined in separate studies, making it challenging to evaluate whether emotion recognition difficulties in ASD generalize across domains and age cohorts. The present study investigated: (i) the recognition of basic emotions (angry, scared, happy, and sad) across four domains (human faces, face-like objects, speech prosody, and song) in 38 autistic and 38 neurotypical (NT) children, adolescents, and adults in a forced-choice labeling task, and (ii) the impact of pitch and visual processing profiles on this ability. Results showed similar recognition accuracy between the ASD and NT groups across age groups for all domains and emotion types, although processing speed was slower in the ASD compared to the NT group. Age-related differences were seen in both groups, which varied by emotion, domain, and performance index. Visual processing style was associated with facial emotion recognition speed and pitch perception ability with auditory emotion recognition in the NT group but not in the ASD group. These findings suggest that autistic individuals may employ different emotion processing strategies compared to NT individuals, and that emotion recognition difficulties as manifested by slower response times may result from a generalized, rather than a domain-specific underlying mechanism that governs emotion recognition processes across domains in ASD.

The dynamic effect of speaking fast on speech prosody

Speaking fast causes several changes in speech prosody. In addition, it can be associated with a decrease in speech intelligibility. In this study, prosodic changes in fast speech were investigated using common prosodic measurements and syllabic prosody index (SPI), a novel prominence measure that combines f0, energy and duration features. Dynamic changes in long-term prosodic prominence were investigated using functional data analysis (FDA), in which the SPI is transformed into a functional form. The possibly decreasing effect of speaking fast on speech intelligibility was evaluated using automatic speech recognition. Phonetic analyses of syllabic units showed that speaking fast decreases duration, f0 and SPI, and increases articulation rate and proportional acoustic energy in the frequency range of 0–1 kHz. FDA supported the aforementioned results by revealing dynamically decreased overall prominence in fast speech. Furthermore, in comparison to regular speech, speech intelligibility was found to be significantly lower in fast speech: word error rate (WER) for regular speech was 0.27, whereas for fast speech it was 0.86.

An evaluation of a multi‐component intervention for loud speech among children with autism spectrum disorder

An evaluation of a multi‐component intervention for loud speech among children with autism spectrum disorder

Speech prosody supports speaker selection and auditory stream segregation in a multi-talker situation

To process speech in a multi-talker environment, listeners need to segregate the mixture of incoming speech streams and focus their attention on one of them. Potentially, speech prosody could aid the segregation of different speakers, the selection of the desired speech stream, and detecting targets within the attended stream. For testing these issues, we recorded behavioral responses and extracted event-related potentials and functional brain networks from electroencephalographic signals recorded while participants listened to two concurrent speech streams, performing a lexical detection and a recognition memory task in parallel. Prosody manipulation was applied to the attended speech stream in one group of participants and to the ignored speech stream in another group. Naturally recorded speech stimuli were either intact, synthetically F0-flattened, or prosodically suppressed by the speaker. Results show that prosody – especially the parsing cues mediated by speech rate – facilitates stream selection, while playing a smaller role in auditory stream segmentation and target detection.

Modelling individual and cross-cultural variation in the mapping of emotions to speech prosody

The existence of a mapping between emotions and speech prosody is commonly assumed. We propose a Bayesian modelling framework to analyse this mapping. Our models are fitted to a large collection of intended emotional prosody, yielding more than 3,000 minutes of recordings. Our descriptive study reveals that the mapping within corpora is relatively constant, whereas the mapping varies across corpora. To account for this heterogeneity, we fit a series of increasingly complex models. Model comparison reveals that models taking into account mapping differences across countries, languages, sexes and individuals outperform models that only assume a global mapping. Further analysis shows that differences across individuals, cultures and sexes contribute more to the model prediction than a shared global mapping. Our models, which can be explored in an online interactive visualization, offer a description of the mapping between acoustic features and emotions in prosody.

Talking Drums – A Study

Membranophones are one of the main divisions of Musical Instruments. They are further classified by many features. One such feature is shape and structure. Hour Glass Drums comes under this one. These are the ancient drums seen from early Times. They are called as Talking Drums in West Africa. In Talking drum pitch can be regulated according to the tone and prosody of human speech. Most of the talking drums sound like a human humming depending on the way they are played As they can change the pitch lower or higher it may be used for Music also. But there are many such Drums seen all over the World with similar shape,structure, mode of construction, playing techniques and so on. So they may also be called as Talking drums

Dimensionality Reduction with Weighted Voting Ensemble Classification Model Using Speech Data Based Parkinson’s Disease Diagnosis

Parkinson’s disease (PD) is a progressive neurodegenerative illness that frequently affects phonation, articulation, fluency, and prosody of speech. Speech impairment is a major sign of PD which can be employed for the earlier identification of the disease and provide proper treatment. Besides, the machine learning (ML) models can be commonly employed for PD detection and classification by the use of speech data. Since the speech data has the features of maximum data redundancy, high aliasing, and small sample sizes, dimensionality reduction (DR) techniques become essential for effective PD diagnosis. Therefore, this paper presents a new DR with weighted voting ensemble classification (DR-WVEC) model for PD diagnosis. The presented DR-WVEC model operates on different stages such as pre-processing, DR, classification, and voting process. Primarily, the speech data undergoes min–max normalization process in order to normalize the speech data. Besides, linear discriminant analysis (LDA) technique is applied for reducing the dimensionality of the features. In addition, an ensemble of two ML models, namely extreme learning machine (ELM) and Adaboost models, is employed for classification. Finally, a weighted voting-based classification process is carried out where the integration of two ML models takes place and the highest outcome is chosen as the final results. In order to assess the effective PR diagnostic outcome, an extensive set of simulations were carried out on Parkinson’s telemonitoring dataset. The obtained experimental results reported the betterment of the DR-VWEC technique over the other compared methods in terms of different aspects.

Adult Cochlear Implant Users Versus Typical Hearing Persons: An Automatic Analysis of Acoustic-Prosodic Parameters.

The aim of this study was to investigate the speech prosody of postlingually deaf cochlear implant (CI) users compared with control speakers without hearing or speech impairment. Speech recordings of 74 CI users (37 males and 37 females) and 72 age-balanced control speakers (36 males and 36 females) are considered. All participants are German native speakers and read Der Nordwind und die Sonne (The North Wind and the Sun), a standard text in pathological speech analysis and phonetic transcriptions. Automatic acoustic analysis is performed considering pitch, loudness, and duration features, including speech rate and rhythm. In general, duration and rhythm features differ between CI users and control speakers. CI users read slower and have a lower voiced segment ratio compared with control speakers. A lower voiced ratio goes along with a prolongation of the voiced segments' duration in male and with a prolongation of pauses in female CI users. Rhythm features in CI users have higher variability in the duration of vowels and consonants than in control speakers. The use of bilateral CIs showed no advantages concerning speech prosody features in comparison to unilateral use of CI. Even after cochlear implantation and rehabilitation, the speech of postlingually deaf adults deviates from the speech of control speakers, which might be due to changed auditory feedback. We suggest considering changes in temporal aspects of speech in future rehabilitation strategies. https://doi.org/10.23641/asha.21579171.

Theta Band (4-8 Hz) Oscillations Reflect Online Processing of Rhythm in Speech Production.

How speech prosody is processed in the brain during language production remains an unsolved issue. The present work used the phrase-recall paradigm to analyze brain oscillation underpinning rhythmic processing in speech production. Participants were told to recall target speeches aloud consisting of verb-noun pairings with a common (e.g., [2+2], the numbers in brackets represent the number of syllables) or uncommon (e.g., [1+3]) rhythmic pattern. Target speeches were preceded by rhythmic musical patterns, either congruent or incongruent, created by using pure tones at various temporal intervals. Electroencephalogram signals were recorded throughout the experiment. Behavioral results in 2+2 target speeches showed a rhythmic priming effect when comparing congruent and incongruent conditions. Cerebral-acoustic coherence analysis showed that neural activities synchronized with the rhythmic patterns of primes. Furthermore, target phrases that had congruent rhythmic patterns with a prime rhythm were associated with increased theta-band (4-8 Hz) activity in the time window of 400-800 ms in both the 2+2 and 1+3 target conditions. These findings suggest that rhythmic patterns can be processed online. Neural activities synchronize with the rhythmic input and speakers create an abstract rhythmic pattern before and during articulation in speech production.

On the representation of hierarchical structure: Revisiting Darwin's musical protolanguage.

In this article, we address the tenability of Darwin's musical protolanguage, arguing that a more compelling evolutionary scenario is one where a prosodic protolanguage is taken to be the preliminary step to represent the hierarchy involved in linguistic structures within a linear auditory signal. We hypothesize that the establishment of a prosodic protolanguage results from an enhancement of a rhythmic system that transformed linear signals into speech prosody, which in turn can mark syntactic hierarchical relations. To develop this claim, we explore the role of prosodic cues on the parsing of syntactic structures, as well as neuroscientific evidence connecting the evolutionary development of music and linguistic capacities. Finally, we entertain the assumption that the capacity to generate hierarchical structure might have developed as part of tool-making in human prehistory, and hence was established prior to the enhancement of a prosodic protolinguistic system.

System for Automatic Assignment of Lexical Stress in Croatian

It is very popular today to integrate voice interfaces into IoT devices. The pronunciation and proper prosody of speech play a major role in the intelligibility and naturalness of synthesized voices. Each language has its own prosodic characteristics. In this paper, we present the results of a study aimed at testing the applicability of methods for modelling and predicting the prosodic features of the Croatian language. The extent to which their performance can be improved by incorporating linguistic features and linguistic peculiarities specific to the Croatian language was investigated. In the model learning process, tree classification was used to predict the lexical stress position and the type of stress in a word, and a lexicon of 1,011,785 word forms was used as the model learning set. Separate models were created for predicting the position and type of lexical stress. The results improved significantly after the rules for atonic words (clitics) were applied. A hybrid approach combining a rule-based approach and a modelling approach was also proposed. The final accuracy of assigning lexical stress using the hybrid approach was 95.3%.

Speech Prosody in Mental Disorders

In response to uncovering brain mechanisms underlying vocal communication and searching for biomarkers for mental illnesses, speech prosody has been increasingly studied in recent years in multiple disciplines, including psycholinguistics. In this article, we provide an up-to-date synthesis of the theoretical foundation and empirical evidence to profile linguistic and emotional prosody in the proper characterization of mental disorders, including schizophrenia, autism, Alzheimer's disease, and depression. Our review reveals a need to develop theoretically motivated and methodologically integrated approaches to the study of context-driven comprehension and expression of pragmatic-affective prosody, which will help elucidate the core features of socio-communicative problems in individuals with mental disorders. We propose that comprehensive models within and across the conventional cognition-emotion-language trichotomy need to be developed to integrate current findings and guide future research. In particular, there needs to be due emphasis on investigating multisensory and cross-modal effects in normal and pathological prosody research. Our review calls for multidisciplinary efforts to address the challenging issues to inform and inspire the advancement of linguistic theories and psychiatric diagnosis and treatment.

Prosodic Development During the Early School-Age Years.

This study used a cross-sequential design to identify developmental changes in narrative speech rhythm and intonation. The aim was to provide a robust, clinically relevant characterization of normative changes in speech prosody across the early school-age years. Structured spontaneous narratives were elicited annually from 60 children over a 3-year period. Children were aged 5-7 years at study outset and then were aged 7-9 years at study offset. Articulation rate, prominence spacing, and intonational phrase length and duration were calculated for each narrative to index speech rhythm; measures of pitch variability and pitch range indexed intonation. Linear mixed-effects (LME) models tested for cohort-based and within-subject longitudinal change on the prosodic measures; linear regression was used to test for the simple effect of age-in-months within year on the measures. The LME analyses indicated systematic longitudinal changes in speech rhythm across all measures except phrase duration; there were no longitudinal changes in pitch variability or pitch range across the school-age years. Linear regression results showed an increase in articulation rate with age; there were no systematic differences between age cohorts across years in the study. The results indicate that speech rhythm continues to develop during the school-age years. The results also underscore the very strong relationship between the rate and rhythm characteristics of speech and so suggest an important influence of speech motor skills on rhythm production. Finally, the results on pitch variability and pitch range are interpreted to suggest that these are inadequate measures of typical intonation development during the school-age years.