Speech Prosody Research Articles

PHONOLOGICAL ANALYSIS OF LANGUAGE PRODUCTION IN PRESCHOOL CHILDREN

The paper presents an analysis of the phonological features of preschool children. The article aims to find patterns of typical and atypical errors. A method was created to conduct the study, which includes three levels of research - at the level of words, sentence repetition, and generating one's speech by picture (spontaneous speech). The results show that children's most common phonological errors are as follows: sound substitution, elision of sound, metathesis of sound, elision of syllable, and deletion of consonants clustered in a syllable. The data obtained from the noun generation task was informative enough about the phonological development of the children being studied. The repeated speech task was accessible for the children and of little value in analyzing the phonological processes. The sentence generation task provides sufficient information about sentence structure, prosody, rate, and force of speech. The test serves a diagnostic and therapeutic purpose. The diagnostic analysis is clear and informative enough to create a therapeutic program based on the examination results. Subsequently, it could be applied for re-testing to measure the progress in the child's output. In addition, the average time children take to administer the whole test is less than 18 minutes, which is an additional positive aspect of the test. The ultimate goal is to standardize this test to assist clinicians in establishing norms by examining Bulgarian-speaking children and assessing their phonological development.

Estimation of Speech Features Using a Wearable Inertial Sensor

Speech features have been investigated as novel digital biomarkers for many psychiatric and neurocognitive diseases. Microphones are the most used devices for speech recording but inevitably suffering from several disadvantages such as privacy leakage and environmental noises, limiting their clinical applications particularly for long-term ambulatory monitoring. The aim of the present study is therefore to explore the feasibility of extracting speech features from the acceleration recorded on the sternum. Ten healthy subjects volunteered in our study. Two speech tasks, i.e., repeating one sentence 20 times and reading 20 different sentences, were performed by each subject, with each task repeated 8 times under different speech rate and loudness. Voice signals and speech-caused chest vibrations were simultaneously recorded by a microphone and an accelerometer placed on the sternum. 42 acoustic features and 6 time-related prosodic features were extracted from both signals using a standard toolbox, and then compared by a linear fit and correlation analysis. Good agreement between the acceleration features and microphone features is observed in all 6 time-related prosodic features for both tasks, but only in 19 and 17 acoustic features for task 1 and 2, respectively, with most of them loudness- or pitch-related. Our results suggest the sternum acceleration to track time-related speech prosody, loudness, and pitch very well, demonstrating the feasibility of deriving digital biomarkers from the acceleration signal for diseases strongly related to time-related prosodic and loudness features.

Decline of Affective Prosody Recognition With a Positivity Bias Among Older Adults: A Systematic Review and Meta-Analysis.

Understanding how older adults perceive and interpret emotional cues in speech prosody contributes to our knowledge of cognitive aging. This study provides a systematic review with meta-analysis to investigate the extent of the decline in affective prosody recognition (APR) among older adults in terms of overall and emotion-specific performance and explore potential moderators that may cause between-studies heterogeneity. The literature search encompassed five electronic databases, with a specific emphasis on studies comparing the APR performance of older adults with that of younger adults. This comparison was focused on basic emotions. Meta-regression analyses were executed to pinpoint potential moderators related to demographic and methodological characteristics. A total of 19 studies were included in the meta-analysis, involving 560 older adults with a mean age of 69.15 years and 751 younger adults with a mean age of 23.02 years. The findings indicated a substantial negative effect size (g = -1.21). Furthermore, the magnitude of aggregated effect sizes showed a distinct valence-related recognition pattern with positive prosody exhibiting smaller effect sizes. Language background and years of education were found to moderate the overall and emotion-specific (i.e., disgust and surprise) performance effect estimate, and age and gender significantly influenced the effect estimate of happiness. The results confirmed a significant decline in APR ability among older adults compared to younger adults, but this decline was unbalanced across basic emotions. Language background and educational level emerged as significant factors influencing older adults' APR ability. Moreover, participants with a higher mean age exhibited notably poorer performance in recognizing happy prosody. These findings underscore the need to further investigate the neurobiological mechanisms for APR decline associated with aging. https://doi.org/10.23641/asha.26407888.

A cross-linguistic review of citation tone production studies: Methodology and recommendations.

The study of citation tones, lexical tones produced in isolation, is one of the first steps towards understanding speech prosody in tone languages. However, methodologies for investigating citation tones vary significantly, often leading to limited comparability of tone inventories, both within and across languages. This paper presents a systematic review of research methods and practices in 136 citation tone studies on 129 tonal language varieties in China, including 99 studies published in Chinese, which are therefore not easily available to an international scientific readership. The review provides an overview of possible analytical decisions along the research pipeline, and unveils considerable variation in data collection, analysis, and reporting conventions, particularly in how f0, the primary acoustic correlate for tone, is operationalised and reported across studies. Key methodological issues are identified, including small sample sizes and inadequate transparency in communicating methodological decisions and procedure. This paper offers a clear road map for citation tone production research and proposes a range of recommendations on speaker sampling, experimental design, acoustic processing techniques, f0 analysis, and result reporting, with the goal of facilitating future tonal research and enhancing resources for underrepresented tonal varieties.

Memorization-Based Training and Testing Paradigm for Robust Vocal Identity Recognition in Expressive Speech Using Event-Related Potentials Analysis.

Recognizing familiar speakers from vocal streams is a fundamental aspect of human verbal communication. However, it remains unclear how listeners can still discern the speaker's identity in expressive speech. This study develops a memorization-based individual speaker identity recognition approach and an accompanying electroencephalogram (EEG) data analysis pipeline, which monitors how listeners recognize familiar speakers and tell unfamiliar ones apart. EEG data captures online cognitive processes during new versus old speaker distinction based on voice, offering a real-time measure of brain activity, overcoming limits of reaction times and accuracy measurements. The paradigm comprises three steps: listeners establish associations between three voices and their names (training); listeners indicate the name corresponding to a voice from three candidates (checking); listeners distinguish between three old and three new speaker voices in a two-alternative forced-choice task (testing). The speech prosody in testing was either confident or doubtful. EEG data were collected using a 64-channel EEG system, followed by preprocessing and imported into RStudio for ERP and statistical analysis and MATLAB for brain topography. Results showed an enlarged late positive component (LPC) was elicited in the old-talker compared to the new-talker condition in the 400-850 ms window in the Pz and other wider range of electrodes in both prosodies. Yet, the old/new effect was robust in central and posterior electrodes for doubtful prosody perception, whereas the anterior, central, and posterior electrodes are for confident prosody condition. This study proposes that this experiment design can serve as a reference for investigating speaker-specific cue-binding effects in various scenarios (e.g., anaphoric expression) and pathologies in patients like phonagnosia.

Association between neural prosody discrimination and language abilities in toddlers: a functional near-infrared spectroscopy study

BackgroundLanguage delay affects near- and long-term social communication and learning in toddlers, and, an increasing number of experts pay attention to it. The development of prosody discrimination is one of the earliest stages of language development in which key skills for later stages are mastered. Therefore, analyzing the relationship between brain discrimination of speech prosody and language abilities may provide an objective basis for the diagnosis and intervention of language delay.MethodsIn this study, all cases(n = 241) were enrolled from a tertiary women’s hospital, from 2021 to 2022. We used functional near-infrared spectroscopy (fNIRS) to assess children’s neural prosody discrimination abilities, and a Chinese communicative development inventory (CCDI) were used to evaluate their language abilities.ResultsNinety-eight full-term and 108 preterm toddlers were included in the final analysis in phase I and II studies, respectively. The total CCDI screening abnormality rate was 9.2% for full-term and 34.3% for preterm toddlers. Full-term toddlers showed prosody discrimination ability in all channels except channel 5, while preterm toddlers showed prosody discrimination ability in channel 6 only. Multifactorial logistic regression analyses showed that prosody discrimination of the right angular gyrus (channel 3) had a statistically significant effect on language delay (odd ratio = 0.301, P < 0.05) in full-term toddlers. Random forest (RF) regression model presented that prosody discrimination reflected by channels and brain regions based on fNIRS data was an important parameter for predicting language delay in preterm toddlers, among which the prosody discrimination reflected by the right angular gyrus (channel 4) was the most important parameter. The area under the model Receiver operating characteristic (ROC) curve was 0.687.ConclusionsNeural prosody discrimination ability is positively associated with language development, assessment of brain prosody discrimination abilities through fNIRS could be used as an objective indicator for early identification of children with language delay in the future clinical application.

A simple psychophysical procedure separates representational and noise components in impairments of speech prosody perception after right-hemisphere stroke

After a right hemisphere stroke, more than half of the patients are impaired in their capacity to produce or comprehend speech prosody. Yet, and despite its social-cognitive consequences for patients, aprosodia following stroke has received scant attention. In this report, we introduce a novel, simple psychophysical procedure which, by combining systematic digital manipulations of speech stimuli and reverse-correlation analysis, allows estimating the internal sensory representations that subtend how individual patients perceive speech prosody, and the level of internal noise that govern behavioral variability in how patients apply these representations. Tested on a sample of N = 22 right-hemisphere stroke survivors and N = 21 age-matched controls, the representation + noise model provides a promising alternative to the clinical gold standard for evaluating aprosodia (MEC): both parameters strongly associate with receptive, and not expressive, aprosodia measured by MEC within the patient group; they have better sensitivity than MEC for separating high-functioning patients from controls; and have good specificity with respect to non-prosody-related impairments of auditory attention and processing. Taken together, individual differences in either internal representation, internal noise, or both, paint a potent portrait of the variety of sensory/cognitive mechanisms that can explain impairments of prosody processing after stroke.

Beyond musical training: Individual influences on the perception of the speech-to-song illusion

When a spoken phrase is repeated several times, listeners often report a perceptual illusion during which speech is transformed into song. The speech-to-song (STS) illusion is often attributed to prosodic elements of speech, though listeners can vary greatly in their STS experience. While previous research established robust links between music aptitude and STS, the present study asks whether other cognitive traits may also influence STS. Individual (in)sensitivity to nonverbal aspects of speech, specifically speech prosody, has been previously linked to autistic traits and emotional intelligence. We test whether the presence of autistic traits, the level of emotional intelligence and musical training, as well as syntactic complexity influence the likelihood, speed, and strength of STS among native British English listeners. The results provide evidence for the involvement of some but not all studied traits. We found sentence complexity to be interacting with a composite score of musical training, and emotional intelligence for the likelihood of STS, whereas sentence complexity influenced the strength of the transformation. These results suggest that individual listener variability may interact with the linguistic parameters of sentences in STS. Crucially, sensitivity to prosody through emotional intelligence or by the presence of autistic traits does not mediate the transformation.

How Speech Prosody Affects Leadership Perceptions: A Machine Learning Approach

How Speech Prosody Affects Leadership Perceptions: A Machine Learning Approach

Emotional Intelligence in Text-To-Speech Synthesis in Pali Language Using Fuzzy Logic

The field of emotional text-to-speech (TTS) synthesis is making swift progress within the realm of artificial intelligence, holding immense promise to transform our interaction with technology. By using advanced algorithms to analyze and understand the emotional content of text, these systems are able to produce spoken language that accurately conveys the intended emotional tone of the message. Despite the existence of several Text-To-Speech systems across various languages, Pali language is yet to have its own. As a result, we have taken the initiative to create a Text-To-Speech synthesizer exclusively for Pali. Our system offers an end-to-end solution for emotional speech synthesis via Text-To-Speech. We address the problem by incorporating disentangled, well-grained prosody features with global, sentence-level emotion implanting. These well-grained features learn to denote local prosodic differences disentangled from the speaker, tone, and worldwide emotion label. Prosody is usually modeled by rules, so we have implemented the fuzzy logic system to develop a controller for the prosody of Pali speech. The fuzzy controller handles different linguistic parameters in three types of sentences: interrogative, exclamatory, and declarative. The final system produces comprehensible speech that mimics the appropriate intonation for every type of sentence. In this paper, we introduce and outline the application of a fuzzy paradigm to incorporate a Text-To-Speech system for the Pali language while preserving a rule-based Concatenative synthesizer. In the outline of classic Concatenative TTS systems, we recommend a new method in order to increase Concatenative unit selection computation, directed at increasing synthetic speech perceptual superiority. In order to tackle the problem of phonemes that are prone to multiple descriptions in rule-based speech synthesis, the proposed solution involves a fuzzy system. In the introductory section, we offer a concise description of the current context surrounding the challenge of emotional speech synthesis. The second section of this paper outlines the notable advancements made in emotional speech synthesis, acknowledging the contributions of various researchers in this field. The third section delves into the technical details of implementing a fuzzy system The last section of the paper presents the main conclusions and future research scope.

L2 prosody effects on pronunciation teaching and oral communication: updated

Pronunciation accuracy in a foreign language requires mastering production of both segmental (consonants and vowels) and prosody (features that operate from the syllable domain, such as lexical/phrasal stress, pitch accent, rhythm, intonation, and voice quality) of speech. This study aims to elucidate the significance of instruction in pronunciation, with a focus on L2 prosody as a pivotal component in the enhancement of oral communication skills. The objective is to reduce pronunciation discrepancies and augment conversational interactions in spoken communication. The theoretical review of our research is based on studies such as, Munro and Derwing (2020 [1999]) Derwing and Munro (2015), Levis (2018), which underscore the role of prosody in the context of pronunciation instruction and in L2 oral communication. We also have supported in studies such as, Ortega-Lebaria et al. (2023), Silva Jr. (2023), for the selection of techniques and guidelines for extracting both durational and melodic parameters from L2 prosody-focused research. Regarding the methodology, we gathered data from Brazilian English speakers and native speakers of English from the USA. Each participant read a passage, from which we selected ten utterances for subsequent analysis. The acoustic analysis was conducted based on the duration of phonetic syllables (V-V units) and the fundamental frequency (F0), both normalized using Lobanov’s z-score. We also employed ANOVA statistical tests to examine the group effect for both duration and F0 parameters. Our findings indicate notable differences between native and Brazilian English speakers’ productions for both duration and F0. We conclude, even on a preliminary basis, that the application of L2 prosody is relevant in improving pronunciation instruction and enhancing oral communication.

A meta-analysis of expressive prosody in cochlear implant users

IntroductionProsody is used to express indexical (identifying the talker), linguistic (e.g., question intonation, lexical stress), pragmatic (e.g., contrastive stress, sarcasm), and emotional/affective functions. It is manifested through changes in fundamental frequency (f0), intensity, and duration. F0 and intensity are degraded when perceived through a cochlear implant (CI). The purpose of this meta-analysis is to compare expressive prosody in speech produced by CI users versus normal hearing peers. MethodsA systematic search of the literature found 25 articles that met all inclusion criteria. These articles were assessed for quality, and data pertaining to the expression of f0, intensity, and duration, as well as classification accuracy and appropriateness ratings from normal hearing listeners, were extracted and meta-analyzed using random effects models. ResultsThe articles included in the meta-analysis were generally of acceptable or high quality. Meta-analyses revealed significant differences between individuals with CIs vs. normal hearing on all measures except mean f0, mean intensity, and rhythm. Effect sizes were generally medium to large. There was significant heterogeneity across studies, but little evidence of publication bias. ConclusionsCI users speak with less variable f0, smaller f0 contours, more variable intensity, a slower speech rate, and reduced final lengthening at syntactic boundaries. These acoustic differences are reflected in significantly poorer ratings of speech produced by CI users compared to their normal hearing peers, as assessed by groups of normal hearing listeners. Because atypical expressive prosody is associated with negative outcomes, clinicians should consider targeting prosody when working with individuals who use CIs.

Predicting Social Competence in Autistic and Non-Autistic Children: Effects of Prosody and the Amount of Speech Input.

Autistic individuals often face challenges perceiving and expressing emotions, potentially stemming from differences in speech prosody. Here we explore how autism diagnoses between groups, and measures of social competence within groups may be related to, first, children's speech characteristics (both prosodic features and amount of spontaneous speech), and second, to these two factors in mothers' speech to their children. Autistic (n = 21) and non-autistic (n = 18) children, aged 7-12 years, participated in a Lego-building task with their mothers, while conversational speech was recorded. Mean F0, pitch range, pitch variability, and amount of spontaneous speech were calculated for each child and their mother. The results indicated no differences in speech characteristics across autistic and non-autistic children, or across their mothers, suggesting that conversational context may have large effects on whether differences between autistic and non-autistic populations are found. However, variability in social competence within the group of non-autistic children (but not within autistic children) was predictive of children's mean F0, pitch range and pitch variability. The amount of spontaneous speech produced by mothers (but not their prosody) predicted their autistic children's social competence, which may suggest a heightened impact of scaffolding for mothers of autistic children. Together, results suggest complex interactions between context, social competence, and adaptive parenting strategies in driving prosodic differences in children's speech.

Changes in speech prosody perception during Parkinson's disease: A comprehensive analysis

IntroductionParkinson's disease (PD) is a progressive neurodegenerative disorder that affects approximately 1%–2% of individuals aged 60 and above. Communication disorders in PD can significantly impact the overall quality of life. As prosody plays a vital role in verbal communication, the present study examines Persian prosody perception in PD, focusing on linguistic and emotional aspects of prosody. MethodsThis cross-sectional study aimed to compare the perception of linguistic and emotional prosody in three groups: middle-aged adults (n = 22; mean age = 50.40 years), healthy older adults (n = 22; mean age = 68.31 years), and individuals with Parkinson's disease (n = 22; mean age = 65years). All individuals with PD were classified in stages 1; 1.5; 2; 2.5, and 3 of the disease using the Hoehn and Yahr scale. All participants had an MMSE score of 24 or above. The Florida Affect Battery (FAB) was used to evaluate prosody perception. This Battery was validated in the Persian language and its reliability and validity were reported as 94 % and 100 % respectively. ResultsParticipants with PD presented significantly lower scores than the older adults in all subtests of the FAB (p < 0.05), while healthy older adults were significantly different only in linguistic discrimination (β = -2.14; -3.68 to -0.61), and linguistic naming of prosody (β = 1.25; 0.17 to 2.33) compared to middle-aged adults. ConclusionsThe present study sheds light on the influence of PD on Persian prosody perception. Given the crucial role of prosody in verbal communication, these findings enhance our understanding of communication disorders in PD and could bring attention to consider prosody perception, among other aspects, when assessing individuals affected by PD.

Voice Synthesis Improvement by Machine Learning of Natural Prosody.

Since the advent of modern computing, researchers have striven to make the human-computer interface (HCI) as seamless as possible. Progress has been made on various fronts, e.g., the desktop metaphor (interface design) and natural language processing (input). One area receiving attention recently is voice activation and its corollary, computer-generated speech. Despite decades of research and development, most computer-generated voices remain easily identifiable as non-human. Prosody in speech has two primary components-intonation and rhythm-both often lacking in computer-generated voices. This research aims to enhance computer-generated text-to-speech algorithms by incorporating melodic and prosodic elements of human speech. This study explores a novel approach to add prosody by using machine learning, specifically an LSTM neural network, to add paralinguistic elements to a recorded or generated voice. The aim is to increase the realism of computer-generated text-to-speech algorithms, to enhance electronic reading applications, and improved artificial voices for those in need of artificial assistance to speak. A computer that is able to also convey meaning with a spoken audible announcement will also improve human-to-computer interactions. Applications for the use of such an algorithm may include improving high-definition audio codecs for telephony, renewing old recordings, and lowering barriers to the utilization of computing. This research deployed a prototype modular platform for digital speech improvement by analyzing and generalizing algorithms into a modular system through laboratory experiments to optimize combinations and performance in edge cases. The results were encouraging, with the LSTM-based encoder able to produce realistic speech. Further work will involve optimizing the algorithm and comparing its performance against other approaches.

Exploring auditory emotion perception in cochlear implant users: Investigating the interplay of speech processing and affective signals

Human emotions are intricately expressed through vocal sounds, encompassing affective prosody in speech and non-verbal cues such as screams and laughter. Recent evidence indicates that vocalizations take neurophysiological precedence over speech-embedded emotions and are generally easier to identify. However, Cochlear implant (CI) users still face challenges deciphering the subtle nuances in these primal signals. The implant's limited fidelity in transmitting acoustic information results in highly variable levels of emotion perception abilities among its users. Identifying the factors explaining this significant variability in abilities among CI users remains of great interest. Our recent investigations into CI users' abilities to perceive emotions and speaker sincerity have often incorporated diverse aspects of auditory proficiency, including pitch discrimination, music processing, and speech intelligibility. The combination of results from these different projects can help shed light on the intricate interplay between speech processing and emotional recognition in CI users. Surprisingly, even when presented with emotional musical stimuli, CI users' proficiency often leaned toward processes related to speech intelligibility, proposing common mechanisms underlying linguistic and affective processes in CI users that do not readily relate to musical skills or pitch sensitivity. Hence, maintaining a clinical focus on speech processing remains crucial, even when exploring affective skills in CI users.

Age-related decline in hearing and emotional prosody processing: A multi-feature oddball study

This study investigated how age and hearing loss contributed to the challenges in perceiving emotional prosody in speech. A multi-feature oddball paradigm was employed to assess mismatch negativity (MMN) and P3a responses to three deviant prosodies (happy, angry and sad) against the standard neutral prosody in spoken words. The participants were 22 adults in the age range of 18 to 70 with various degrees of hearing loss as assessed with pure tone audiometry. Linear mixed effects models revealed heightened MMN response to angry voice and strongest P3a response to happy voice among the three deviants. There was a significant positive correlation between age and MMN amplitude for the angry prosody. Significant correlations were also observed between hearing loss and MMN amplitude in response to both angry and happy stimuli. In addition, negative correlations were found between P3a amplitude and hearing loss as well as aging for happy stimuli. However, aging and hearing loss did not significantly affect the processing of sadness in this paradigm. These findings showed category-sensitive age-related decline in emotional prosody processing. The intricate correlations patterns among age, hearing loss, MMN, and P3a responses called for further exploration of broader implications in cognitive aging.

Prosody expression in prelingually- and postlingually- implanted deaf adults

Speech prosody - its melody, accent, and quantity - plays a significant role in language communication, including the expression of emotions. Limited prosodic skills in individuals with profound hearing disabilities affect their communicative functioning. Modern hearing prostheses, such as cochlear implants, enable deaf people to develop and utilize auditory and oral language abilities, although not always prosodic ones. The presented results of our own research indicate the importance of the period of hearing loss and the duration of implant use for prosodic effectiveness in cochlear implant users. They also emphasize the need for rehabilitation activities that focus on early prosodic stimulation in prelingually deaf children and prosodic training in prelingually deaf adults. / Keywords: speech prosody, prelingual and postlingual deafness, cochlear implant

Indian Classical Music Recognition using Deep Convolution Neural Network

A divine approach to communicate feelings about the world occurs through music. There is a huge variety in the language of music. One of the principal variables of Indian social legacy is classical music. Hindustani and Carnatic are the two primary subgenres of Indian classical music. Models have been trained and taught to distinguish between Carnatic and Hindustani songs. This paper presents Indian classical music recognition based on multiple acoustic features (MAF) consisting of various statistical, spectral, and time domain features. The MAF provides the changes in intonation, timbre, prosody and pitch of the musical speech due to different ragas. The lightweight DCNN is used to improve the representation of the raga sound and to provide higher order abstract level features. The overall performance of the raga type is estimated using various performance metrics, including accuracy, precision, recall and F1-score. The proposed DCNN achieves an accuracy, precision, recall, and F1-score of 89.38%, 0.89, 0.89, and 0.89, respectively, for eight raga classifications. The extensive experimentation on eight classical ragas has shown a noteworthy improvement over the traditional state of art.

Indian Classical Music Recognition using Deep Convolution Neural Network

A divine approach to communicate feelings about the world occurs through music. There is a huge variety in the language of music. One of the principal variables of Indian social legacy is classical music. Hindustani and Carnatic are the two primary subgenres of Indian classical music. Models have been trained and taught to distinguish between Carnatic and Hindustani songs. This paper presents Indian classical music recognition based on multiple acoustic features (MAF) consisting of various statistical, spectral, and time domain features. The MAF provides the changes in intonation, timbre, prosody and pitch of the musical speech due to different ragas. The lightweight DCNN is used to improve the representation of the raga sound and to provide higher order abstract level features. The overall performance of the raga type is estimated using various performance metrics, including accuracy, precision, recall and F1-score. The proposed DCNN achieves an accuracy, precision, recall, and F1-score of 89.38%, 0.89, 0.89, and 0.89, respectively, for eight raga classifications. The extensive experimentation on eight classical ragas has shown a noteworthy improvement over the traditional state of art.