Prosodic Features Research Articles

The Prosodic Role in English Production of Native Mandarin Children: A Systematic Review Proposal

English, as the most widely used language globally, is increasingly valued by parents and educators. In China, a growing number of children choose English as their second language for learning after being able to use Mandarin relatively proficiently. Mandarin speakers, learning English may face many challenges, one of which is that native Mandarin speakers find it difficult to speak English naturally like native English speakers. Some scholars believe that the disparity in prosodic features between Mandarin and English is one of the key factors causing this difficulty. Compared to adult learners, children have an unstable prosodic development system, resulting in significant differences in language acquisition. Therefore, understanding the relationship between the prosodic features of Mandarin-speaking children and their English production is crucial. From the perspective of structure and purpose, this paper Outlines a systematic literature review framework, hoping to summarize the previous articles on the role of prosody in the English production of Mandarin speaking children, and provide some useful results that can help people in this field to improve their work. From the perspective of content, in this systematic literature review, we are going to propose and solve two questions: (1) Does the prosody of Mandarin influence Mandarin-speaking children’s English production? (2) If the influences do exist, what are the specific influences? To solve these two questions, we first search some articles with the searching strategies that we design. Then we plan to screen the articles that we get from searching with the inclusion criteria and the exclusion criteria that we design. Finally, we plan to use quantitative analysis to analyze these samples and give the eventual result.

Emotion Classification from Speech Waveform Using Machine Learning and Deep Learning Techniques

Emotions play a key role in determining the human mental state and indirectly express an individual’s well- being. A speech emotion recognition system can extract a person’s emotions from his/her speech inputs. There are some universal emotions such as anger, disgust, fear, happiness, pleasantness, sadness and neutral. These emotions are of significance especially in a situation like the Covid pandemic, when the aged or sick are vulnerable to depression. In the current paper, we examined various classification models with finite computational strength and resources in order to determine the emotion of a person from his/her speech. Speech prosodic features like pitch, loudness, and tone of speech, and work spectral features such as Mel Frequency Capstral Coefficients (MFCCs) of the voice were used to analyze the emotions of a person. Although sequence to sequence state of the art models for speech detection that offer high levels of accuracy and precision are currently in use, the computational needs of such approaches are high and inefficient. Therefore, in this work, we emphasised analysis and comparison of different classification algorithms such as multi layer perceptron, decision tree, support vector machine, and deep neural networks such as convolutional neural network and long short term memory. Given an audio file, the emotions that were exhibited by the speaker were recognized using machine learning and deep learning techniques. A comparative study was performed to identify the most appropriate algorithms that could be used to recognize emotions. Based on the experiment results, the MLP classifier and convolutional neural network model offered better accuracy with smaller variations when compared with other models used for the study.

Constructional variation and stabilization in [sei] and [aham sei] as disbelief answers in Brazilian Portuguese

This paper aims to analyze how constructional variation and constructional stabilization occurwith [sei] and [aham sei] as disbelief answers in Brazilian Portuguese (BP) instead of theircommon use linked to cognitive sense (the act of knowing something/knowing about somethingor knowing how to do something), perceived primarily by differences in intonation andconversational context. For this, we’ll see the results found by Souza (2024) who, based oncorpora data and tests made with native speakers, seek to understand a little further howconstructional variation happens in this case and what are the preponderant factors for thesespeakers to choose those constructions instead of others or vice-versa in some specificdialogical scenarios. His work, elaborated under a socioconstructionist profile (MachadoVieira, 2016; Machado Vieira and Wiedemer, 2018; 2019a; 2019b) studied how ConstructionGrammar (Goldberg 1995, 2006; Traugott and Trousdale, 2013) and VariationistSociolinguistic (Weinreich, Labov and Herzog 1968; Eckert, 2012) can contribute to explainempirically the results found, which led us to follow the same theoretical path in order to expandthe examples and expose interesting data shown at his work, specially how schematicity andproductivity (Traugott and Trousdale, 2013) happen within these constructions. In sum, wepresent the most prominent results of a work that studies how prosodic features interfere in theform-meaning paring in the analyzed data, an interface still underexplored at the constructionalstudies field.

Estimation of Speech Features Using a Wearable Inertial Sensor

Speech features have been investigated as novel digital biomarkers for many psychiatric and neurocognitive diseases. Microphones are the most used devices for speech recording but inevitably suffering from several disadvantages such as privacy leakage and environmental noises, limiting their clinical applications particularly for long-term ambulatory monitoring. The aim of the present study is therefore to explore the feasibility of extracting speech features from the acceleration recorded on the sternum. Ten healthy subjects volunteered in our study. Two speech tasks, i.e., repeating one sentence 20 times and reading 20 different sentences, were performed by each subject, with each task repeated 8 times under different speech rate and loudness. Voice signals and speech-caused chest vibrations were simultaneously recorded by a microphone and an accelerometer placed on the sternum. 42 acoustic features and 6 time-related prosodic features were extracted from both signals using a standard toolbox, and then compared by a linear fit and correlation analysis. Good agreement between the acceleration features and microphone features is observed in all 6 time-related prosodic features for both tasks, but only in 19 and 17 acoustic features for task 1 and 2, respectively, with most of them loudness- or pitch-related. Our results suggest the sternum acceleration to track time-related speech prosody, loudness, and pitch very well, demonstrating the feasibility of deriving digital biomarkers from the acceleration signal for diseases strongly related to time-related prosodic and loudness features.

Multimodal Spatio-Temporal Framework for Real-World Affect Recognition

Deep learning models show great potential in applications involving video-based affect recognition, including human-computer interaction, robotic interfaces, stress and depression assessment, and Alzheimer's disease detection. The low complex Multimodal Diverse Spatio-Temporal Network (MDSTN) has been analysed to effectively capture spatio-temporal information from audio-visual modalities for affect recognition using the Acted Facial Expressions in the Wild (AFEW) dataset. The scarcity of data is handled by data augmented parallel feature extraction for visual network. Visual features extracted by carefully reviewing and customizing Convolutional 3D architecture over different ranges are combined to train a neural network for classification. Multi-resolution Cochleagram (MRCG) features from speech, along with spectral and prosodic audio features, are processed by a supervised classifier. The late fusion technique is explored to integrate audio and video modalities, considering their processing over different temporal spans. The MDSTN approach significantly boosts the accuracy of basic emotion recognition to 71.54% on the AFEW dataset. It demonstrates exceptional proficiency in identifying emotions such as disgust and surprise, thus exceeding current benchmarks in real-world affect recognition.

Cortical Responses to Mother's Voice in Comparison with Unfamiliar Voice in the First Trimester of Life: A fNIRS Study.

Introduction The use of functional near-infrared light spectroscopy (fNIRS) may be applied to study cortical responses in children and could offer insight into auditory and speech perception during the early stages of life. Some literature suggests that babies are already able to identify familiar voices at birth, and fNIRS is a non-invasive technique that can be used to study this population. Objective To characterize the cortical responses of infants during their first trimester of life to infant-directed speech using near-infrared light spectroscopy and to verify whether there is a difference in responses when infant-directed speech is performed by their mother compared with an unknown person. Methods Twenty-three children between 0 and 3 months, healthy, without risk indicators for hearing loss, and with results considered normal in the audiological evaluation were tested with near-infrared spectroscopy using infant-directed speech as an auditory stimulus produced by their own mother and by an unknown source. Results Bilateral cortical activation was observed. The responses were present in the temporal, frontal, and parietal regions. Regarding the familiarity aspect, no significant difference was observed for the mother's voice compared with an unknown voice. Conclusion Infant-directed speech has prosodic characteristics capable of activating several cortical regions in the infant's first trimester of life, especially the temporal region. The familiarity effect needs to be better defined for this type of stimulus during this period.

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody.

This research aims to examine both the prosodic-acoustic features and the perceptual correlates of foreign-accented English and foreign-accented Brazilian Portuguese and check how the speakers' productions of foreign and native accents are correlated to the listeners' perception. In the Methodology, we conducted a speech production procedure with a group of American speakers of L2 Brazilian Portuguese and a group of Brazilian speakers of L2 English, and a speech perception procedure in which we performed voice lineups for both languages.For the speech production statistical analysis, we ran Generalized Additive Models to evaluate the effect of the language groups on each class (metric or prosodic-acoustic) of features controlled for the smoothing effect of the covariate(s) of the opposite class. For the speech perception statistical analysis, we ran a Kruskal-Wallis test and a post-hoc Dunn's test to evaluate the effect of the voices of the lineups on the scores judged by the listeners. We nevertheless conducted acoustic (voice) similarity tests based on Cosine and Euclidean distances. Results showed significant acoustic differences between the language groups in terms of variability of the f0, duration, and voice quality. For the lineups, the results indicated that prosodic features of f0, intensity, and voice quality correlated to the listeners' perceived judgments.

Strong Finals: A prosodic feature projecting ‘more to come’ in a Danish urban dialect

Abstract This article presents structural and interactional aspects of Strong Finals, a prosodic feature characterised by lengthening, increased volume, and non-falling intonation on word-final syllables. Interactionally, Strong Finals support five types of action: listing, projecting a description, stating conditions, asking questions, and announcing reported speech. In general, Strong Finals project that there is more to come, and this ‘more’ may in some cases be provided by either participant. Strong Finals are often found in multi-speaker settings, where they assist speakers in taking the floor or changing the topic. The article’s descriptions are based on recordings of natural spoken interaction in linguistically diverse areas in Aarhus, Denmark. Here, a new urban dialect has developed like other urban dialects that have been described in Copenhagen and other North Germanic cities. Strong Finals are a local phenomenon, however, and are not found in the Copenhagen studies.

HRT in DMC? the orthographic representation of high rising terminals in WhatsApp

Contemporary research has shown that a combination of qualitative and quantitative methods is productive in exploring patterns of Digitally Mediated Communication (DMC). In this paper, I demonstrate the analytical potential of this approach by studying the typographic representation of a prosodic feature of spoken language – High Rising Terminals (HRTs, e.g., that beer pong place I went for my birthday?) – in a large corpus of WhatsApp messages (96,471 messages; 594,183 words) sent by 15 young British adults. Combining methods and approaches from variationist and interactional sociolinguistics, I show that the orthographic representation of HRTs patterns in pragmatically similar ways to the feature in speech in that it most frequently functions as a way of verifying the interlocutors’ comprehension of discourse-new information. The precise rate and pragmatic function of this feature, however, appears to be constrained by the textual modality of the platform. Concluding, I join others in arguing for the analytical potential of employing a multidimensional approach to studying variable patterns of DMC.

An Efficient Voice-Based Emotion Recognition Using Hybrid Capsule Slime Mould Dense Deep Learning Framework

Emotion recognition is an acceptable task of understanding the other’s emotions and thoughts. Modern technology allows machines to recognize objects without the need for human intervention. The existing emotion recognition system faces more difficulties in making an accurate result with limited audio files. To address this problem, a Bag of audio terms-based hybrid deep learning models will be introduced it is known as the pioneering deep learning model. Input voice data is considered from a large dataset and pre-processed using a Data normalization and adaptive bilinear filtering approach. Afterward, acoustic features are taken out from the voice signals to capture related information for emotion recognition. These features can include linear prediction coefficients (LPC), three-dimensional (3D) log-mel spectrum, mel-frequency cepstral coefficients (MFCCs), and Prosodic features. Subsequently, feature selection is performed using an improved wild horse optimization (WHO) approach. Finally, a hybrid capsule slime mould dense deep learning framework (HCSDN) is used for voice-based emotion recognition. IEMOCAP and EMODB datasets are used to calculate system performance. The performance metrics denote the proposed system achieves 96.78% accuracy, 96.45% specificity, 95.81% precision, 4.256% error rate, and 94.256% sensitivity, 0.75% false positive rate in terms of the IEMOCAP dataset. Similarly, the proposed system achieves 96.85% accuracy, 95.74% specificity, 96.12% precision, 3.432% error rate, 95.25% sensitivity, and 0.62% false positive rate in terms of the EMODB dataset.

Rhythmic and Poetic Qualities in Sinhala Speech. By Mahagama Sekera. Translated by Garrett Field and Ravinda Mahagamasekera.

Mahagama Sekera (1929–1976) investigates how rhythmic qualities and poetics inhere in figurative language found in everyday Sinhala speech; the formation of original constructions in poetry and song lyrics; prosodic features in child storytelling; colloquial expressions generated from one light verb; aperiodic rhythms in context-specific utterances and proverbs; and the moraic structures of doublets (yugala pada). Citation: Sekera, Mahagama. “Rhythmic and Poetic Qualities in Sinhala Speech.” Translated by Garrett Field and Ravinda Mahagamasekera; edited by Richard K. Wolf. Ethnomusicology Translations, no. 16. Bloomington, IN: Society for Ethnomusicology, 2024. DOI: Originally published in Sinhala as pages 39–51 in chapter 2, “Rhythm and [Sinhala] Speech” in Sinhala Gadya Padya Nirmānayanhi Ridma Lakṣana (Rhythmic Qualities of Sinhala Prose and Verse), S. Godage & Brothers, 2007 [2001].

Pitfalls in using ML to predict cognitive function performance.

Machine learning analyses are widely used for predicting cognitive abilities, yet there are pitfalls that need to be considered during their implementation and interpretation of the results. Hence, the present study aimed at drawing attention to the risks of erroneous conclusions incurred by confounding variables illustrated by a case example predicting executive function performance by prosodic features. Healthy participants (n = 231) performed speech tasks and EF tests. From 264 prosodic features, we predicted EF performance using 66 variables, controlling for confounding effects of age, sex, and education. A reasonable model fit was apparently achieved for EF variables of the Trail Making Test. However, in-depth analyses revealed indications of confound leakage, leading to inflated prediction accuracies, due to a strong relationship between confounds and targets. These findings highlight the need to control confounding variables in ML pipelines and caution against potential pitfalls in ML predictions.

Voice of leadership: Prosodic features as predictors of leadership perception.

Voice of leadership: Prosodic features as predictors of leadership perception.

Functional near-infrared spectroscopy and language development: An integrative review.

Functional near-infrared spectroscopy (fNIRS) stands poised to revolutionize our understanding of auditory detection, speech perception, and language development in infants. In this study, we conducted a meticulous integrative review across Medline, Scopus, and LILACS databases, targeting investigations utilizing fNIRS to explore language-related features and cortical activation during auditory stimuli in typical infants. We included studies that used the NIRS technique to study language and cortical activation in response to auditory stimuli in typical infants between 0 and 3years old. We used the ROBINS-I tool to assess the quality and the risk of bias in the studies. Our analysis, encompassing 66 manuscripts, is presented in standardized tables for streamlined data extraction. We meticulously correlated findings with children's developmental stages, delineating crucial insights into brain development and its intricate interplay with language outcomes. Although most studies have a high risk for overall bias, especially due to the high loss of data in NIRS studies, the low risk in the other domains is predominant and homogeneous among the studies. Highlighted are the unique advantages of fNIRS for pediatric studies, underscored by its innate suitability for use in children. This review accentuates fNIRS' capacity to elucidate the neural correlates of language processing and the sequential steps of language acquisition. From birth, infants exhibit abilities that lay the foundation for language development. The progression from diffuse to specific neural activation patterns is extremely influenced by exposure to languages, social interaction, and prosodic features and, reflects the maturation of brain networks involved in language processing. In conclusion, fNIRS emerges as an indispensable functional imaging modality, providing insights into the temporal dynamics of language acquisition and associated developmental milestones. This synthesis presents the pivotal role of fNIRS in advancing our comprehension of early language development and paves the way for future research endeavors in this domain.

Acoustic features from speech as markers of depressive and manic symptoms in bipolar disorder: A prospective study.

Voice features could be a sensitive marker of affective state in bipolar disorder (BD). Smartphone apps offer an excellent opportunity to collect voice data in the natural setting and become a useful tool in phase prediction in BD. We investigate the relations between the symptoms of BD, evaluated by psychiatrists, and patients' voice characteristics. A smartphone app extracted acoustic parameters from the daily phone calls of n = 51 patients. We show how the prosodic, spectral, and voice quality features correlate with clinically assessed affective states and explore their usefulness in predicting the BD phase. A smartphone app (BDmon) was developed to collect the voice signal and extract its physical features. BD patients used the application on average for 208 days. Psychiatrists assessed the severity of BD symptoms using the Hamilton depression rating scale -17 and the Young Mania rating scale. We analyze the relations between acoustic features of speech and patients' mental states using linear generalized mixed-effect models. The prosodic, spectral, and voice quality parameters, are valid markers in assessing the severity of manic and depressive symptoms. The accuracy of the predictive generalized mixed-effect model is 70.9%-71.4%. Significant differences in the effect sizes and directions are observed between female and male subgroups. The greater the severity of mania in males, the louder (β = 1.6) and higher the tone of voice (β = 0.71), more clearly (β = 1.35), and more sharply they speak (β = 0.95), and their conversations are longer (β = 1.64). For females, the observations are either exactly the opposite-the greater the severity of mania, the quieter (β = -0.27) and lower the tone of voice (β = -0.21) and less clearly (β = -0.25) they speak - or no correlations are found (length of speech). On the other hand, the greater the severity of bipolar depression in males, the quieter (β = -1.07) and less clearly they speak (β = -1.00). In females, no distinct correlations between the severity of depressive symptoms and the change in voice parameters are found. Speech analysis provides physiological markers of affective symptoms in BD and acoustic features extracted from speech are effective in predicting BD phases. This could personalize monitoring and care for BD patients, helping to decide whether a specialist should be consulted.

An interactional practice of registering expectation discrepancy: the use of the turn-initial token are in Japanese

ABSTRACT This conversation analytic study offers an empirical analysis of the Japanese turn-initial interjection are. The interjectional are is said to be pragmatized from its use as a distal demonstrative and has been considered as an expression of a speaker’s internal state of being surprised at something. In contrast, this study argues that are is meticulously used for interactional purposes. Through the analysis of video- and audio-recorded noninstitutional conversation data, this study examines how speakers, by utilizing the turn-initial are, signal discrepancy in their expectation, assumption, or prior knowledge to accomplish various interactional tasks. This study does so by documenting are’s workings in two different sequential contexts, (i) are-prefaced display of noticing unexpected physical events during embodied activities and (ii) are-prefaced repair initiation as remediation in understanding. This research also examines prosodic features of are and points out a possible correlation between are’s prosodic features and are’s sequential locations.

MKELM based multi-classification model for foreign accent identification

The automatic identification of foreign accents can play a crucial role in various speech systems, including speaker identification, e-learning, telephone banking, and more. Additionally, it can greatly enhance the robustness of Automatic Speech Recognition (ASR) systems. Non-native accents in speech signals are characterized by distinct pronunciations, prosody, and voice characteristics of the speaker. However, automatically identifying foreign accents poses significant challenges, particularly in the context of multi-class modeling. Multi-classification models face difficulties in achieving high performance and dealing with computational challenges when confronted with multi-dimensional and unbalanced datasets, such as those with more than two accents. Furthermore, the choice of features remains a bottleneck problem for Foreign Accent Identification (FAID), further hindering performance in these tasks. Consequently, the accuracy of current systems is typically low. To address these challenges, this paper proposes a framework based on the Multi-Kernel Extreme Learning Machine (MKELM) model for the multi-classification of FAID. The MKELM model utilizes a novel weighted scheme to classify various non-native English accents, including Arabic, Chinese, Korean, French, and Spanish. The model first combines Mel-frequency cepstral coefficients (MFCCs) and prosodic features as input, trains pairwise binary classifiers independently, and subsequently employs a weighting scheme to distinguish between classes and identify accents. Through experiments, the proposed model achieves an accuracy rate of 84.72% using a paired weighting scheme. In contrast, the accuracy rate drops to 66.5% when employing the traditional non-weighted multi-classification scheme. A comparison with other models demonstrates the significant advantages of the proposed model in FAID multi-class classification, showcasing improved accuracy, reduced computational complexity (requiring fewer computations, faster learning rates, and shorter training time), and enhanced stability compared to state-of-the-art classification methods.

Prosody in narratives: An exploratory study with children with sex chromosomes trisomies

Although language delays are common in children with sex chromosome trisomies [SCT], no studies have analysed their prosodic abilities. Considering the importance of prosody in communication, this exploratory study aims to analyse the prosodic features of the narratives of 4-year-old children with SCT.Participants included 22 children with SCT and 22 typically developing [TD] children. The Narrative Competence Task was administered to elicit the child's narrative. Each utterance was prosodically analysed considering pitch and timing variables.Considering pitch, the only difference was the number of movements since the utterances of children with SCT were characterised by a lower speech modulation. However, considering the timing variables, children with SCT produced a faster speech rate and a shorter final syllable duration than TD children.Since both speech modulation and duration measures have important syntactic and pragmatic functions, further investigations should deeply analyse the prosodic skills of children with SCT in interaction with syntax and pragmatics.

ORAL POETIC SUBCORPUS: PROSODIC FEATURES OF POETIC DISCOURSE

ORAL POETIC SUBCORPUS: PROSODIC FEATURES OF POETIC DISCOURSE

The role of lexical and prosodic characteristics of mothers' child-directed speech for the early vocabulary development of Italian children with cochlear implants.

Variability in the vocabulary outcomes of children with cochlear implants (CIs) is partially explained by child-directed speech (CDS) characteristics. Yet, relatively little is known about whether and how mothers adapt their lexical and prosodic characteristics to the child's hearing status (before and after implantation, and compared with groups with normal hearing (NH)) and how important they are in affecting vocabulary development in the first 12 months of hearing experience. To investigate whether mothers of children with CIs produce CDS with similar lexical and prosodic characteristics compared with mothers of age-matched children with NH, and whether they modify these characteristics after implantation. In addition, to investigate whether mothers' CDS characteristics predict children's early vocabulary skills before and after implantation. A total of 34 dyads (17 with NH, 17 with children with CIs; ages = 9-32 months), all acquiring Italian, were involved in the study. Mothers' and children's lexical quantity (tokens) and variety (types), mothers' prosodic characteristics (pitch range and variability), and children's vocabulary skills were assessed at two time points, corresponding to before and 1 year post-CI activation for children with CIs. Children's vocabulary skills were assessed using parent reports; lexical and prosodic characteristics were observed in semi-structured mother-child interactions. Results showed that mothers of children with CIs produced speech with similar lexical quantity but lower lexical variety, and with increased pitch range and variability, than mothers of children with NH. Mothers generally increased their lexical quantity and variety and their pitch range between sessions. Children with CIs showed reduced expressive vocabulary and lower lexical quantity and variety than their peers 12 months post-CI activation. Mothers' prosodic characteristics did not explain variance in children's vocabulary skills; their lexical characteristics predicted children's early vocabulary and lexical outcomes, especially in the NH group, but were not related to later language development. Our findings confirm previous studies on other languages and support the idea that the lexical characteristics of mothers' CDS have a positive effect on children's early measures of vocabulary development across hearing groups, whereas prosodic cues play a minor role. Greater input quantity and quality may assist children in the building of basic language model representations, whereas pitch cues may mainly serve attentional and emotional processes. Results emphasize the need for additional longitudinal studies investigating the input received from other figures surrounding the child and its role for children's language development. What is already known on the subject Mothers' CDS is thought to facilitate and support language acquisition in children with various language developmental trajectories, including children with CIs. Because children with CIs are at risk for language delays and have acoustic processing limitations, their mothers may have to produce a lexically simpler but prosodically richer input, compared to mothers of children with NH. Yet, the literature reports mixed findings and no study to our knowledge has concurrently addressed the role of mothers' lexical and prosodic characteristics for children's vocabulary development before implantation and in the first 12 months of hearing experience. What this study adds to the existing knowledge The study shows that mothers of children with CIs produce input of similar quantity but reduced variety, and with heightened pitch characteristics, compared to mothers of children with NH. There was also a general increase in mothers' lexical quantity and variety, and in their pitch range, between sessions. Only their lexical characteristics predicted children's early vocabulary skills. Their lexical variety predicted children's expressive vocabulary and lexical variety only in the NH group. What are the practical and clinical implications of this work? These findings expand our knowledge about the effects of maternal input and may contribute to the improvement of early family-centred intervention programmes for supporting language development in children with CIs.