Speech Features Research Articles

A Study on Speech Recognition by a Neural Network Based on English Speech Feature Parameters

A Study on Speech Recognition by a Neural Network Based on English Speech Feature Parameters

Effects of Speech Characteristics on Electroglottographic and Instrumental Acoustic Voice Analysis Metrics in Women With Structural Dysphonia Before and After Treatment.

Literature suggests a dependency of the acoustic metrics, smoothed cepstral peak prominence (CPPS) and harmonics-to-noise ratio (HNR), on human voice loudness and fundamental frequency (F0). Even though this has been explained with different oscillatory patterns of the vocal folds, so far, it has not been specifically investigated. In the present work, the influence of three elicitation levels, calibrated sound pressure level (SPL), F0 and vowel on the electroglottographic (EGG) and time-differentiated EGG (dEGG) metrics hybrid open quotient (OQ), dEGG OQ and peak dEGG, as well as on the acoustic metrics CPPS and HNR, was examined, and their suitability for voice assessment was evaluated. In a retrospective study, 29 women with a mean age of 25 years (± 8.9, range: 18-53) diagnosed with structural vocal fold pathologies were examined before and after voice therapy or phonosurgery. Both acoustic and EGG signals were recorded simultaneously during the phonation of the sustained vowels /ɑ/, /i/, and /u/ at three elicited levels of loudness (soft/comfortable/loud) and unconstrained F0 conditions. A linear mixed-model analysis showed a significant effect of elicitation effort levels on peak dEGG, HNR, and CPPS (all p < .01). Calibrated SPL significantly influenced HNR and CPPS (both p < .01). Furthermore, F0 had a significant effect on peak dEGG and CPPS (p < .0001). All metrics showed significant changes with regard to vowel (all p < .05). However, the treatment had no effect on the examined metrics, regardless of the treatment type (surgery vs. voice therapy). The value of the investigated metrics for voice assessment purposes when sampled without sufficient control of SPL and F0 is limited, in that they are significantly influenced by the phonatory context, be it speech or elicited sustained vowels. Future studies should explore the diagnostic value of new data collation approaches such as voice mapping, which take SPL and F0 effects into account.

Research Paper on Emotion Recognition

As a recognizing in machine learning algorithm a significant amount of in different various field has been done in many technologies field of machines through which speech has a major impact research interest, especially in the affective computing domain. Increasing potential, algorithmic advancements, and applications in real-world. This human speech contains para-linguistic information that can be represented using different various quantitative features such as pitch, intensity for its deltaic result. It is commonly achieved following three key steps: data processing, feature extraction, and classification based on the underlying emotional features. The nature of these steps, help with the distinct features of human speech, to get the exact result through the underpin with the use of ML methods. Many techniques have been utilized to extract emotions from signals, including many well-established speech analysis and classification techniques. Emotion recognition the review covers databases used, emotions extracted, contributions made toward emotion recognition and limitations related to it. signals are an important but challenging component of Human-Computer Interaction (HCI) in machine learning aspect in computer machines through various different perspective and given signals

Impaired speech input and output processing abilities in children with cleft palate speech disorder.

Cleft lip and palate is one of the most common oral and maxillofacial deformities associated with a variety of functional disorders. Cleft palate speech disorder (CPSD) occurs the most frequently and manifests a series of characteristic speech features, which are called cleft speech characteristics. Some scholars believe that children with CPSD and poor speech outcomes may also have weaknesses in speech input processing ability, but evidence is still lacking so far. (1) To explore whether children with CPSD and speech output disorders also have defects in speech input processing abilities; (2) to explore the correlation between speech input and output processing abilities. Children in the experimental group were enrolled from Beijing Stomatological Hospital, Capital Medical University, and healthy volunteers were recruited as controls. Then three tasks containing real and pseudo words were performed sequentially. Reaction time, accuracy and other indicators in three tasks were collected and then analysed. The indicators in the experimental group were significantly lower than those in the control group. There was a strong correlation between speech input and output processing tasks. The performance of both groups when processing pseudo words in the three tasks was worse than that when dealing with real words. Compared with normal controls, children with CPSD have deficits in both speech input and output processing, and there is a strong correlation between speech input and output speech processing abilities. In addition, the pseudo words task was more challenging than the real word task for both groups. What is already known on the subject Children with cleft lip and palate often have speech sound disorders known as cleft palate speech disorder (CPSD). CPSD is characterised by consonant errors called cleft speech characteristics, which can persist even after surgery. Some studies suggest that poor speech outcomes in children with CPSD may be associated with deficits in processing speech input. However, this has not been validated in mainland China. What this paper adds to existing knowledge The results of our study indicate that children with CPSD exhibit poorer performance in three tasks assessing speech input and output abilities compared to healthy controls, suggesting their deficits in both speech input and output processing. Furthermore, a significant correlation was observed between speech input and output processing abilities. Additionally, both groups demonstrated greater difficulty in processing pseudo words compared to real words, as evidenced by their worse performance in dealing with pseudo words. What are the potential or actual clinical implications of this work? The pseudo word tasks designed and implemented in our study can be employed in future research and assessment of speech input and output abilities in Chinese Mandarin children with CPSD. Additionally, our findings revealed the significance of considering both speech output processing abilities and potential existence of speech input processing ability for speech and language therapists when evaluating and developing treatment options for children with CPSD as these abilities are also important for the development of literacy development.

WaveVC: Speech and Fundamental Frequency Consistent Raw Audio Voice Conversion

Voice conversion (VC) is a task for changing the speech of a source speaker to the target voice while preserving linguistic information of the source speech. The existing VC methods typically use mel-spectrogram as both input and output, so a separate vocoder is required to transform mel-spectrogram into waveform. Therefore, the VC performance varies depending on the vocoder performance, and noisy speech can be generated due to problems such as train-test mismatch. In this paper, we propose a speech and fundamental frequency consistent raw audio voice conversion method called WaveVC. Unlike other methods, WaveVC does not require a separate vocoder and can perform VC directly on raw audio waveform using 1D convolution. This eliminates the issue of performance degradation caused by the train-test mismatch of the vocoder. In the training phase, WaveVC employs speech loss and F0 loss to preserve the content of the source speech and generate F0 consistent speech using the pre-trained networks. WaveVC is capable of converting voices while maintaining consistency in speech and fundamental frequency. In the test phase, the F0 feature of the source speech is concatenated with a content embedding vector to ensure the converted speech follows the fundamental frequency flow of the source speech. WaveVC achieves higher performances than baseline methods in both many-to-many VC and any-to-any VC. The converted samples are available online.

Conversational Speech Behaviors Are Context Dependent.

According to the interpersonal synergy model of spoken dialogue, interlocutors modify their communicative behaviors to meet the contextual demands of a given conversation. Although a growing body of research supports this postulation for linguistic behaviors (e.g., semantics, syntax), little is understood about how this model applies to speech behaviors (e.g., speech rate, pitch). The purpose of this study is to test the hypothesis that interlocutors adjust their speech behaviors across different conversational tasks with different conversational goals. In this study, 28 participants each engaged in two different types of conversations (i.e., relational and informational) with two partners (i.e., Partner 1 and Partner 2), yielding a total of 112 conversations. We compared six acoustic measures of participant speech behavior across conversational task and partner. Linear mixed-effects models demonstrated significant differences between speech feature measures in informational and relational conversations. Furthermore, these findings were generally robust across conversations with different partners. Results suggest that contextual demands influence speech behaviors. These findings provide empirical support for the interpersonal synergy model and highlight important considerations for assessing speech behaviors in individuals with communication disorders.

Prosodic features of polite speech

Abstract This paper uses interactional data to investigate the acoustic characteristics of polite or deferential speech in Korean. We asked fourteen Korean speakers to perform two tasks with two different interlocutors: a status superior and a friend. Consistent with previous studies of non-interactional data, deferential speech has lower pitch and shimmer, and quieter final syllables. However, divergent from previous studies, deferential speech featured higher jitter (in some locations), higher shimmer and higher H1-H2 (on one task). Through analysis of different locations in prosodic structure, we found that females used more pitch variation on final syllables in deferential speech. We argue that these mixed results show the importance of context in signalling vocal politeness, and also complexities of using interactional data. The findings advance the study of multimodal politeness beyond the analysis of experimental data.

Predicting Social Competence in Autistic and Non-Autistic Children: Effects of Prosody and the Amount of Speech Input.

Autistic individuals often face challenges perceiving and expressing emotions, potentially stemming from differences in speech prosody. Here we explore how autism diagnoses between groups, and measures of social competence within groups may be related to, first, children's speech characteristics (both prosodic features and amount of spontaneous speech), and second, to these two factors in mothers' speech to their children. Autistic (n = 21) and non-autistic (n = 18) children, aged 7-12 years, participated in a Lego-building task with their mothers, while conversational speech was recorded. Mean F0, pitch range, pitch variability, and amount of spontaneous speech were calculated for each child and their mother. The results indicated no differences in speech characteristics across autistic and non-autistic children, or across their mothers, suggesting that conversational context may have large effects on whether differences between autistic and non-autistic populations are found. However, variability in social competence within the group of non-autistic children (but not within autistic children) was predictive of children's mean F0, pitch range and pitch variability. The amount of spontaneous speech produced by mothers (but not their prosody) predicted their autistic children's social competence, which may suggest a heightened impact of scaffolding for mothers of autistic children. Together, results suggest complex interactions between context, social competence, and adaptive parenting strategies in driving prosodic differences in children's speech.

Squeeze-and-excitation 3D convolutional attention recurrent network for end-to-end speech emotion recognition

Speech emotion recognition (SER) is difficult since emotions are complex and dynamic processes involving multiple dimensions and sub-dimensions. Feature extraction is a challenging step in SER, where relevant features are extracted from the speech to identify emotional states accurately. Overcoming these challenges is essential to ensuring the effectiveness and robustness of the SER system. 3D-convolutional neural networks (3D-CNNs) are successfully used for feature extraction in SER. Speech signals can be converted into spectrogram-like representations where one axis represents time, another frequency, and the third can represent additional context or features. For modeling the dynamic nature of speech, a squeeze-and-excitation-based 3D-CNN model is employed for capturing temporal and spatial features of speech. Attention Gated Recurrent Units (AGRU) are applied to the extracted features for learning long-range temporal dependencies and selecting more informative features. The extraction and selection of the spatio-temporal feature representation lead to a hierarchical representation of the input speech. The fusion of squeeze-and-excitation 3D-CNN and AGRU (named SE3D-CARN) is evaluated on two datasets, EMO-BD and IEMOCAP, to identify various emotional states in speech. The proposed SER model reached an accuracy of 94.2% and 81.1% on the EMO-BD and IEMOCAP datasets, respectively.

Optimization-based planning of speech articulation using general Tau Theory

This paper presents a model of speech articulation planning and generation based on General Tau Theory and Optimal Control Theory. Because General Tau Theory assumes that articulatory targets are always reached, the model accounts for speech variation via context-dependent articulatory targets. Targets are chosen via the optimization of a composite objective function. This function models three different task requirements: maximal intelligibility, minimal articulatory effort and minimal utterance duration. The paper shows that systematic phonetic variability can be reproduced by adjusting the weights assigned to each task requirement. Weights can be adjusted globally to simulate different speech styles, and can be adjusted locally to simulate different levels of prosodic prominence. The solution of the optimization procedure contains Tau equation parameter values for each articulatory movement, namely position of the articulator at the movement offset, movement duration, and a parameter which relates to the shape of the movement’s velocity profile. The paper presents simulations which illustrate the ability of the model to predict or reproduce several well-known characteristics of speech. These phenomena include close-to-symmetric velocity profiles for articulatory movement, variation related to speech rate, centralization of unstressed vowels, lengthening of stressed vowels, lenition of unstressed lingual stop consonants, and coarticulation of stop consonants.

PERLOCATIVE PRAGMATICS OF DIRECT AND INDIRECT INTERROGATIVE SPEECH ACTS IN THE KAZAKH LANGUAGE

The article analyzes the concepts of categories of direct and indirect communication in linguistics, types and features of indirect interrogative speech acts, reveals the prelocutive pragmatics of direct and indirect interrogative speech acts in the Kazakh language. Namely: if the direct meaning of interrogative sentences is getting an answer, then indirect questions are used in such functions as showing politeness, indirect posing a question, emphasizing a request, posing a question in a suspicious expression, transmitting a request with gestures, evaluation, resentment, chagrin. In a direct interrogative speech act, the illocutionary effect is expressed through the linguistic meaning of the linguistic means that make up speech, while the perlocutionary pragmatics of indirect interrogative speech acts depends on the use of language units in a figurative sense, on the background knowledge of the speaker and listener and the conditions of communication. At the same time, an indirect interrogative speech act enhances the artistic potential of the language and allows expressing various semantic shades.The study used methods of structural grammar «from form to meaning», functional grammar «from meaning to form», structural-semantic description, functional-semantic and contextual analysis, differentiation and hypothetical interpretation.The scientific and theoretical basis of the scientific conclusions obtained in the course of the study were the works on structural, communicative, functional grammar and pragmatic linguistics in linguistics. In addition, the given language examples from various sources (colloquial speech, works of poets and writers) contributed to the argumentation of scientific and practical conclusions presented in the article

Machine Learning Based Speech Emotion Recognition System

In the last decade, there has been significant research into Automatic Speech Emotion Recognition (SER). The primary goal of SER is to improve human-machine interfaces. It can also monitor someone's psychological state for lie detection applications. Recently, speech emotion recognition has found uses in medicine and forensics. This paper recognizes 7 emotions using pitch and prosody features. The majority of speech features used here are in the time domain. A Support Vector Machine (SVM) classifier categorizes the emotions. The Berlin emotional database was used for this task. A good recognition rate of 81% was achieved. The reference paper for this work recognized 4 emotions and obtained a 94.2% recognition rate. However, the reference paper used a more complex hybrid classifier, while this work focuses on recognizing more emotions with a simpler model. . KeyWords: Blindness, Smart Stick, BlinDar, GPS, ESP8266, Internet of Things, RF Tx/Rx, MQ2.

Does Native Language Matter in Perceptual Ratings of Dysarthria?

Despite the general agreement that dysarthria characteristics are largely language-independent, few efforts have attempted a systematic comparison across languages. To examine the role of native languages in the perception of speech characteristics of dysarthria secondary to Parkinson's disease (PD), auditory-perceptual ratings of dysarthria, and confidence level of the judgments were compared between two listener groups: language-matched and language-crossed. A total of 60 listeners (35 native speakers of Korean and 25 native speakers of American English) estimated speech abnormality for 20 speech dimensions using a visual analog scale method for both language-matched and language-crossed speech stimuli. Speech stimuli were passage readings of the respective languages obtained from individuals with and without PD. For speech dimension ratings, eight of 20 speech dimensions revealed significant differences in response to PD speech between the two listener groups, for most of which, language-crossed listeners' estimation was lower (i.e., more impaired) than language-matched listeners. For confidence-level ratings, language-matched listeners were less confident in the ratings of speakers with PD compared to the language-crossed listeners. The data support both language-universal and language-specific aspects in perceiving dysarthria characteristics, such that native language plays a role, especially when rating articulatory- and rhythmic-related characteristics. The findings are discussed with respect to the role of linguistic information, such as phonetic inventories and prosodic structures, in perceiving dysarthria characteristics.

Integrating Large Language Models (LLMs) and Deep Representations of Emotional Features for the Recognition and Evaluation of Emotions in Spoken English

This study is dedicated to developing an innovative method for evaluating spoken English by integrating large language models (LLMs) with effective space learning, focusing on the analysis and evaluation of emotional features in spoken language. Addressing the limitation of current spoken English evaluation software that primarily focuses on acoustic features of speech (such as pronunciation, frequency, and prosody) while neglecting emotional expression, this paper proposes a method capable of deeply recognizing and evaluating emotional features in speech. The core of the method comprises three main parts: (1) the creation of a comprehensive spoken English emotion evaluation dataset combining emotionally rich speech data synthesized using LLMs with the IEMOCAP dataset and student spoken audio; (2) an emotion feature encoding network based on transformer architecture, dedicated to extracting effective spatial features from audio; (3) an emotion evaluation network for the spoken English language that accurately identifies emotions expressed by Chinese students by analyzing different audio characteristics. By decoupling emotional features from other sound characteristics in spoken English, this study achieves automated emotional evaluation. This method not only provides Chinese students with the opportunity to improve their ability to express emotions in spoken English but also opens new research directions in the fields of spoken English teaching and emotional expression evaluation.

Perception of the prosodic features of newscaster speech

Perception of the prosodic features of newscaster speech

Research Of Singing With Note-name Recognition Algorithm Based On End-to-End Model

With the rapid development of artificial intelligence, and people are not only seeking material needs, but also pursuing happiness and spiritual aspirations. Especially some singing talent shows appear, but there are many controversial results. In order to solve the problem of singing with note-name evaluation and evaluate the singer's singing level in the professional music education of solfeggio and ear training, the paper uses artificial intelligence algorithms to recognize the note-name. Firstly, preprocess the singing sound data, including pre emphasis, framing, and windowing to reduce the impact of noise, and extract the acoustic features of the singing sound, the acoustic speech features are organized into a form suitable for DCNN input and a DCNN model is designed to obtain the pinyin form of the singing sound, the softmax layer adopts an end-to-end CTC structure, which classifies the input singing speech, finds corresponding phoneme sequences, and obtains the output results. The end-to-end CTC structure is used to classify and optimize the recognition process, and finally the state of the obtained features is output. And then singing pronunciation dictionary generates candidate word sequences based on the mapping relationship between phonemes and notes. Finally, based on the acoustic model score, the candidate note sequence with the highest score is obtained through decoder processing, and obtained the singing with note-name result of the singing sound. In order to further improve recognition performance, the paper introduces a new CTC-DCNN acoustic model. In this model, residual blocks can transfer input features to the output part through shortcuts, allowing the multi-layer convolutional speech features to be preserved as much as possible. At the same time, the deep structure can also better achieve the extraction and analysis of speech features. A new and improved CTC-DCNN acoustic model is obtained by optimizing the maxout function. The algorithm proposed in this paper is fair and equal in obtaining information, and no one participates in the entire process of obtaining scores. It is believed that the scoring results obtained in this way should be more objective.

Cepstral coefficients effectiveness for gunshot classifying

This paper analyses the efficiency of various frequency cepstral coefficients (FCC) in a non-speech application, specifically in classifying acoustic impulse events-gunshots. There are various methods for such event identification available. The majority of these methods are based on time or frequency domain algorithms. However, both of these domains have their limitations and disadvantages. In this article, an FCC, combining the advantages of both frequency and time domains, is presented and analyzed. These originally speech features showed potential not only in speech-related applications but also in other acoustic applications. The comparison of the classification efficiency based on features obtained using four different FCC, namely mel-FCC (MFCC), inverse mel-frequency cepstral coefficients (IMFCC), linear-frequency cepstral coefficients (LFCC), and gammatone-frequency cepstral coefficients (GTCC) is presented. An optimal frame length for an FCC calculation is also explored. Various gunshots from short guns and rifle guns of different calibers and multiple acoustic impulse events, similar to the gunshots, to represent false alarms are used. More than 600 acoustic events records have been acquired and used for training and validation of two designed classifiers, support vector machine, and neural network. Accuracy, recall and Matthew’s correlation coefficient measure the classification success rate. The results reveal the superiority of GFCC to other analyzed methods.

A review of social background profiling of speakers from speech accents.

Social background profiling of speakers is heavily used in areas, such as, speech forensics, and tuning speech recognition for accuracy improvement. This article provides a survey of recent research in speaker background profiling in terms of accent classification and analyses the datasets, speech features, and classification models used for the classification tasks. The aim is to provide a comprehensive overview of recent research related to speaker background profiling and to present a comparative analysis of the achieved performance measures. Comprehensive descriptions of the datasets, speech features, and classification models used in recent research for accent classification have been presented, with a comparative analysis made on the performance measures of the different methods. This analysis provides insights into the strengths and weaknesses of the different methods for accent classification. Subsequently, research gaps have been identified, which serve as a useful resource for researchers looking to advance the field.

Using Objective Speech Analysis Techniques for the Clinical Diagnosis and Assessment of Speech Disorders in Patients with Multiple Sclerosis.

Multiple sclerosis (MS) is one of the chronic and neurodegenerative diseases of the central nervous system (CNS). It generally affects motor, sensory, cerebellar, cognitive, and language functions. It is thought that identifying MS speech disorders using quantitative methods will make a significant contribution to physicians in the diagnosis and follow-up of MS patients. In this study, it was aimed to investigate the speech disorders of MS via objective speech analysis techniques. The study was conducted on 20 patients diagnosed with MS according to McDonald's 2017 criteria and 20 healthy volunteers without any speech or voice pathology. Speech data obtained from patients and healthy individuals were analyzed with the PRAAT speech analysis program, and classification algorithms were tested to determine the most effective classifier in separating specific speech features of MS disease. As a result of the study, the K-nearest neighbor algorithm (K-NN) was found to be the most successful classifier (95%) in distinguishing pathological sounds which were seen in MS patients from those in healthy individuals. The findings obtained in our study can be considered as preliminary data to determine the voice characteristics of MS patients.

Examining Language, Speech and Behaviour Characteristics: A Cross-Sectional Study in Saudi Arabia Using the Arabic Version of Gilliam Autism Rating Scale-Third Edition.

Autism spectrum disorder (ASD) exhibits diverse manifestations influenced by demographic factors. This study evaluates these variations within Saudi Arabia, aiming to investigate language, speech and behaviour characteristics across different demographics in Saudi Arabia using the Arabic Version of the Gilliam Autism Rating Scale-Third Edition (A-GARS-3). Employing a cross-sectional design, 178 participants were stratified by developmental status (n = 124 school settings, n = 54 clinical setting), sex (Females = 77, Males =101), age (range = 3-22), and geographical region (different provinces in Saudi Arabia). The A-GARS-3 measured ASD manifestations across six subscales. The study identified significant differences in ASD manifestations by developmental status, with higher ASD likelihood and severity in clinical settings. Younger children showed more pronounced ASD characteristics, and males were slightly more likely to be diagnosed with ASD. Geographical analysis revealed regional differences in severity. The findings underline the importance of demographic considerations in ASD assessment and diagnosis, suggesting the need for age-specific and culturally sensitive approaches. The A-GARS-3 is a reliable tool for the Saudi context. Regional disparities in ASD prevalence and severity indicate a need for tailored health policies and resources across Saudi provinces.