Speech Task Research Articles

Automatic Assessment of Parkinson's Disease Using Speech Representations of Phonation and Articulation

Speech from people with Parkinson's disease (PD) are likely to be degraded on phonation, articulation, and prosody. Motivated to describe articulation deficits comprehensively, we investigated 1) the universal phonological features that model articulation manner and place, also known as speech attributes, and 2) glottal features capturing phonation characteristics. These were further supplemented by, and compared with, prosodic features using a popular compact feature set and standard MFCC. Temporal characteristics of these features were modeled by convolutional neural networks. Besides the features, we were also interested in the speech tasks for collecting data for automatic PD speech assessment, like sustained vowels, text reading, and spontaneous monologue. For this, we utilized a recently collected Finnish PD corpus (PDSTU) as well as a Spanish database (PC-GITA). The experiments were formulated as regression problems against expert ratings of PD-related symptoms, including ratings of speech intelligibility, voice impairment, overall severity of communication disorder on PDSTU, as well as on the Unified Parkinson's Disease Rating Scale (UPDRS) on PC-GITA. The experimental results show: 1) the speech attribute features can well indicate the severity of pathologies in parkinsonian speech; 2) combining phonation features with articulatory features improves the PD assessment performance, but requires high-quality recordings to be applicable; 3) read speech leads to more accurate automatic ratings than the use of sustained vowels, but not if the amount of speech is limited to correspond to the sustained vowels in duration; and 4) jointly using data from several speech tasks can further improve the automatic PD assessment performance.

Machine Learning Based Scoring System to Predict the Risk and Severity of Ataxic Speech Using Different Speech Tasks.

The assessment of speech in Cerebellar Ataxia (CA) is time-consuming and requires clinical interpretation. In this study, we introduce a fully automated objective algorithm that uses significant acoustic features from time, spectral, cepstral, and non-linear dynamics present in microphone data obtained from different repeated Consonant-Vowel (C-V) syllable paradigms. The algorithm builds machine-learning models to support a 3-tier diagnostic categorisation for distinguishing Ataxic Speech from healthy speech, rating the severity of Ataxic Speech, and nomogram-based supporting scoring charts for Ataxic Speech diagnosis and severity prediction. The selection of features was accomplished using a combination of mass univariate analysis and elastic net regularization for the binary outcome, while for the ordinal outcome, Spearman's rank-order correlation criterion was employed. The algorithm was developed and evaluated using recordings from 126 participants: 65 individuals with CA and 61 controls (i.e., individuals without ataxia or neurotypical). For Ataxic Speech diagnosis, the reduced feature set yielded an area under the curve (AUC) of 0.97 (95% CI 0.90-1), the sensitivity of 97.43%, specificity of 85.29%, and balanced accuracy of 91.2% in the test dataset. The mean AUC for severity estimation was 0.74 for the test set. The high C-indexes of the prediction nomograms for identifying the presence of Ataxic Speech (0.96) and estimating its severity (0.81) in the test set indicates the efficacy of this algorithm. Decision curve analysis demonstrated the value of incorporating acoustic features from two repeated C-V syllable paradigms. The strong classification ability of the specified speech features supports the framework's usefulness for identifying and monitoring Ataxic Speech.

Development and validation of a Spanish-language spatial release from masking task in a Mexican population.

This study validates a new Spanish-language version of the Coordinate Response Measure (CRM) corpus using a well-established measure of spatial release from masking (SRM). Participants were 96 Spanish-speaking young adults without hearing complaints in Mexico City. To present the Spanish-language SRM test, we created new recordings of the CRM with Spanish-language Translations and updated the freely available app (PART; https://ucrbraingamecenter.github.io/PART_Utilities/) to present materials in Spanish. In addition to SRM, we collected baseline data on a battery of non-speech auditory assessments, including detection of frequency modulations, temporal gaps, and modulated broadband noise in the temporal, spectral, and spectrotemporal domains. Data demonstrate that the newly developed speech and non-speech tasks show similar reliability to an earlier report in English-speaking populations. This study demonstrates an approach by which auditory assessment for clinical and basic research can be extended to Spanish-speaking populations for whom testing platforms are not currently available.

Speaker Verification Based on Single Channel Speech Separation

In multi-speaker scenarios, speech processing tasks like speaker identification and speech recognition are susceptible to noise and overlapped voices. As the overlapped voices are a complicated mixture of signals, a target extraction method from this mixture is a good front end solution for further processing like understanding and classifying. The quality of speech separation can be assessed by the noise ratio or subjective scoring and can also be assessed by accuracy of the downstream processing tasks like speaker identification. In order to make the separation model and speaker identification model more adapted to complex multi-speaker speech overlapping scenarios, this research investigates the speech separation model and incorporate with a voiceprint recognition task. This paper proposes a feature-scale single channel speech separation network connected to a back end speaker verification network with MFCCT feature, so the accuracy of speaker identification indicates the quality of speech separation task. The datasets are prepared by synthesizing Voxceleb1 data, and used for training and testing. The results show that using an objective downstream evaluation can effectively improve the overall performance, as the optimized speech separation model significantly reduced the error rate of speaker verification.

Perfil da fluência na fala espontânea, leitura e no reconto de textos de adultos que gaguejam

ABSTRACT Purpose to describe the profile of fluency concerning the typology of disfluencies, speed, and frequency of disruptions in spontaneous speech, reading, and retelling; to compare the fluency profile in adults who stutter in spontaneous speech, reading, and retelling of text. Methods The present work is a cross-sectional comparative study with a sample composed of 15 adults who stutter of both sexes, with higher education or equivalent to complete elementary school II. Samples were collected in the tasks of spontaneous speech, reading, and text retelling through video calls made individually with the participants. The first 200 syllables expressed in each task were transcribed and analyzed according to the Fluency Profile Assessment Protocol (FPAP). The study compared the frequency of common and stuttering disfluencies and the speed in the different tasks surveyed. The Kruskal & Wallis test was used together with Duncan's multiple comparisons test to compare the medians and verify possible differences between the tasks researched with a significance level of 5%. Results The reading task presented a lower number of common disfluencies and a percentage of speech discontinuity about spontaneous speech and retelling tasks. No statistically significant differences were found between stuttering disfluencies in the three tasks surveyed. Conclusion This study showed that there are differences in the occurrence of common disfluencies - hesitations, interjections, and revisions - and in the percentage of speech discontinuity during an oral reading of adults who stutter concerning spontaneous speech and text retelling.

Validation of automated pipeline for the assessment of a motor speech disorder in amyotrophic lateral sclerosis (ALS).

Amyotrophic lateral sclerosis (ALS) frequently causes speech impairments, which can be valuable early indicators of decline. Automated acoustic assessment of speech in ALS is attractive, and there is a pressing need to validate such tools in line with best practices, including analytical and clinical validation. We hypothesized that data analysis using a novel speech assessment pipeline would correspond strongly to analyses performed using lab-standard practices and that acoustic features from the novel pipeline would correspond to clinical outcomes of interest in ALS. We analyzed data from three standard speech assessment tasks (i.e., vowel phonation, passage reading, and diadochokinesis) in 122 ALS patients. Data were analyzed automatically using a pipeline developed by Winterlight Labs, which yielded 53 acoustic features. First, for analytical validation, data were analyzed using a lab-standard analysis pipeline for comparison. This was followed by univariate analysis (Spearman correlations between individual features in Winterlight and in-lab datasets) and multivariate analysis (sparse canonical correlation analysis (SCCA)). Subsequently, clinical validation was performed. This included univariate analysis (Spearman correlation between automated acoustic features and clinical measures) and multivariate analysis (interpretable autoencoder-based dimensionality reduction). Analytical validity was demonstrated by substantial univariate correlations (Spearman's ρ > 0.70) between corresponding pairs of features from automated and lab-based datasets, as well as interpretable SCCA feature groups. Clinical validity was supported by strong univariate correlations between automated features and clinical measures (Spearman's ρ > 0.70), as well as associations between multivariate outputs and clinical measures. This novel, automated speech assessment feature set demonstrates substantial promise as a valid tool for analyzing impaired speech in ALS patients and for the further development of these technologies.

Multiscale Convolutional Transformer for EEG Classification of Mental Imagery in Different Modalities.

A new kind of sequence-to-sequence model called a transformer has been applied to electroencephalogram (EEG) systems. However, the majority of EEG-based transformer models have applied attention mechanisms to the temporal domain, while the connectivity between brain regions and the relationship between different frequencies have been neglected. In addition, many related studies on imagery-based brain-computer interface (BCI) have been limited to classifying EEG signals within one type of imagery. Therefore, it is important to develop a general model to learn various types of neural representations. In this study, we designed an experimental paradigm based on motor imagery, visual imagery, and speech imagery tasks to interpret the neural representations during mental imagery in different modalities. We conducted EEG source localization to investigate the brain networks. In addition, we propose the multiscale convolutional transformer for decoding mental imagery, which applies multi-head attention over the spatial, spectral, and temporal domains. The proposed network shows promising performance with 0.62, 0.70, and 0.72 mental imagery accuracy with the private EEG dataset, BCI competition IV 2a dataset, and Arizona State University dataset, respectively, as compared to the conventional deep learning models. Hence, we believe that it will contribute significantly to overcoming the limited number of classes and low classification performances in the BCI system.

A Time-Frequency Attention Module for Neural Speech Enhancement

Speech enhancement plays an essential role in a wide range of speech processing applications. Recent studies on speech enhancement tend to investigate how to effectively capture the long-term contextual dependencies of speech signals to boost performance. However, these studies generally neglect the time-frequency (T-F) distribution information of speech spectral components, which is equally important for speech enhancement. In this paper, we propose a simple yet very effective network module, which we term the T-F attention (TFA) module, that uses two parallel attention branches, i.e., time-frame attention and frequency-channel attention, to explicitly exploit position information to generate a 2-D attention map to characterise the salient T-F speech distribution. We validate our TFA module as part of two widely used backbone networks (residual temporal convolution network and Transformer) and conduct speech enhancement with four most popular training objectives. Our extensive experiments demonstrate that our proposed TFA module consistently leads to substantial enhancement performance improvements in terms of the five most widely used objective metrics, with negligible parameter overheads. In addition, we further evaluate the efficacy of speech enhancement as a front-end for a downstream speech recognition task. Our evaluation results show that the TFA module significantly improves the robustness of the system to noisy conditions.

Profile of fluency in spontaneous speech, reading, and retelling of texts by adults who stutter.

to describe the profile of fluency concerning the typology of disfluencies, speed, and frequency of disruptions in spontaneous speech, reading, and retelling; to compare the fluency profile in adults who stutter in spontaneous speech, reading, and retelling of text. The present work is a cross-sectional comparative study with a sample composed of 15 adults who stutter of both sexes, with higher education or equivalent to complete elementary school II. Samples were collected in the tasks of spontaneous speech, reading, and text retelling through video calls made individually with the participants. The first 200 syllables expressed in each task were transcribed and analyzed according to the Fluency Profile Assessment Protocol (FPAP). The study compared the frequency of common and stuttering disfluencies and the speed in the different tasks surveyed. The Kruskal & Wallis test was used together with Duncan's multiple comparisons test to compare the medians and verify possible differences between the tasks researched with a significance level of 5%. The reading task presented a lower number of common disfluencies and a percentage of speech discontinuity about spontaneous speech and retelling tasks. No statistically significant differences were found between stuttering disfluencies in the three tasks surveyed. This study showed that there are differences in the occurrence of common disfluencies - hesitations, interjections, and revisions - and in the percentage of speech discontinuity during an oral reading of adults who stutter concerning spontaneous speech and text retelling.

Distributed Sensor Selection for Speech Enhancement With Acoustic Sensor Networks

In distributed acoustic sensor networks, only a few nodes make a significant contribution to speech enhancement tasks. Using these most informative nodes instead of the entire network not only avoids unnecessary energy consumption but also prolongs the lifetime of sensors. To this end, a sensor selection method for distributed speech enhancement is proposed. The best subset of microphone nodes is determined by maximizing the signal-to-noise ratio (SNR), while keeping the activated nodes connected with each other. The above criterion involves an integer and non-linear programming, which is linearized with multiple base-3 sub-optimization problems, and each of them is solved by a state-of-the-art steepest descent (SD) algorithm. In addition, a greedy searching strategy is presented to select sensors rapidly. Finally, a distributed SD algorithm is further derived, which is more suitable for distributed sensor networks. The proposed method can obtain the optimal subnetwork in noisy and reverberant environments. Unlike the existing approaches, it can select nodes from a microphone network with arbitrary communication graphs. Moreover, it requires only local communications among nodes without an external central processor. Experimental results confirm the validity of the proposed method.

Utilidad del índice diadococinético DDKCVP% para la detección de imprecisiones articulatorias en enfermedad de Parkinson: estudio exploratorio preliminar

ABSTRACT Purpose: to explore whether the diadochokinetic index of syllable production variability (DDKcvp%) is useful for the detection of articulatory inaccuracies in different stages of Parkinson's disease (PD). Methods: a diadochokinetic speech task of the consonant-vowel type [pa-pa-pa] was applied to 18 people with PD (classified in stages of progression from 1 to 5 according to the Hoehn & Yahr scale) to determine their diadochokinetic performance. Records of the study group were compared to a reference value obtained from a control group made up of 40 subjects. Results: 77.78% of the study group showed articulatory inaccuracies. The DDKcvp% index allowed to detect articulatory difficulties from stage 1 of PD, and its variation showed that articulatory deficits were permanent from stage 2 onwards. A significant and direct correlation was obtained between the increase in the DDKcvp% index and the increase in the stage of evolution of the participants. Conclusion: DDKcvp% index evaluated by means of a simple and fast consonant-vowel diadochokinetic task [pa-pa-pa], could be useful to determine articulatory inaccuracies in different stages of PD, although it is suggested that investigation of this index in larger sample sizes be conducted.

Proposal of requirements for the development of a training simulator for the auditory-perceptual judgment of voice.

to identify a set of requirements for the development of an auditory-perceptual training simulator (APT) based on the experience of professors who provide APT. This is a cross-sectional, descriptive study with a quantitative approach. Twenty-two professors answered an online questionnaire containing 31 items related to APT, involving items about the professional profile, conditions for APT in undergraduate and postgraduate courses in Speech Therapy, APT structure, and evaluation of the APT effect. it was observed that there is a variation in APT procedures performed in Brazil. The main requirements indicated by the respondents for the APT involve the use of synthesized voices in the initial moments, followed by human voices later; the use of speech tasks with sustained vowels and connected speech; the insertion of complementary information such as gender, age, the profession of the speaker and the spectrography of the vocal signal; training with a minimum time of six hours; the evaluation of the training effect by comparing intra- and inter-judge agreement before and after training; the addition of the parameters of general degree of vocal deviation, roughness, breathiness, and strain; the use of validated continuous and numerical scales; and offering it from the second year of the undergraduate program. although there is variability in the response of experts, a minimum set of requirements indicated for performing APT with new judges was identified.

Интернационализмы как лингводидактический ресурс для эффективного обучения немецкому языку

Foreign language teachers, working with non-linguistic students who have low level of language, are frequently facing the challenges that require intensive and effective methods in teaching. One of the methods to master foreign vocabulary is the use of modern internationalisms. The purpose of this work is to analyze the didactic potential of internationalisms in foreign language teaching, to develop a set of exercises for internationalisms acquisition at the elementary level of learning the German language and to test it in groups of non-linguistic beginners. The methodological basis of the study was the existing approaches for selection of lexical units, the analysis of methodological techniques and strategies when working with international vocabulary. As a result, we have developed and visualized a set of exercises based on the international vocabulary of the German language; then we tested it in student groups of a technical university. The learning kit comprising three stages is designed for the first month of training. The first two stages are aimed at reading skills development through reproductive exercises with the gradual introduction of conditional speech tasks in order to immerse students in natural communicative context step by step. At the third stage, the text and audio material is added. The productive exercises get the students prepared for the basic curriculum. Testing of acquisition showed positive outcomes. Internationalisms served as lexical basis for further enlargement of students' vocabulary; it became an assimilation mechanism in learning and mastering German by students of non-linguistic majoring.

Комплекс проблемных иноязычных заданий, направленных на профессиональную ориентацию обучающихся среднего профессионального образования

Importance. The development of secondary vocational education is the main feature of the education system today. Now graduates of schools after the 9th grade prefer to study under the programs of secondary vocational education for the purpose of subsequent admission to the university. In this connection, there is a need for professional oriented training, which can be implemented in foreign language lessons through an integrated approach (subject-language integrated learning). The purpose of the study is to develop a set of problematic foreign language tasks aimed at professional orientation of students of secondary vocational education. Materials and methods. The developed complex of problematic foreign language tasks of a professional oriented nature for students in the secondary vocational education system is based on works on integrated teaching of foreign languages, professional orientation teaching of a foreign language, various typologies of foreign language tasks. In our study, the typology of E.I. Passes exercises is taken as a basis, which consists of conditional speech and speech tasks. The research method was theoretical analysis of literature in the field of foreign language teaching methods, processing of the received material and its synthesis. Results and Discussion. For the practical implementation of professional oriented training, a set of problematic foreign language tasks aimed at the professional orientation of students of secondary vocational education was developed. The complex of problem tasks consists of conditional speech tasks (on vocabulary, phonetics, grammar and translation) and speech exercises (professionally oriented). Conclusion. The novelty of the conducted research lies in the development of its own complex of problematic foreign language tasks – conditional speech tasks (for vocabulary, grammar, phonetics, exercises in translation) and speech integrated tasks of a professional orientation. This typology of tasks will fully realize the potential of an integrated approach with a professional oriented goal for students in the system of secondary vocational education.

Extracting Phonetic Posterior-Based Features for Detecting Multiple Sclerosis From Speech.

Multiple sclerosis (MS) is a chronic inflammatory disease of the central nervous system which, in addition to affecting motor and cognitive functions, may also lead to specific changes in the speech of patients. Speech production, comprehension, repetition and naming tasks, as well as structural and content changes in narratives, might indicate a limitation of executive functions. In this study we present a speech-based machine learning technique to distinguish speakers with relapsing-remitting subtype MS and healthy controls (HC). We exploit the fact that MS might cause a motor speech disorder similar to dysarthria, which, with our hypothesis, might affect the phonetic posterior estimates supplied by a Deep Neural Network acoustic model. From our experimental results, the proposed posterior posteriorgram-based feature extraction approach is useful for detecting MS: depending on the actual speech task, we obtained Equal Error Rate values as low as 13.3%, and AUC scores up to 0.891, indicating a competitive and more consistent classification performance compared to both the x-vector and the openSMILE 'ComParE functionals' attributes. Besides this discrimination performance, the interpretable nature of the phonetic posterior features might also make our method suitable for automatic MS screening or monitoring the progression of the disease. Furthermore, by examining which specific phonetic groups are the most useful for this feature extraction process, the potential utility of the proposed phonetic features could also be utilized in the speech therapy of MS patients.

Auditory-perceptual evaluation of voice: comparing different speech tasks to identify children with and without laryngeal lesions

To compare the vowel emission and number counting tasks in perceptual-auditory differentiation among children with and without laryngeal lesions. Observational, analytical, and cross-sectional methods were used. Medical records of 44 children were selected from a database of an otorhinolaryngology service at a University Hospital and they were divided into groups: without laryngeal lesion (WOLL), and with laryngeal lesion (WLL), with 33 and 11 children. For the auditory-perceptual evaluation, the vocal samples were separated according to the type of task. They were analyzed separately by a judge who analyzed the general degree of vocal deviation and assessed whether the child would pass or fail in the face of a screening situation. There was a difference between the WOLL and WLL groups in terms of the overall degree of vocal deviation for the task of number counting, with a predominance of mild deviations in WOLL and moderate in WLL. In the screening, there was a difference between the groups during the number counting task, with more failures in the WLL. The groups were similar in the sustained vowel task, both in terms of the overall degree of vocal deviation and the vocal screening. Most children in the WLL failed in both tasks during vocal screening compared to the children in the WOLL who, in general, failed in only one task. The task of number counting contributes to the auditory differentiation in children with and without laryngeal lesion, by identifying deviations of greater intensity in children with laryngeal lesion.

Proposição de requisitos para o desenvolvimento de um simulador de treinamento para julgamento perceptivo-auditivo da voz

ABSTRACT Purpose to identify a set of requirements for the development of an auditory-perceptual training simulator (APT) based on the experience of professors who provide APT. Methods This is a cross-sectional, descriptive study with a quantitative approach. Twenty-two professors answered an online questionnaire containing 31 items related to APT, involving items about the professional profile, conditions for APT in undergraduate and postgraduate courses in Speech Therapy, APT structure, and evaluation of the APT effect. Result it was observed that there is a variation in APT procedures performed in Brazil. The main requirements indicated by the respondents for the APT involve the use of synthesized voices in the initial moments, followed by human voices later; the use of speech tasks with sustained vowels and connected speech; the insertion of complementary information such as gender, age, the profession of the speaker and the spectrography of the vocal signal; training with a minimum time of six hours; the evaluation of the training effect by comparing intra- and inter-judge agreement before and after training; the addition of the parameters of general degree of vocal deviation, roughness, breathiness, and strain; the use of validated continuous and numerical scales; and offering it from the second year of the undergraduate program. Conclusion although there is variability in the response of experts, a minimum set of requirements indicated for performing APT with new judges was identified.

Effects of Instruction on the Use of Discourse Markers in Japanese EFL Learners’ Speech

This study investigated the effects of explicit and implicit discourse marker (DM) instruction on English as a foreign language (EFL) learners’ speech. Two different teaching methods were used over seven weeks of online English classes at a university in Japan. Participants were split into two groups, and in the first week, a pre-test comprising two speech tasks was administered to both groups. The groups were then given explicit or implicit feedback about the usage of target DMs using video clips and emails. The results of the post-test in Week 5 revealed that, although the frequency of the marker for example was significantly higher after explicit instruction, there was no significant difference in the frequency of the other target markers in both groups. However, some students’ repertoires of DMs increased after explicit feedback. In Week 7, a speaking test including tasks similar to those in the pre- and post-tests was administered as a substitute for the delayed test. Although some easily accessible markers such as and and because were used in speech, both groups’ repertoire was limited. Hence, explicit instruction on DMs may have a limited effect on EFL learners’ speech.

Responsiveness of the Absolute Error of Estimated Speaking Fundamental Frequency (AEF0) after Surgical Intervention of Voice Disorders

Objectives: The purpose of this study was to investigate the responsiveness of the absolute error of estimated speaking fundamental frequency (AEF0) after intervention of voice disorders. The correlation between the changes in AEF0 measures and multidimensional voice measurements were explored. Usefulness of the changes in the AEF0 (ΔAEF0) measures for predicting vocal improvement was also explored. Methods: 30 male and 30 female patients with benign vocal fold lesions participated in the study. Participants were asked to perform voice and speech range profile and passage-reading tasks to yield the AEF0. Pre- vs. post-operative AEF0 in voice and speech range profile tasks were compared. The correlation between the ΔAEF0 and changes in the voice parameters was explored. The cutoff score of the ΔAEF0 for the improved group was calculated. Results: For both male and female patients, voice and speech range profile measures did not show significant change after surgery. On the other hand, AEF0 measured by voice and speech range profiles decreased regardless of gender. ΔAEF0 after surgery showed weak to moderate correlation with the changes in the multidimensional voice measurements. The area under the curve of the ΔAEF0VRP, ΔAEF0SRP, ΔAEF0SUM for screening vocal improvement were .693, .669, and .753; and the cutoff scores were ≤ 4.035, ≤ -2.403, ≤ 0.389, respectively. Conclusion: The results revealed that AEF0 measures could function as an indicator of post-treatment vocal improvement reflecting vocal ranges.

Comprehensibility improvements in integrated pronunciation instruction

Abstract Integration of pronunciation into content courses is appealing because of its potential in helping learners apply their developing pronunciation skills in spontaneous speech. However, the effectiveness of pronunciation instruction (PI) when it is integrated still needs to be demonstrated. This study evaluates whether a group receiving integrated PI (targeting suprasegmentals) improved in controlled and spontaneous speech tasks, compared to a group who did not receive any specific PI. We measured improvements as comprehensibility ratings and proportion of perceived word stress and vowel reduction errors. The results show that integration appears to be beneficial overall, including in spontaneous speech tasks when comprehensibility is measured. We interpret these findings considering the different tasks used and examine the contribution of the specific instructional approach (form-focused with communicative contextualization) to the observed changes between pre- and post-test.