Speech Corpus Research Articles

The FRA Corpus from the DISease and AGEing Project

The FRA corpus, from the DISAGE project (University of Naples Federico II), is an Italian speech corpus of elderly people with and without neurodegenerative pathologies. The project's main aim is to provide an Italian tool useful to observe language in pathological aging. Therefore, the project enrolled 20 patients diagnosed with Alzheimer's Disease (AD), 20 individuals suffering from Mild Cognitive Impairment (MCI), and 20 Healthy Controls (HC) balanced with respect to sex, age, and education. Participants underwent neuro-psychological testing (MMSE; Raven's Matrices) and only mild AD and amnestic MCI were recruited. The speech corpus is semi-spontaneous and dialogic: speech samples were collected via a three-stage interview designed to encourage the production of descriptive, conversational and narrative speech. Currently, 18 out of the 60 recordings collected have been manually and orthographically transcribed. Data is available on request.

Determinante e doppio determinante nel code-switching arabo-italiano

This article explores the phenomenon of code-switching, specifically focusing on bilingual Arabic-Italian determiner phrases which consist of a determiner, such as an article or a demonstrative, followed by a noun phrase as a complement. The data examined were selected from a corpus of Arabic-Italian spontaneous speech with the aim of (1) comparing and describing the characteristics and differences of the determiner systems of the two languages; (2) illustrating the features of code-switching in determiner phrases and the various types found in the collected data; and (3) studying the mutual influence of the two languages in contact contexts. The data analysis revealed two types of code-switching: switching at the beginning of a determiner phrase, where both the determiner and its complement are in the same language, and switching within a determiner phrase, where the determiner and the head noun are from different languages. This latter category also includes determiner phrases with double determiners, which will be analyzed in detail in this article to discuss the features of this linguistic structure in Arabic-Italian bilingual phrases.

What’s so complex about conversational speech? A comparison of HMM-based and transformer-based ASR architectures

Highly performing speech recognition is important for more fluent human-machine interaction (e.g., dialogue systems). Modern ASR architectures achieve human-level recognition performance on read speech but still perform sub-par on conversational speech, which arguably is or, at least, will be instrumental for human-machine interaction. Understanding the factors behind this shortcoming of modern ASR systems may suggest directions for improving them. In this work, we compare the performances of HMM- vs. transformer-based ASR architectures on a corpus of Austrian German conversational speech. Specifically, we investigate how strongly utterance length, prosody, pronunciation, and utterance complexity as measured by perplexity affect different ASR architectures. Among other findings, we observe that single-word utterances – which are characteristic of conversational speech and constitute roughly 30% of the corpus – are recognized more accurately if their F0 contour is flat; for longer utterances, the effects of the F0 contour tend to be weaker. We further find that zero-shot systems require longer utterance lengths and are less robust to pronunciation variation, which indicates that pronunciation lexicons and fine-tuning on the respective corpus are essential ingredients for the successful recognition of conversational speech.

Community-Supported Shared Infrastructure in Support of Speech Accessibility.

The Speech Accessibility Project (SAP) intends to facilitate research and development in automatic speech recognition (ASR) and other machine learning tasks for people with speech disabilities. The purpose of this article is to introduce this project as a resource for researchers, including baseline analysis of the first released data package. The project aims to facilitate ASR research by collecting, curating, and distributing transcribed U.S. English speech from people with speech and/or language disabilities. Participants record speech from their place of residence by connecting their personal computer, cell phone, and assistive devices, if needed, to the SAP web portal. All samples are manually transcribed, and 30 per participant are annotated using differential diagnostic pattern dimensions. For purposes of ASR experiments, the participants have been randomly assigned to a training set, a development set for controlled testing of a trained ASR, and a test set to evaluate ASR error rate. The SAP 2023-10-05 Data Package contains the speech of 211 people with dysarthria as a correlate of Parkinson's disease, and the associated test set contains 42 additional speakers. A baseline ASR, with a word error rate of 3.4% for typical speakers, transcribes test speech with a word error rate of 36.3%. Fine-tuning reduces the word error rate to 23.7%. Preliminary findings suggest that a large corpus of dysarthric and dysphonic speech has the potential to significantly improve speech technology for people with disabilities. By providing these data to researchers, the SAP intends to significantly accelerate research into accessible speech technology. https://doi.org/10.23641/asha.27078079.

Frequency and Universal Principles in the Acquisition of Word-Initial Consonant Clusters in Palestinian Arabic.

In this article, we present the results of our study on the acquisition of word-initial consonant clusters in Palestinian Arabic. The remarkable property of word-initial clusters in Palestinian Arabic is that they often violate the Sonority Sequencing Principle, a universal constraint that disfavors sonority fall (e.g., lba:n "gum") and, to a lesser degree, sonority plateau (e.g., kta:b "book"). Given the violation of the Sonority Sequencing Principle, the acquisition of word-initial consonant clusters serves as a fruitful ground for studying the interaction of universal principles and language-specific effects. The goal of the study was to shed light on the role of universal principles in language acquisition and how they interact with frequency. We carried out picture naming and sentence completion tasks with 60 monolingual and monodialectal children aged 2;00-5;00 (years;months), equally divided into six age groups. In addition, we built a small corpus of child-directed speech (CDS) to evaluate the distribution of the three sonority profiles of word-initial consonant clusters-rise, plateau, and fall. We found a major language-specific effect in the cluster simplification strategies, with prothesis being the most common strategy (e.g., lbá:n → ilbá:n "gum"). However, this effect was not found in the youngest group (2;00-2;05), where the cross-linguistically favorite strategy of C-deletion was most common (e.g., lbá:n → bá:n "gum"). We also found a major effect of the Sonority Sequencing Principle in the children's productions, contrary to insignificant differences in CDS. Our study supports the claim that language acquisition is affected by both language-specific effects and universal principles. As for the interaction of these factors, we show that the effect of universal principles emerges under two conditions: (a) in early speech (youngest age group), before children accumulate sufficient data that allow language-specific properties to override, and/or (b) when frequency does not play a role.

Comparing neural network architectures for non-intrusive speech quality prediction

Non-intrusive speech quality predictors evaluate speech quality without the use of a reference signal, making them useful in many practical applications. Recently, neural networks have shown the best performance for this task. Two such models in the literature are the convolutional neural network based DNSMOS and the bi-directional long short-term memory based Quality-Net, which were originally trained to predict subjective targets and intrusive PESQ scores, respectively. In this paper, these two architectures are trained on a single dataset, and used to predict the intrusive ViSQOL score. The evaluation is done on a number of test sets with a variety of mismatch conditions, including unseen speech and noise corpora, and common voice over IP distortions. The experiments show that the models achieve similar predictive ability on the training distribution, and overall good generalization to new noise and speech corpora. Unseen distortions are identified as an area where both models generalize poorly, especially DNSMOS. Our results also suggest that a pervasiveness of ambient noise in the training set can cause problems when generalizing to certain types of noise. Finally, we detail how the ViSQOL score can have undesirable dependencies on the reference pressure level and the voice activity level.

Building a Speech Dataset and Recognition Model for the Minority Tu Language

Speech recognition technology has many applications in our daily life. However, for many low-resource languages without written forms, acquiring sufficient training data remains a significant challenge for building accurate ASR models. The Tu language, spoken by an ethnic minority group in Qinghai Province in China, is one such example. Due to the lack of written records and the great diversity in regional pronunciations, there has been little previous research on Tu-language speech recognition. This work seeks to address this research gap by creating the first speech dataset for the Tu language spoken in Huzhu County, Qinghai. We first formulated the relevant pronunciation rules for the Tu language based on linguistic analysis. Then, we constructed a new speech corpus, named HZ-TuDs, through targeted data collection and annotation. Based on the HZ-TuDs dataset, we designed several baseline sequence-to-sequence deep neural models for end-to-end Tu-language speech recognition. Additionally, we proposed a novel SA-conformer model, which combines convolutional and channel attention modules to better extract speech features. Experiments showed that our proposed SA-conformer model can significantly reduce the character error rate from 23% to 12%, effectively improving the accuracy of Tu language recognition compared to previous approaches. This demonstrates the effectiveness of our dataset construction and model design efforts in advancing speech recognition technology for this low-resource minority language.

PaSCoNT - Parallel Speech Corpus of Northern-central Thai for automatic speech recognition

This paper proposed a Parallel Speech Corpus of Northern-central Thai (PaSCoNT). The purpose of this research is not only to understand the different linguistic characteristics between Northern and Central Thai, but also to utilize this corpus for automatic speech recognition. The corpus is composed of speech data from dialogues of daily life among northern Thai people. We designed 2,000 Northern Thai sentences covering all phonemes, in collaboration with linguists specialized in the Northern Thai dialect. The samples in this study are 200 Northern Thai dialect speakers who had been living in Chiang Mai province for more than 18 years. The speech was recorded in both open and closed environments. In the speech recording, each speaker must read 100 pairs of Northern-Central Thai sentences to ensure that the speech data comes from the same speaker. In total, 100 h of speech were recorded: 50 h of Northern Thai and 50 h of Central Thai. Overall, PaSCoNT consists of 907,832 words and 6,279 vocabulary items. Statistical analysis of the PaSCoNT corpus revealed that 49.64 % of words in the lexicon belongs to the Northern Thai dialect, 50.36 % from the Central Thai dialect, and 1,621 vocabulary items appeared in both Northern and Central Thai. Statistical analysis is used to examine the difference in speech tempo, i.e. time per phoneme (TTP), syllable per minute (SPM), between Northern and Central Thai. The results revealed that there were statistically significant differences speech tempo between Central and Northern Thai. The TTP speaking and articulation rate of Central Thai is lower than Northern Thai whereas SPM speaking and articulation rate of Central Thai is higher than Northern Thai. The results also showed that the ASR model training using Northern Thai speech corpus provides the lower WER% when testing using Northern Thai testing speech data and provides the higher WER% when testing using Central Thai Testing speech data and vice versa. However, the ASR model training using the PaSCoNT speech corpus provides the lower WER% for both Northern Thai and Central Thai testing speech data.

Using Transfer Learning to Realize Low Resource Dungan Language Speech Synthesis

This article presents a transfer-learning-based method to improve the synthesized speech quality of the low-resource Dungan language. This improvement is accomplished by fine-tuning a pre-trained Mandarin acoustic model to a Dungan language acoustic model using a limited Dungan corpus within the Tacotron2+WaveRNN framework. Our method begins with developing a transformer-based Dungan text analyzer capable of generating unit sequences with embedded prosodic information from Dungan sentences. These unit sequences, along with the speech features, provide <unit sequence with prosodic labels, Mel spectrograms> pairs as the input of Tacotron2 to train the acoustic model. Concurrently, we pre-trained a Tacotron2-based Mandarin acoustic model using a large-scale Mandarin corpus. The model is then fine-tuned with a small-scale Dungan speech corpus to derive a Dungan acoustic model that autonomously learns the alignment and mapping of the units to the spectrograms. The resulting spectrograms are converted into waveforms via the WaveRNN vocoder, facilitating the synthesis of high-quality Mandarin or Dungan speech. Both subjective and objective experiments suggest that the proposed transfer learning-based Dungan speech synthesis achieves superior scores compared to models trained only with the Dungan corpus and other methods. Consequently, our method offers a strategy to achieve speech synthesis for low-resource languages by adding prosodic information and leveraging a similar, high-resource language corpus through transfer learning.

A voice and speech corpus of patients who underwent upper airway surgery in pre- and post-operative states

Many research articles have explored the impact of surgical interventions on voice and speech evaluations, but advances are limited by the lack of publicly accessible datasets. To address this, a comprehensive corpus of 107 Spanish Castilian speakers was recorded, including control speakers and patients who underwent upper airway surgeries such as Tonsillectomy, Functional Endoscopic Sinus Surgery, and Septoplasty. The dataset contains 3,800 audio files, averaging 35.51 ± 5.91 recordings per patient. This resource enables systematic investigation of the effects of upper respiratory tract surgery on voice and speech. Previous studies using this corpus have shown no relevant changes in key acoustic parameters for sustained vowel phonation, consistent with initial hypotheses. However, the analysis of speech recordings, particularly nasalised segments, remains open for further research. Additionally, this dataset facilitates the study of the impact of upper airway surgery on speaker recognition and identification methods, and testing of anti-spoofing methodologies for improved robustness.

Pitch changes as the marker of a phrase edge in standard Lithuanian

In many languages, pitch changes at phrase edges are often described as significant cues of juncture. The aim of this study is to analyse pitch changes at the beginning and end of declarative phrases in standard Lithuanian. The study consists of two parts: 1) a pilot study analysing whether the F0 difference is a reliable measure to determine the F0 rise/fall at the end of declarative phrasesand whether such a difference depends on the speaker’s sex; 2) an annotated speech corpus analysis on the tendencies of pitch changes at phrase edges. The pilot study showed that in phrases with a rising F0 at the end, the F0 mean of the last stressed syllable is significantly lower than F0 at the end of a phrase. In phrases with a falling F0, the F0 mean of the last stressed syllable is significantly higher than the F0 at the end of a phrase. The F0 difference is a reliable measure to determine the F0 rise or fall at the end of a phrase, and all the abovementioned tendencies were found in both male and female speakers’ recordings. The annotated speech corpus analysis revealed that both intonational and intermediate phrases usually begin and end in a low tone. Hence, the high and rising or falling tones at the beginning or end of a phrase are the marked members of the phrase edge tone system, and they can help identify phrase junctures. However, it is important to pay attention to other acoustic features such as pauses and intensity, as well as grammatical and semantic aspects.

Using deep learning to improve the intelligibility of a target speaker in noisy multi-talker environments for people with normal hearing and hearing loss.

Understanding speech in noisy environments is a challenging task, especially in communication situations with several competing speakers. Despite their ongoing improvement, assistive listening devices and speech processing approaches still do not perform well enough in noisy multi-talker environments, as they may fail to restore the intelligibility of a speaker of interest among competing sound sources. In this study, a quasi-causal deep learning algorithm was developed that can extract the voice of a target speaker, as indicated by a short enrollment utterance, from a mixture of multiple concurrent speakers in background noise. Objective evaluation with computational metrics demonstrated that the speaker-informed algorithm successfully extracts the target speaker from noisy multi-talker mixtures. This was achieved using a single algorithm that generalized to unseen speakers, different numbers of speakers and relative speaker levels, and different speech corpora. Double-blind sentence recognition tests on mixtures of one, two, and three speakers in restaurant noise were conducted with listeners with normal hearing and listeners with hearing loss. Results indicated significant intelligibility improvements with the speaker-informed algorithm of 17% and 31% for people without and with hearing loss, respectively. In conclusion, it was demonstrated that deep learning-based speaker extraction can enhance speech intelligibility in noisy multi-talker environments where uninformed speech enhancement methods fail.

Perception of Voice Attractiveness: Effects of Speaking Rate, Gender, and Age

Objectives: The purpose of this study was to investigate the effects of the speaker’s speech rate level, listener’s age, and gender on the listener’s attractiveness evaluation of the speaker’s voice. Methods: A total of 8 speaker voices (4 males and 4 females in their 20s-30s) were selected from the speech corpus provided by the National Institute of Korean Language. These voices were manipulated to be 0.8 times and 1.2 times their original speed, resulting in 24 voice files for attractiveness evaluation. Evaluators were recruited from different age groups: adolescents (14), young adults (16), and middle-aged adults (14). They were instructed to perceptually evaluate the attractiveness of the speaker’s voice. In addition, speech data from evaluators were collected to measure their actual speech rates. Results: Evaluator age groups did not show significant differences in attractiveness scores. However, across all groups, the attractiveness ratings were consistently higher for normal speech rates. Middle-aged male evaluators reported a significant decrease in attractiveness for faster speech rates. The correlation between the actual speech rate of evaluators and speakers was significant only for slower speech rates, indicating that evaluators with faster speech rates, especially males, showed lower attractiveness. Conclusion: The listener’s attractiveness evaluation may depend on the listener’s gender and age according to the speaker’s speaking rate, with middle-aged male listeners being particularly inclined to find fast speech rates unacceptable. This study underscores the significance of speech rate as an acoustic evaluation factor influencing listener attractiveness.

A Small Brazilian Portuguese Speech Corpus for Speaker Recognition Study

A small Brazilian speech corpus was created for educational purposes to study a state-of-the-art speaker recognition system. The system uses the Gaussian Mixture Model (GMM) as a statistical model for speakers and employs the Mel-frequency cepstral coefficients (MFCC) as acoustic features. The results using clean and noisy speech are compatible with the expected results, showing that the bigger the mismatch between training and test conditions, the worse the results. The results also improve with the increase in the utterance length. Finally, the obtained results can be used as baselines to compare with other speaker statistical models created with different acoustic features in different acoustic conditions.

Ga-RBP: A Rule-based Parser for the Syntactic Analysis of the Ga Language of Ghana

This research presents a Parts of Speech (POS) corpus and a static rule-based technique, which we refer to as Ga Rule-based Parser (Ga-RBP), for the syntactic analysis and the parsing of sentences for the Ga language of Ghana. The technique is developed to parse sentences by utilising the POS tagged corpus; the corpus was developed by manually tagging the Ga words with their corresponding POS tags following a standard Ga-English dictionary and custom Tagset for the language. The syntax rules were computationally defined using production rules, which establish how a word should follow the other in the right sequence to form a correct grammatical statement based on their POS. The model generally analyses the sentence structure of the language to assert its syntactic state for correctness or otherwise.

Recurrence plot embeddings as short segment nonlinear features for multimodal speaker identification using air, bone and throat microphones

Speech is produced by a nonlinear, dynamical Vocal Tract (VT) system, and is transmitted through multiple (air, bone and skin conduction) modes, as captured by the air, bone and throat microphones respectively. Speaker specific characteristics that capture this nonlinearity are rarely used as stand-alone features for speaker modeling, and at best have been used in tandem with well known linear spectral features to produce tangible results. This paper proposes Recurrent Plot (RP) embeddings as stand-alone, non-linear speaker-discriminating features. Two datasets, the continuous multimodal TIMIT speech corpus and the consonant-vowel unimodal syllable dataset, are used in this study for conducting closed-set speaker identification experiments. Experiments with unimodal speaker recognition systems show that RP embeddings capture the nonlinear dynamics of the VT system which are unique to every speaker, in all the modes of speech. The Air (A), Bone (B) and Throat (T) microphone systems, trained purely on RP embeddings perform with an accuracy of 95.81%, 98.18% and 99.74%, respectively. Experiments using the joint feature space of combined RP embeddings for bimodal (A–T, A–B, B–T) and trimodal (A–B–T) systems show that the best trimodal system (99.84% accuracy) performs on par with trimodal systems using spectrogram (99.45%) and MFCC (99.98%). The 98.84% performance of the B–T bimodal system shows the efficacy of a speaker recognition system based entirely on alternate (bone and throat) speech, in the absence of the standard (air) speech. The results underscore the significance of the RP embedding, as a nonlinear feature representation of the dynamical VT system that can act independently for speaker recognition. It is envisaged that speech recognition too will benefit from this nonlinear feature.

The Impact of Three Social Factors on the Variation of “Qaf” in Ammani Arabic

This study aims to explore the effects of three social factors on the variation of (Qaf) in Ammani Arabic. In specific, the social factors investigated were gender, age, and level of education. In order to elicit a corpus of vernacular speech, thirty sociolinguistic interviews with 30 speakers who were all born and raised in Amman were conducted. Using computer software GOLDVARB X (Sankoff et al., 2005), it was found that all social factors were statistically significant. Furthermore, gender was considered the most significant factor while education appeared to be the least significant factor. The results of the study support previous research that females frequently use [?], presenting their feminine identity. In regard of age, old participants favored the use of [g] rather than [?] while [?] was most frequently used by young speakers. Moreover, education was also found as a significant social factor. In this respect, low-educated preferred [g] and highly-educated speakers preferred [q]. In regard to education, the researcher followed Abdel-Jawad’s (1987) proposal that education in Arabic is a genuine variable which directly affects language variation in Arabic.

Developing corpus literacy: A perspective of Latgalian language and cultural studies

ABSTRACT Corpus literacy has been described as the ability to use the technology of corpus linguistics to investigate language and enhance the language development of students. In recent decades, text and speech corpora for regional and lesser-used languages have increasingly been created, and they are considered an important resource of authentic texts for language and culture studies and learning. The Modern Latgalian Text Corpus (MuLa 2022) and the Modern Latgalian Speech Corpus (MuLaR) are, in this sense, available for researchers, learners and teachers of Latgalian. Researchers, teachers, and students as potential discoverers of language and culture often lack knowledge, skills and practice to work with language corpora. The purpose of this article is therefore to discuss the opportunities of using Latgalian language corpora in language and culture education. In this vein, we call for the development of corpus literacy of researchers and teachers as part of their professional development. In support of this call, we offer suggestions on how to facilitate students’ cultural awareness and foster their proficiency in the Latgalian language through language corpora. These recommendations aim to enrich both the research and teaching of language (socio-onomastics in a cultural context) and culture (folklore concept) with the help of corpus tools.

Emotion recognition for human–computer interaction using high-level descriptors

Recent research has focused extensively on employing Deep Learning (DL) techniques, particularly Convolutional Neural Networks (CNN), for Speech Emotion Recognition (SER). This study addresses the burgeoning interest in leveraging DL for SER, specifically focusing on Punjabi language speakers. The paper presents a novel approach to constructing and preprocessing a labeled speech corpus using diverse social media sources. By utilizing spectrograms as the primary feature representation, the proposed algorithm effectively learns discriminative patterns for emotion recognition. The method is evaluated on a custom dataset derived from various Punjabi media sources, including films and web series. Results demonstrate that the proposed approach achieves an accuracy of 69%, surpassing traditional methods like decision trees, Naïve Bayes, and random forests, which achieved accuracies of 49%, 52%, and 61% respectively. Thus, the proposed method improves accuracy in recognizing emotions from Punjabi speech signals.

StereoHate: Toward identifying stereotypical bias and target group in hate speech detection

Abstract Though social media helps spread knowledge more effectively, it also stimulates the propagation of online abuse and harassment, including hate speech. It is crucial to prevent hate speech since it may have serious adverse effects on both society and individuals. Therefore, it is not only important for models to detect these speeches but to also output explanations of why a given text is toxic. While plenty of research is going on to detect online hate speech in English, there is very little research on low-resource languages like Hindi and the explainability aspect of hate speech. Recent laws like the “right to explanations” of the General Data Protection Regulation have spurred research in developing interpretable models rather than only focusing on performance. Motivated by this, we create the first interpretable benchmark hate speech corpus hate speech explanation (HHES) in the Hindi language, where each hate post has its stereotypical bias and target group category. Providing descriptions of internal stereotypical bias as an explanation of hate posts makes a hate speech detection model more trustworthy. Current work proposes a commonsense-aware unified generative framework, CGenEx, by reframing the multitask problem as a text-to-text generation task. The novelty of this framework is it can solve two different categories of tasks (generation and classification) simultaneously. We establish the efficacy of our proposed model (CGenEx-fuse) on various evaluation metrics over other baselines when applied to the Hindi HHES dataset. Disclaimer The article contains profanity, an inevitable situation for the nature of the work involved. These in no way reflect the opinion of authors.