Speech Corpus Research Articles

Query-by-Example Spoken Term Detection for Zero-Resource Languages Using Heuristic Search

Query-by-Example spoken content retrieval is a demanding and challenging task when a large volume of spoken content is piled up in the repositories without annotation. In the absence of annotation, spoken content retrieval is achieved by capturing the similarities between the query and spoken terms from the acoustic feature representation itself. Dynamic Time Warping (DTW) centric techniques identify the optimal alignment between the acoustic feature representations and capture the similarities between query and spoken terms. Despite feasibility, the DTW-centric techniques produce a lot of false alarms due to the variabilities that exist in natural speech and degrade the performance. In the proposed approach, the variability challenges are addressed in two stages. At first, the speaker-independent acoustic feature representation was obtained from the deep convolutional neural networks that reduce the speaker variabilities. In the second stage, the similarities between the query and spoken term were captured using the heuristic search method. The proposed approach was compared with other state-of-the-art methods using Microsoft Low-Resource Language speech corpus. A 3% improvement and 32% reduction in the hit and false alarm ratio were achieved across languages.

Collections of Livvic dialectal materials in the Phonogram archive of the Institute of Linguistics, Literature and History of the Karelian Research Centre of the Russian Academy of Sciences

Introduction. The article presents the overview of the Livvic dialectal audio materials stored in the Phonogram Archive of the Institute of Linguistics, Literature and History of the Karelian Research Centre of the Russian Academy of Science (Petrozavodsk). Materials and Methods. The article is based on the digital information resources, folklore-ethnographic and linguistic data obtained in the ILLH KarRC RAS archives. The research was carried out using descriptive, comparative, historical and chronolohical methods. Results and Discussion. The Institute’s Phonogram Archive stores field materials which have been collected by the Institute’s specialists since the 1930s. These are speech samples, folklore works, as well as information about the culture and life of the peoples living in the Republic of Karelia, and neighboring regions. The article considers collections of audio materials in the Livvik dialect of the Karelian language, which were collected on the territory of South Karelia in the Olonetsian, Pryazha’s, Suoyarvi’s, Pitkyaranta’s districts. They include not only speech samples, but also some data on ethnography, toponymy, folklore, etc. Part of the material from the Livvic collection (dialect and folklore-ethnographic materials) is presented in the article as the tables that help navigate it. It can be useful for a wide range of specialists: ethnographers, linguists, folklorists, historians. Conclusion. The collections of Livvic dialect and folklore-ethnographic materials of the Phonogram Archive of the ILLH of the Karelian Research Centre of the Russian Academy of Sciences include valuable linguistic, folklore and ethnographic material. The authors have presented a small part of the unique material stored in the archive. Digitization of archival and field audio samples of Karelian speech in the format of a speech corpus can further simplify the processing and storage of materials, can make it possible to introduce into academic and research environment and make available the unique audio materials which reflect the state of Karelian and Vepsian dialects since the middle of the last century.

Creaky Voice in Chilean Spanish: A Tool for Organizing Discourse and Invoking Alignment

This study relies on an interactional, conversational–analytic approach to elucidate what meanings Chilean Spanish speakers convey via creaky voice quality in informal conversations. Highly creaky utterances produced by 18 speakers were derived from a larger corpus of sociolinguistic interview speech from Santiago, Chile, and examined via an interactional approach that accounted for how creaky voice figured in the process of meaning-making and meaning negotiations throughout the conversation. Results indicate that approximately 40% of highly creaky utterances were used to organize the speaker’s discourse, signaling the end of turns, hedges or uncertainty, and a change in communicative purpose, while the majority of the highly creaky utterances were used to invoke alignment with the listener via ensuring that their messages or stances were understood and potentially endorsed. This study offers evidence from a non-English language for creaky voice as a tool for both discursive organization and interactional alignment.

Using known words to learn more words: A distributional model of child vocabulary acquisition

Why do children learn some words before others? A large body of behavioral research has identified properties of the language environment that facilitate word learning, emphasizing the importance of particularly informative language contexts that build on children’s prior knowledge. However, these findings have not informed research that uses distributional properties of words to predict vocabulary composition. In the current work, we introduce a predictor of word learning that emphasizes the role of prior knowledge. We investigate item-based variability in vocabulary development using lexical properties of distributional statistics derived from a large corpus of child-directed speech. Unlike previous analyses, we predicted word trajectories cross-sectionally across child age, shedding light on trends in vocabulary development that may not have been evident at a single time point. We also show that regardless of a word’s grammatical class, the best distributional predictor of whether a child knows a word is the number of other known words with which that word tends to co-occur.

Code-switching input for machine translation: a case study of Vietnamese–English data

ABSTRACT Multilingualism presents both a challenge and an opportunity for Natural Language Processing, with code-switching representing a particularly interesting problem for computational models trained on monolingual datasets. In this paper, we explore how code-switched data affects the task of Machine Translation, a task which only recently has started to tackle the challenge of multilingual data. We test three Machine Translation systems on data from the Canberra Vietnamese–English Codeswitching Natural Speech Corpus (CanVEC) and evaluate translation output using both automatic and human metrics. We find that, perhaps counter-intuitively, Machine Translation performs better on code-switching input than monolingual input. In particular, comparison of human and automatic evaluation suggests that codeswitching input may boost the semantic faithfulness of the translation output, an effect we term lexico-semantic enrichment. We also report two cases where this effect is most and least clear in Vietnamese–English, namely gender-neutral 3SG pronouns and interrogative constructions respectively. Overall, we suggest that Machine Translation, and Natural Language Processing more generally, ought to view multilingualism as an opportunity rather than an obstacle. Abbreviations: 1: First person; 2: Second person; 3: Third person; CLF: Classifier; COP: Copula; DET: Determiner; PL: Plural; POSS: Possessive marker; PRT: Particle; PST: Past tense; Q: Question marker; SG: Singular

Kencorpus: A Kenyan Language Corpus of Swahili, Dholuo and Luhya for Natural Language Processing Tasks

Indigenous African languages are categorized as under-served in Natural Language Processing. They therefore experience poor digital inclusivity and information access. The processing challenge with such languages has been how to use machine learning and deep learning models without the requisite data. The Kencorpus project intends to bridge this gap by collecting and storing text and speech data that is good enough for data-driven solutions in applications such as machine translation, question answering and transcription in multilingual communities. The Kencorpus dataset is a text and speech corpus for three languages predominantly spoken in Kenya: Swahili, Dholuo and Luhya (three dialects of Lumarachi, Lulogooli and Lubukusu). Data collection was done by researchers who were deployed to the various data collection sources such as communities, schools, media, and publishers. The Kencorpus' dataset has a collection of 5,594 items, being 4,442 texts of 5.6 million words and 1,152 speech files worth 177 hours. Based on this data, other datasets were also developed such as Part of Speech tagging sets for Dholuo and the Luhya dialects of 50,000 and 93,000 words tagged respectively. We developed 7,537 Question-Answer pairs from 1,445 Swahili texts and also created a text translation set of 13,400 sentences from Dholuo and Luhya into Swahili. The datasets are useful for downstream machine learning tasks such as model training and translation. Additionally, we developed two proof of concept systems: for Kiswahili speech-to-text and a machine learning system for Question Answering task. These proofs provided results of a performance of 18.87% word error rate for the former, and 80% Exact Match (EM) for the latter system. These initial results give great promise to the usability of Kencorpus to the machine learning community. Kencorpus is one of few public domain corpora for these three low resource languages and forms a basis of learning and sharing experiences for similar works especially for low resource languages. Challenges in developing the corpus included deficiencies in the data sources, data cleaning challenges, relatively short project timelines and the Coronavirus disease (COVID-19) pandemic that restricted movement and hence the ability to get the data in a timely manner.

The speech signal enhancement approach with multiple sub-frames analysis for complex magnitude and phase spectrum recompense

The intended speech must be dealt with in the process of speech communication while under the impact of noise experienced in a variety of situations that degrade speech intelligibility and quality. This work proposes a multiple sub-frames analysis for the elimination of noise variants with compensation of the magnitude and phase spectrum from speech degraded by noise. The clean speech samples are extracted from the ITU-T recommended dataset at a 16 kHz sampling rate and down-sampled to an 8 kHz sampling rate. The noise signal variants are added from the AURORA and NOISEX-92 datasets at diverse input SNR levels (0 dB, 5 dB, 10 dB, 15 dB). The duration of window frames is chosen to be 25 msec in length, together with a shift percentage of 40%, to maintain the continuous nature of frames in speech. The smoothing factor for noise updating in a specific sub-frame is set to 9, and the spectral floor parameter for determining the precise amount of noise elimination in the corrupted spectrum is set to 0.03. The phase spectrum is compensated by incorporating a recompense function that is updated in combination with the sub-frame analysis. The accomplishment of the suggested approach is assessed with regard to objective metrics, including Segmental-Signal-to-Noise-Ratio (SegSNR), Mean-Square-Error (MSE), and Perceptual-Evaluation-of-Speech-Quality (PESQ) scores corresponding to specific sub-frames of speech, respectively. To further analyze the improved quality, simple listening assessment and spectrogram analysis are incorporated, followed by a comparative investigation with prior noise-suppressive algorithms on the corrupted speech corpus.

The role of the input frequency in L1 Spanish phonological acquisition. A corpus-based study

This study presents the phonological system exhibited by children (n=59) aged 3;0 to 6;0 and focuses on the role of input frequency. Using a spontaneous child speech corpus of Spanish (CHIEDE) as a data source, as well as computational processing techniques -including an automatic phonological transcriber-, data relating to the phonological level was retrieved. This resulted in a phonological inventory of Spanish-speaking children, ordered by frequency of use, which may serve as a model for research on typical and atypical child language development. Additionally, a study was carried out on the stability of the participants’ phonological systems by calculating the variability that the different age groups displayed, and outcomes were compared with other similar corpora. Results obtained from the comparison of the phonological inventory of children and adults show that there is a relationship between frequency of use in adult speech and the order of acquisition of phonemes.

Form and Function of Connectives in Chinese Conversational Speech

Connectives convey discourse functions that provide textual and pragmatic information in speech communication on top of canonical, sentential use. This paper proposes an applicable scheme with illustrative examples for distinguishing Sentential, Conclusion, Disfluency, Elaboration, and Resumption uses of Mandarin connectives, including conjunctions and adverbs. Quantitative results of our annotation works are presented to gain an overview of connectives in a Mandarin conversational speech corpus. A fine-grained taxonomy is also discussed, but it requires more empirical data to approve the applicability. By conducting a multinomial logistic regression model, we illustrate that connectives exhibit consistent patterns in positional, phonetic, and contextual features oriented to the associated discourse functions. Our results confirm that the position of Conclusion and Resumption connectives orient more to positions in semantically, rather than prosodically, determined units. We also found that connectives used for all four discourse functions tend to have a higher initial F0 value than those of sentential use. Resumption and Disfluency uses are expected to have the largest increase in initial F0 value, followed by Conclusion and Elaboration uses. Durational cues of the preceding context enable distinguishing Sentential use from discourse uses of Conclusion, Elaboration, and Resumption of connectives.

Os princípios da Escola Fonológica de São Petersburgo para a elaboração de corpora de fala

ABSTRACT The paper discusses the main principles in designing and annotating speech corpora within the framework of the Saint Petersburg phonological school, and provides examples of using corpus data in phonetic research. One of the major principles that we follow is to analyse the speech material at all levels: from segmental to intonational, including speech disfluencies. During segmental phonetic annotation, we suggest listening to each speech sound in isolation (without knowing its context) and relying on spectrographic data. At the syllabic tier, it is crucial to reflect resyllabification. During prosodic annotation, we suggest to rely on listener’s perception of the intonation pattern first, then analyse the actual melodic curves. A speech corpus with multi-level annotation that follows these principles is a valuable source of phonetic data — as segmental and prosodic factors are in constant interaction with each other, and one cannot analyse units of one annotation tier without reference to other tiers.

An approach for Correcting the Word-level Mispronunciations for non-native English-speaking Indian Children

Speech Recognition and its potential applications in terms of “talking devices” have become indispensable in today’s world. Technological advances like mobiles, smart home assistants or tablets extensively use the techniques of automatic speech recognition that works good for adults but cannot always follow and understand children’s speech. The primary goal of this paper is to bridge the gap of communication between voice assistants and Indian children speaking English as secondary language. The issue of lack of children’s speech corpora with English as non-native language, is addressed by creating a dataset of children in the age group of 5-15 years, speaking Hindi or Marathi as their mother tongue and English as their second language. The analysis and implementation of the proposed work shows the accuracy of approximately 96% and potential for further scope by increasing the size of dataset in lower age group. The key contributions of our work are (i) creating speech dataset of Indian children whose mother-tongue is Hindi or Marathi, (ii) employing and evaluating hybrid Convolutional Neural Network (CNN) as an age classifier, (iii) language modeling to customize children vocabulary, (iv) checking accuracy and performance of the system.

The structure of acoustic voice variation in bilingual speech.

When a bilingual switches languages, do they switch their voice? Using a conversational corpus of speech from early Cantonese-English bilinguals (n = 34), this paper examines the talker-specific acoustic signatures of bilingual voices. Following the psychoacoustic model of voice, 24 filter and source-based acoustic measurements are estimated. The analysis summarizes mean differences for these dimensions and identifies the underlying structure of each talker's voice across languages with principal component analyses. Canonical redundancy analyses demonstrate that while talkers vary in the degree to which they have the same voice across languages, all talkers show strong similarity with themselves, suggesting an individual's voice remains relatively constant across languages. Voice variability is sensitive to sample size, and we establish the required sample to settle on a consistent impression of one's voice. These results have implications for human and machine voice recognition for bilinguals and monolinguals and speak to the substance of voice prototypes.

Compressive speech enhancement using semi-soft thresholding and improved threshold estimation

<span lang="EN-US">Compressive speech enhancement is based on the compressive sensing (CS) sampling theory and utilizes the sparsity of the signal for its enhancement. To improve the performance of the discrete wavelet transform (DWT) basis-function based compressive speech enhancement algorithm, this study presents a semi-soft thresholding approach suggesting improved threshold estimation and threshold rescaling parameters. The semi-soft thresholding approach utilizes two thresholds, one threshold value is an improved universal threshold and the other is calculated based on the initial-silence-region of the signal. This study suggests that thresholding should be applied to both detail coefficients and approximation coefficients to remove noise effectively. The performances of the hard, soft, garrote and semi-soft thresholding approaches are compared based on objective quality and speech intelligibility measures. The normalized covariance measure is introduced as an effective intelligibility measure as it has a strong correlation with the intelligibility of the speech signal. A visual inspection of the output signal is used to verify the results. Experiments were conducted on the noisy speech corpus (NOIZEUS) speech database. The experimental results indicate that the proposed method of semi-soft thresholding using improved threshold estimation provides better enhancement compared to the other thresholding approaches.</span>

Научное наследие профессора В. Е. Гольдина: о книге избранных работ ученого

The review is dedicated to the book of selected works by Professor V. E. Goldin, Linguistic Consciousness. Speech Communication (Saratov, Saratov State University Publ., 2020), prepared for publication by O. Kryuchkova and A. Sdobnova. The scientific interests of V. E. Goldin were extremely broad. The list of his scientific works includes studies on the Russian dialectology, problems of the general theory of language, speech communication, speech genres, the problems of psycholinguistics and applied linguistics. This variety of focuses of the scientific attention is also reflected in the book of his selected works. The book includes works by V. E. Goldin, related to the three most important areas of his scientific interests: the associative-verbal network of a native speaker, problems of speech communication, communicative and corpus dialectology. The composition of the book expresses the idea of linguistic consciousness as a deep cognitive basis of verbal communication and, accordingly, of verbal communication as a verbal embodiment of linguistic consciousness. Such an understanding of the relationship between linguistic consciousness and speech communication stems from the complexity of the scientific research by V. E. Goldin, and is the result of summing up the scientific heritage of the scientist. The publication outlines the chronology of the development of V. E. Goldin’s scientific interests – from individual spheres of speech communication to its holistic understanding and further to linguistic consciousness. Such a development of the researcher’s scientific work allowed him to gradually build up and enlarge theoretical generalizations, build them on a solid foundation of comprehensive specific empirical material. The review presents V. E. Goldin’s most significant scientific ideas: the principles of communicative dialectology; the construction, scientific and cultural significance of dialect corpora, the need for a holistic study of the Russian speech in the sum of all its social and functional varieties; the criteria for an effective cultural and language policy; problems of studying the semantics of communication; tasks of genre studies; the method of conducting associative experiments, associative dictionaries, the heuristic potential of verbal associations and associative fields.

Identity Retention and Emotion Converted StarGAN for low-resource emotional speaker recognition

In speaker recognition (SR) task, the inconsistency of emotion between enrolled speech and test speech will lead to emotion domain mismatch, which decreases the recognition accuracy and is more serious under low-resource training data condition. To help SR model learn the various emotion distribution when only neutral speech is enrolled, Identity Retention and Emotion Converted StarGAN (IREC-StarGAN) is proposed to generate speech feature in other emotions. Emotion and speaker relevant multi-task loss is designed to retain origin speaker information during adversarial training progress for the generation of emotion feature. To avoid essential speaker or emotion information being contaminated by irrelevant regions, generator uses SC-bottleneck with multi kernel size channels to learn the deep feature of data in various scales. Discriminator adopts densenet with two branches, one of which classifies whether input feature belongs to target emotion domain and the another one focuses on the speaker information. The origin neutral feature will be enrolled with generated feature together to be recognized by Resnet18 for speaker recognition task. The proposed framework obtains the EER of 18.33% on Mandarin Affective Speech Corpus and 8.14% on IEMOCAP, which outperform the existing embedding and GAN methods and reveals the robustness of IREC-StarGAN under low-resource emotion condition.

SEC-GAN for robust speaker recognition with emotional state dismatch

Speaker recognition is often dependent on the speaker and susceptible to emotional factors, thus mostly decreasing recognition performance, we propose a framework by combining generative adversarial networks and speaker recognition for generating additional speaker-related emotional training speech feature to enhance robustness under different emotional conditions. In this framework, a new speaker emotion-converted generative adversarial network (SEC-GAN) is developed for speaker recognition. Given the neutral speech of the target speaker, SEC-GAN learns speech information to generate speech feature in other emotions based on neutral speech while retaining speaker identity. In addition, a new loss function is designed to retain the speaker’s internal information during feature reconstruction, and an emotion discriminator is introduced to classify the speech feature’s emotion for better emotion generation quality. Based on the origin neutral and generated training data from native speakers of Mandarin Affective Speech Corpus (MASC), the negative impact of emotion mismatch between speech can be decreased by using our framework. This strategy could solve the common problem in reality that most voice control devices enroll user’s calm speech but fail to recognize user’s identity when they are in other emotion. The experimental results on MASC which is 57.59% show the improvement of 8.27% compared with VGG baseline and 5.62% compared with x-vector in accuracy. Our framework also outperforms the existing state-of-the-art method ECAPA-TDNN and other comparison methods.

Gender variation across the Oromo dialects: A corpus‐based study*

AbstractThis study aims to (1) demonstrate the position of the Oromo gender system in Corbett's (1991) typology of gender; (2) illustrate major syntactic gender variation across the Oromo dialects; (3) identify factors that contributed to the gender variation, and (4) illustrate the morphosyntax of the Oromo gender system. The data obtained from the Oromo Speech Corpus shows a high degree of lexical and syntactic variation between the Western‐Northern dialects on the one hand, and the Eastern‐Southern dialects on the other hand. The Western and Northern dialects have shifted from the historically Cushitic phonology‐based gender assignment pattern to the semantic‐based assignment pattern. This shift has resulted in a widespread neutralization of feminine gender markers. The contact between Oromo and the neighboring non‐Cushitic languages contributed to these changes. The study also argues that Kramer's (2015) morphosyntactic approach can be extended to the analysis of Oromo gender system.

A dynamic neural field model of leaky prosody: proof of concept

Recent work has shown that lexical items come to take on the phonetic characteristics of the prosodic environments in which they are typically produced, a phenomenon referred to as "leaky prosody". Focusing on pitch patterns in Mandarin, we show that leaky prosody can be derived from a flat (i.e., non-transformational, non-optimizing) model of speech production. Formalized using Dynamic Field Theory, in our model, lexical, phonological, and prosodic inputs each exert forces on a Dynamic Neural Field representing pitch. Notably, the forces exerted by these inputs reflect surface distributions in a large corpus of spontaneous speech. Our simulations showed that the flat model derives the short timescale effect of prosodic prominence on pitch production as well as the longer timescale effect of leaky prosody. By updating lexical items based on surface phonetic form, words that are consistently produced in high/low prosodic prominence positions take on the phonetic characteristics of those environments.

Effective preservation of higher-frequency contents in the context of short utterance based children’s speaker verification system

Developing an automatic speaker verification (ASV) system for children is extremely challenging due to the unavailability of children’s speech corpora. The challenges are further exacerbated in the case of short utterances. Voice-based biometric systems require adequate amount of speech data for enrollment and verification; otherwise the performance considerably degrades. In this paper, we have focussed on data paucity and preserving the higher-frequency contents in order to enhance the performance of a short-utterance based children’s speaker verification system. To deal with data scarcity, several out-of-domain data augmentation techniques have been utilized. Since the out-of-domain data used is from adult speakers which are acoustically very different from children’s speech, we have resorted to techniques like prosody modification, formant modification and voice-conversion in order to render it acoustically similar to children’s speech prior to augmentation. This helps in not only increasing the amount of training data but also in effectively capturing the missing target attributes. A relative improvement of 33.6% in equal error rate (EER) with respect to the baseline system trained solely on child data-set is achieved when the proposed data augmentation technique is applied. Further to that, for the preservation of the higher-frequency contents, we have resorted to concatenation of the classical Mel-frequency cepstral coefficients (MFCC) features with the linear-frequency cepstral coefficient (LFCC) or with the inverse-Mel-frequency cepstral coefficient (IMFCC) features. The use of Mel-filter-bank leads to poor resolution of higher-frequency components. On the other hand, linear- or inverse-Mel-filter-banks yield better resolution of higher-frequency components. Moreover, MFCC and IMFCC features exhibit low canonical correlation. Consequently, the frame-level concatenation of MFCC and LFCC or IMFCC features leads to better resolution of both lower- as well as higher-frequency components. Therefore, the EER considerably reduces when either LFCC features or IMFCC features are concatenated with MFCC features. The EER for the full test set shows a relative reduction of 10.56% (with respect to the EER for the MFCC features) when IMFCC features are concatenated with the MFCC features. This novel approach of incorporating data augmentation followed by frame-level feature concatenation helps in achieving an overall reduction of 40.6% in EER.

Filled Pauses Produced by Autistic Adults Differ in Prosodic Realisation, but not Rate or Lexical Type

We examined the use of filled pauses in conversations between homogeneous pairs of autistic and non-autistic adults. A corpus of semi-spontaneous speech was used to analyse the rate, lexical type (nasal “uhm” or non-nasal “uh”), and prosodic realisation (rising, level or falling) of filled pauses. We used Bayesian modelling for statistical analysis. We found an identical rate of filled pauses and an equivalent preference of “uhm” over “uh” across groups, but also a robust group-level difference regarding the intonational realisation of filled pauses: non-autistic controls produced a considerably higher proportion of filled pause tokens realised with the canonical level pitch contour than autistic speakers. Despite the fact that filled pauses are a frequent and impactful part of speech, previous work on their conversational use in autism spectrum disorder (ASD) is limited. Our account is the first to analyse the intonational realisation of filled pauses in ASD and the first to investigate conversations between autistic adults in this context. Our results on rate and lexical type can help to contextualise previous research, while the novel findings on intonational realisation set the stage for future investigations.