Speech Corpus Research Articles

Effective preservation of higher-frequency contents in the context of short utterance based children’s speaker verification system

Developing an automatic speaker verification (ASV) system for children is extremely challenging due to the unavailability of children’s speech corpora. The challenges are further exacerbated in the case of short utterances. Voice-based biometric systems require adequate amount of speech data for enrollment and verification; otherwise the performance considerably degrades. In this paper, we have focussed on data paucity and preserving the higher-frequency contents in order to enhance the performance of a short-utterance based children’s speaker verification system. To deal with data scarcity, several out-of-domain data augmentation techniques have been utilized. Since the out-of-domain data used is from adult speakers which are acoustically very different from children’s speech, we have resorted to techniques like prosody modification, formant modification and voice-conversion in order to render it acoustically similar to children’s speech prior to augmentation. This helps in not only increasing the amount of training data but also in effectively capturing the missing target attributes. A relative improvement of 33.6% in equal error rate (EER) with respect to the baseline system trained solely on child data-set is achieved when the proposed data augmentation technique is applied. Further to that, for the preservation of the higher-frequency contents, we have resorted to concatenation of the classical Mel-frequency cepstral coefficients (MFCC) features with the linear-frequency cepstral coefficient (LFCC) or with the inverse-Mel-frequency cepstral coefficient (IMFCC) features. The use of Mel-filter-bank leads to poor resolution of higher-frequency components. On the other hand, linear- or inverse-Mel-filter-banks yield better resolution of higher-frequency components. Moreover, MFCC and IMFCC features exhibit low canonical correlation. Consequently, the frame-level concatenation of MFCC and LFCC or IMFCC features leads to better resolution of both lower- as well as higher-frequency components. Therefore, the EER considerably reduces when either LFCC features or IMFCC features are concatenated with MFCC features. The EER for the full test set shows a relative reduction of 10.56% (with respect to the EER for the MFCC features) when IMFCC features are concatenated with the MFCC features. This novel approach of incorporating data augmentation followed by frame-level feature concatenation helps in achieving an overall reduction of 40.6% in EER.

Filled Pauses Produced by Autistic Adults Differ in Prosodic Realisation, but not Rate or Lexical Type

We examined the use of filled pauses in conversations between homogeneous pairs of autistic and non-autistic adults. A corpus of semi-spontaneous speech was used to analyse the rate, lexical type (nasal “uhm” or non-nasal “uh”), and prosodic realisation (rising, level or falling) of filled pauses. We used Bayesian modelling for statistical analysis. We found an identical rate of filled pauses and an equivalent preference of “uhm” over “uh” across groups, but also a robust group-level difference regarding the intonational realisation of filled pauses: non-autistic controls produced a considerably higher proportion of filled pause tokens realised with the canonical level pitch contour than autistic speakers. Despite the fact that filled pauses are a frequent and impactful part of speech, previous work on their conversational use in autism spectrum disorder (ASD) is limited. Our account is the first to analyse the intonational realisation of filled pauses in ASD and the first to investigate conversations between autistic adults in this context. Our results on rate and lexical type can help to contextualise previous research, while the novel findings on intonational realisation set the stage for future investigations.

AVbook, a high-frame-rate corpus of narrative audiovisual speech for investigating multimodal speech perception.

Seeing a speaker's face can help substantially with understanding their speech, particularly in challenging listening conditions. Research into the neurobiological mechanisms behind audiovisual integration has recently begun to employ continuous natural speech. However, these efforts are impeded by a lack of high-quality audiovisual recordings of a speaker narrating a longer text. Here, we seek to close this gap by developing AVbook, an audiovisual speech corpus designed for cognitive neuroscience studies and audiovisual speech recognition. The corpus consists of 3.6 h of audiovisual recordings of two speakers, one male and one female, each reading 59 passages from a narrative English text. The recordings were acquired at a high frame rate of 119.88 frames/s. The corpus includes phone-level alignment files and a set of multiple-choice questions to test attention to the different passages. We verified the efficacy of these questions in a pilot study. A short written summary is also provided for each recording. To enable audiovisual synchronization when presenting the stimuli, four videos of an electronic clapperboard were recorded with the corpus. The corpus is publicly available to support research into the neurobiology of audiovisual speech processing as well as the development of computer algorithms for audiovisual speech recognition.

Detecting somatisation disorder via speech: introducing the Shenzhen Somatisation Speech Corpus

ObjectiveSpeech recognition technology is widely used as a mature technical approach in many fields. In the study of depression recognition, speech signals are commonly used due to their convenience and ease of acquisition. Though speech recognition is popular in the research field of depression recognition, it has been little studied in somatisation disorder recognition. The reason for this is the lack of a publicly accessible database of relevant speech and benchmark studies. To this end, we introduced our somatisation disorder speech database and gave benchmark results. MethodsBy collecting speech samples of somatisation disorder patients, in cooperation with the Shenzhen University General Hospital, we introduced our somatisation disorder speech database, the Shenzhen Somatisation Speech Corpus (SSSC). Moreover, a benchmark for SSSC using classic acoustic features and a machine learning model was proposed in our work. ResultsTo obtain a more scientific benchmark, we compared and analysed the performance of different acoustic features, i. e., the full ComPare feature set, or only Mel frequency cepstral coefficients (MFCCs), fundamental frequency (F0), and frequency and bandwidth of the formants (F1-F3). By comparison, the best result of our benchmark was the 76.0% unweighted average recall achieved by a support vector machine with formants F1–F3. ConclusionThe proposal of SSSC may bridge a research gap in somatisation disorder, providing researchers with a publicly accessible speech database. In addition, the results of the benchmark could show the scientific validity and feasibility of computer audition for speech recognition in somatization disorders.

A socio-pragmatic analysis of the Turkish discourse markers of ‘şey’, ‘yani’, and ‘işte’ based on educational level of speakers

Abstract This paper analyzes the influence of the educational level of speakers on speakers’ preferences of the Turkish discourse markers (DMs) şey ‘uh’, yani ‘I mean’, and işte ‘you know’. 56 participants from age groups 33–50 and over 50 participated in the study. Speech data from each participant in two speech corpora (planned vs. spontaneous) were gathered via face-to-face interviews. Although various trends have been observed in two speech conditions in terms of the educational level of the speakers in the current data, the frequency rates of these markers were mostly higher in the speech of (especially male) participants with a lower educational level, compared to those of participants with a higher educational level. Educational-level related differences were also observed in further analyses of the functions of the three DMs under focus.

SEHC: A Benchmark Setup to Identify Online Hate Speech in English

Thanks to the digital age, online speech and information may now be disseminated anonymously without regard for repercussions. Regulators face a unique problem with social media platforms because of the speed and volume of material and the lack of editorial supervision. The existing datasets on hate speech or offensive language identification lack diversity in the dataset’s content. In this article, we create a multi-domain hate speech corpus (MHC) of English tweets that includes hate speech against religion, nationality, ethnicity, and gender in general and cover diverse domains, such as current affairs, politics, terrorism, technology, natural disasters, and human/drugs trafficking. Each instance in our dataset is manually annotated as hate or non-hate. We use the existing state-of-the-art models and present a stacked-ensemble-based hate speech classifier (SEHC) to identify hate speech from Twitter data. Our results indicate that the proposed method may serve as a strong baseline for future studies using this dataset. (The dataset is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://www.iitp.ac.in/~ai-nlp-ml/resources.html#MHC</uri> .)

An approach to constructing prosodic grammar for Mandarin read speech.

A data-driven approach to constructing a prosodic grammar of Mandarin read speech is proposed. Prosodic labeling is performed, first, on a large speech corpus with syntactic-tree parsing to add four-level break indices. Two types of prosodic grammatical rules are explored. One type is composed of simplified rules to compute break-type distributions at critical junctures for 5 phrase-level and 11 basic syntactic patterns. The other type entails detailed rules to compute break-type distributions conditioned on syntactic function for four determinative-measure (DM)-related syntactic patterns. Effectiveness of the approach was confirmed by meaningful interpretations of the resulting main prosodic patterns and outliers of targeted syntactic patterns by inferred rules. The main findings are given below. Strong paused breaks are found at VE-clause object (VE, active verb with a sentential object) junctures and junctures after idioms. For DM-related patterns, the entropies of break-type distributions decrease significantly as syntactic functions are involved; break-type distributions on both edges are seriously affected by their syntactic functions; when acting as subject (S) and object (O), their prosodic phenomena support the tendency of Mandarin to be S(VO) (V, verb); strong paused breaks at postboundaries of DM-2-DM-4 are caused by their more complex syntactic structures and greater lengths; and the insertions of modifier + DE (special tag for the word DE) into DM-N (N, noun) junctures cause more paused-break insertions at junctures after DMs.

Prosodic Studies on the Spoken Corpus of the Khalkha Mongolian Language: Age and Gender Effects on F0 and Speech Rate

There are many studies on age- and gender-dependent analyses in multiple languages. Still, the fundamental frequency (F0) and speech rate in different age and gender groups with Khalkha Mongolian have not yet been studied. In this investigation, we analyzed the speaker’s age and gender effects on F0 and speech rate based on a spontaneous speech corpus of the Khalkha dialect of the Mongolian language. We have discovered some universal aspects of these phonological phenomena and specific relevant characteristics of speech rate change depending on gender. The average speech rate of Khalkha Mongolian is 5.3 syllables per second. The male speakers spoke much quicker than the female speakers. Age and gender affect the F0 and speech rate in particular systematic ways. The F0 of the female and male speakers tended to decrease with age. Also, that elderly male speakers talk slower than elderly female speakers, while young and middle-aged male speakers tend to talk faster than female speakers of the same age.

Distributional and Acoustic Characteristics of Filler Particles in German with Consideration of Forensic-Phonetic Aspects

In this study, we investigate the use of the filler particles (FPs) uh, um, hm, as well as glottal FPs and tongue clicks of 100 male native German speakers in a corpus of spontaneous speech. For this purpose, the frequency distribution, FP duration, duration of pauses surrounding FPs, voice quality of FPs, and their vowel quality are investigated in two conditions, namely, normal speech and Lombard speech. Speaker-specific patterns are investigated on the basis of twelve sample speakers. Our results show that tongue clicks and glottal FPs are as common as typically described FPs, and should be a part of disfluency research. Moreover, the frequency of uh, um, and hm decreases in the Lombard condition while the opposite is found for tongue clicks. Furthermore, along with the usual F1 increase, a considerable reduction in vowel space is found in the Lombard condition for the vowels in uh and um. A high degree of within- and between-speaker variation is found on the individual speaker level.

Grammar-Supervised End-to-End Speech Recognition with Part-of-Speech Tagging and Dependency Parsing

For most automatic speech recognition systems, many unacceptable hypothesis errors still make the recognition results absurd and difficult to understand. In this paper, we introduce the grammar information to improve the performance of the grammatical deviation distance and increase the readability of the hypothesis. The reinforcement of word embedding with grammar embedding is presented to intensify the grammar expression. An auxiliary text-to-grammar task is provided to improve the performance of the recognition results with the downstream task evaluation. Furthermore, the multiple evaluation methodology of grammar is used to explore an expandable usage paradigm with grammar knowledge. Experiments on the small open-source Mandarin speech corpus AISHELL-1 and large private-source Mandarin speech corpus TRANS-M tasks show that our method can perform very well with no additional data. Our method achieves relative character error rate reductions of 3.2% and 5.0%, a relative grammatical deviation distance reduction of 4.7% and 5.9% on AISHELL-1 and TRANS-M tasks, respectively. Moreover, the grammar-based mean opinion score of our method is about 4.29 and 3.20, significantly superior to the baseline of 4.11 and 3.02.

Analysis of the of training and test data distribution for audio series classification

The effectiveness of machine learning algorithms for any given task largely depends on the training and test datasets. This manifests itself not only in the amount of data, but also in its content (that is, its relevance for the task at hand), as well as in its organization. Generally, the common approach is to split the dataset into training and testing sets to avoid model overfitting. In addition, to achieve better metrics for the selected criteria (accuracy, learning rate, etc.) of model performance, different ratios of training and test sets are used in the partitioning. The goal of this paper is to analyze methods of data set partitioning for use in training neural networks and statistical models. One of the reviewed methods, specifically the cross-validation method, was applied to a dataset developed from the LibriSpeach corpus, an open English speech corpus based on the LirbiVox project of voluntarily contributed audio books. The result of applying the selected data partitioning method on the selected data set is demonstrated.

Automatic evaluation of spontaneous oral cancer speech using ratings from naive listeners

In this paper, we build and compare multiple speech systems for the automatic evaluation of the severity of a speech impairment due to oral cancer, based on spontaneous speech. To be able to build and evaluate such systems, we collected a new spontaneous oral cancer speech corpus from YouTube consisting of 124 utterances rated by 100 non-expert listeners and one trained speech-language pathologist, which we made publicly available. We evaluated the systems in two scenarios: a scenario where transcriptions were available (reference-based) and a scenario where transcriptions might not be available (reference-free).The results of extensive experiments showed that (1) when transcriptions were available, the highest correlation with the human severity ratings was obtained using an automatic speech recognition (ASR) retrained with oral cancer speech. (2) When transcriptions were not available, the best results were achieved by a LASSO model using modulation spectrum features. (3) We found that naive listeners’ ratings are highly similar to the speech pathologist’s ratings for speech severity evaluation. (4) The use of binary labels led to lower correlations of the automatic methods with the human ratings than using severity scores.

Particle stacking in Singlish – New data from the National Speech Corpus

Particle stacking, a phenomenon that sees more than one sentence-final particle used to mark an utterance, occurs in various languages, including Singlish, also commonly known as Colloquial Singapore English (CSE) or Singapore Colloquial English (SCE). In this paper, we provide novel empirical data on particle stacking in Singlish using the National Speech Corpus (NSC). A frequency analysis of particle stacks containing the Singlish particles hor, lah and meh in the NSC shows that most of the particle stacks are declarative-interrogative pairs where the first particle is used in assertions while the second particle has a backchannelling function, which is consistent with particles stacks observed in other languages. We also identify two particle stacks in Singlish – lah dey and lah sia – where the second particle does not overtly solicit the addressee’s response. We propose that dey and sia, together with the backchannelling particles, form a larger class of addressee-oriented particles based on their shared function of targeting the addressee’s attention.

A Helium Speech Unscrambling Algorithm Based on Deep Learning

Helium speech, the language spoken by divers in the deep sea who breathe a high-pressure helium–oxygen mixture, is almost unintelligible. To accurately unscramble helium speech, a neural network based on deep learning is proposed. First, an isolated helium speech corpus and a continuous helium speech corpus in a normal atmosphere are constructed, and an algorithm to automatically generate label files is proposed. Then, a convolution neural network (CNN), connectionist temporal classification (CTC) and a transformer are combined into a speech recognition network. Finally, an optimization algorithm is proposed to improve the recognition of continuous helium speech, which combines depth-wise separable convolution (DSC), a gated linear unit (GLU) and a feedforward neural network (FNN). The experimental results show that the accuracy of the algorithm, upon combining the CNN, CTC and the transformer, is 91.38%, and the optimization algorithm improves the accuracy of continuous helium speech recognition by 9.26%.

The voice as a material clue: a new forensic Algerian Corpus.

Dialects have received bigger interest in recent years as they are increasingly used on the web and social media. Because Algerian Arabic dialects suffer from a lack of appropriate speech corpora for speech recognition, a rich dialect corpus is needed to approach Algerian Accent recognition. The latter remains a key feature in the field of Forensic Voice Comparison (FVC) systems. This paper presents a new large-scale forensic Algerian speech corpus called Sawt El-Djazaïr. An important criterion in dealing with forensic corpora is the presence of session variability. For this purpose, we collected celebrity recordings in various regions of Algeria, from different social networks, in various scenarios, and at different times. In addition, we also recorded 87 participants using cellular calls and voice over IP (VoIP) applications including Viber, WhatsApp, and Google Meet. The corpus of approximately 50hours covers various speech topics and is spoken in twelve Algerian sub-dialects. The design guidelines of the proposed corpus are described along with the grouping of dialects across different geographical locations. Sawt El-Djazaïr is available to the research community upon request.

Factors predicting human performance in error annotation for non-native speech corpus

With low inter-annotator agreement and large time consumption, phonetic annotation of non-native speech is complicated. However, having a large-scale non-native speech corpus with error annotation is important for second language phonology and computer-assisted pronunciation training research. To achieve large-scale annotation, we must clarify the causes of difficulty in error annotation. This study identified the factors that can predict annotation performance of non-native Mandarin speech by considering two perspectives.1) We used a decomposed method (that involved dividing a traditional error annotation task into two different subtasks, namely the error location task and error description task) to determine whether human performance, from the cognitive viewpoint, can be predicted using the decomposed annotation method (Experiment 1). 2) We determined whether the features of annotation task materials (e.g., error severity) can be employed to predict annotation performance (Experiment 2). The results of Experiment 1 revealed that the decomposed method leads to a shorter annotation time and a higher consistent hit rate than does the traditional method. The regressions of Experiment 2 revealed that the features of error severity and sentence length accounted for 96% of the hit rate and the features of sentence length and number of errors accounted for 95.5% of the annotation time. These results suggest that the performance of manual annotation can be predicted through both cognitive factors and material features.

Появата на класове думи в ранната онтогенеза на българския език. Пилотно корпусно изследване

This article discusses early Bulgarian language ontogenesis. The pilot corpus study focuses on the extraction, analysis and systematization of data representative of the earliest evidence of the emergence of word classes in the process of children’s language acquisition. While the discussion and conclusions follow the paradigm of the Theory of natural morphology, a crucial prerequisite for their appropriate validation is provided by the empirically-based CHILDES Bulgarian LabLing Corpus (https://childes.talkbank. org/access/Slavic/Bulgarian/LabLing.html), the first Bulgarian child speech corpus developed as part of the Slavic collection of the interactive plat-form CHILDES.

Extended high frequencies for fricative classification in conversational speech

The current study examines whether the information contained in Extended High Frequencies (EHFs) can improve random forest classification accuracy for fricatives in conversational speech. Prior phonetic research has investigated fricative categorization based on their acoustic characteristics, including spectral, temporal and amplitudinal measures. The spectral measures in these studies have largely been limited to frequency information below 8 kHZ. Only a few studies have examined the contribution of EHFs, energy exceeding 8 kHz. These studies have primarily focused on laboratory speech, so little is currently known about the role of EHFs in more spontaneous, conversational speech styles. Using a corpus of sociolinguistic interview speech from Western Canadian English sampled at 44.1kHz, we compare classification models with and without frequencies above 8 kHz. We discuss the influence of EHFs on the categorization of fricative identity, and share a cost-benefit analysis of sampling at higher frequencies for speech research corpora.

Neural networks’ posterior probability as measure of effects of alcohol on speech

Intoxication has a well-known effect on speech production. Lester and Skousen (1974) reported that the place of articulation for /s/ is retracted and /tʃ/ and /ʤ/ are deaffricated (i.e., substituted by a non-affricate segment) in drunken speech. Zihlmann (2017) further established the robustness of deaffrication as it cannot be consciously suppressed under intoxication. Using these prevalent speech errors as test cases, this study extends a phonologically-informed neural network approach to the study of intoxicated speech. The approach has success in measuring pathological speech and lenition patterns in healthy speakers. Degrees of place retraction for /s/ and deaffrication of /tʃ/ and /ʤ/ are estimated from posterior probabilities calculated by recurrent neural networks trained to recognize [anterior], [continuant] and [strident] features. When applied to a corpus of alcohol English speech, preliminary results suggested that sober versus drunken state could be reliably predicted by the three posterior probabilities. The directions of the effects are largely in line with previous studies. For example, /tʃ/ and /ʤ/ are more fricated (higher strident and continuant probabilities), and /s/ is more retracted (lower anterior probability) in drunken compared to sober speech. The results suggest that the intoxicated speech can be reliably quantified by this new approach.

Spectral degradations in the TIMIT, QuickSIN, NU-6, and other popular bandlimited speech materials

Many popular speech corpora were recorded decades ago using transducers, recording techniques, or post-processing techniques that resulted in spectral degradations. The use of such materials very likely has consequences for measures of speech perception, but this is not often considered in studies that use these materials. We analyzed several of these popular corpora to ascertain spectral degradations that may be present. Spectral degradations we frequently observed included limited extended high-frequency (EHF) (&amp;gt;8 kHz) content, and, for a few corpora, substantial low-frequency roll-off &amp;lt;500 Hz. Two corpora analyzed here, the TIMIT/PRESTO and NU-6, exhibited both high- and low-frequency degradations. Such spectral degradations will likely have an effect on behavioral measures because they do not match the high-fidelity speech signals listeners encounter in their everyday lives. This should be considered when selecting stimuli for speech perception experiments. [Work supported by NIH Grant R01-DC019745]