Corpus Of Read Speech Research Articles

Performance analysis of various fundamental frequency estimation algorithms in the context of pathological speech.

Reliable fundamental frequency (f0) extraction algorithms are crucial in many fields of speech research. The current bulk of studies testing the robustness of different algorithms have focused on healthy speech and/or measurements of sustained vowels. Few studies have tested f0 estimations in the context of pathological speech, and even fewer on continuous speech. The present study evaluated 12 available pitch detection algorithms on a corpus of read speech by 24 speakers (8 healthy speakers, 8 speakers with Parkinson's disease, and 8 with head and neck cancer). Two fusion methods' algorithms have been tested: one based on the median of algorithms and one based on the fusion between the best algorithm for voicing detection and the algorithm that generates the most accurate f0 estimations on voiced parts. Our results show that time-domain algorithms, like REAPER, are best for voicing detection while deep neural network algorithms, like FCN- f0, yield better accuracy for the f0 values on voiced parts. The combination of REAPER and FCN- f0 yields the best ratio performance/implementation complexity, since it generates less than 4% errors on voicing detection and less than 5% of gross errors in the estimation of the f0 values for all speaker groups.

Effects of noise, native language, age, and speaker gender on intelligibility in a large corpus of read speech

Because speech communication takes place under noisy conditions, investigating the effect of noise on intelligibility is an essential part of understanding speech perception. One challenge is having sufficient numbers of speakers and unique sentences to control for sentence and speaker effects. In an online experiment, we used a corpus of 720 IEEE sentences read by 20 native English speakers (10 male, 10 female) from the Pacific Northwest (WA, OR, ID), which were embedded in three corpus-shaped noise conditions (−2, 0, + 2 dB). Stimuli were presented to undergraduate students at the Universities of Alberta and Washington using a design matrix which ensured that no listener heard a sentence more than once. Participants entered responses into a form which appeared immediately after the stimulus completed playing. Data collection is ongoing, but the current number of listeners is 1269, who responded to 120 items each. We use Levenshtein and Jaro distance measures to compare listener transcription accuracy. We investigate the effects of listener native language, age, and gender. We also investigate effects of the speaker’s gender on transcription accuracy. [Work supported by NIH NIDCD R01 DC006014.]

Acquiring French Intonation against the Backdrop of Heritage Bilingualism: The Case of German–Turkish Learners

This paper investigates the intonation of L3 French, produced by six bilingual learners (ages: 15–17) who speak Turkish as a heritage language (HL) along with German and six same-aged monolingual German learners. We examined of a corpus of read speech in two respects: first, we determined the number of accentual phrases (APs) and, second, we extracted F0 values for each segment, normalized them, and calculated the deviations from the average values produced by three native controls (age: 21–23). Although the bilinguals were expected to outperform the monolinguals due to certain similarities between the intonational systems of French and Turkish, their mean deviation from the native contours was only slightly smaller than that of the monolinguals (difference not significant). To determine how strongly the bilinguals’ Turkish intonation was influenced by German and whether it could serve as a basis for positive transfer whatsoever, we compared their production in Turkish with data recorded from three monolingual Turkish L1 speakers (ages: 21–32) and five German learners of L2 Turkish (ages: 22–43). Results show that the bilinguals’ Turkish intonation does not deviate substantially from the monolinguals’ one, in contrast to the contours produced by the L2 learners. This suggests that metalinguistic and prosodic awareness should be fostered in heritage bilinguals to make them benefit from their full linguistic repertoire.

The link between syllabic nasals and glottal stops in American English

Examples of syllabic nasals in English abound in phonological research (e.g., Hammond, 1999; Harris, 1994; Wells, 1995), but there is little explicit discussion about the surrounding consonant environments that condition syllabic nasals. This study examines the production of potential word-final syllabic nasals in American English following preceding consonants including oral stops, glottal stops, fricatives, flap, and laterals. The data come from a laboratory study of read speech with speakers from New York and other regions, a corpus of read speech with speakers from the Pacific Northwest and Northern Cities, and a spontaneous telephone speech corpus. Acoustic analysis indicates that [n̩] is only prevalent after [ʔ], with some extension to [d] or [ɾ]. Variation in rates of [n̩] versus [ən] is found across the speakers in a group, not within individual speakers. An articulatory sketch to account for the prevalence of [n̩] after coronal and glottal stops is laid out. To link this realization to the presence of the [ʔ] allophone in pre-syllabic nasal environment, previous analyses of acoustic enhancement proposed for glottally-reinforced [tʔ] in coda position (e.g., Keyser & Stevens, 2006) are extended to the syllabic nasal case.

Robust vowel region detection method for multimode speech

The aim of this paper is to explore a robust method for vowel region detection from multimode speech. In realistic scenario, speech can be classified into three modes namely; conversation, extempore, and read. The existing method detects the vowel form the speech recorded in clean environment which may not be appropriate for the multimode speech tasks. To address this issue, we proposed an approach based on continuous wavelet transform coefficients and phone boundaries for detecting the vowel regions from different modes of the speech signal. For evaluation of the proposed vowel region (VR) detection technique, TIMIT (read speech) and Bengali (read, extempore, and conversation speech) corpora are used. The proposed VR detection technique is compared to the state-of-the-art methods. The experiments has recorded significant gain in the performance of the proposed technique than the state-of-the-art methods. The efficiency of the proposed technique is shown by extracting vocal tract and excitation source features from automatically detected VRs for developing the multilingual speech mode classification (MSMC) model. The evaluation results report that the performance of the MSMC model is significantly improved when features are extracted from the vowel regions than the entire speech utterance.

Investigation of Automatic Speech Recognition Systems via the Multilingual Deep Neural Network Modeling Methods for a Very Low-Resource Language, Chaha

Automatic speech recognition (ASR) is vital for very low-resource languages for mitigating the extinction trouble. Chaha is one of the low-resource languages, which suffers from the problem of resource insufficiency and some of its phonological, morphological, and orthographic features challenge the development and initiatives in the area of ASR. By considering these challenges, this study is the first endeavor, which analyzed the characteristics of the language, prepared speech corpus, and developed different ASR systems. A small 3-hour read speech corpus was prepared and transcribed. Different basic and rounded phone unit-based speech recognizers were explored using multilingual deep neural network (DNN) modeling methods. The experimental results demonstrated that all the basic phone and rounded phone unit-based multilingual models outperformed the corresponding unilingual models with the relative performance improvements of 5.47% to 19.87% and 5.74% to 16.77%, respectively. The rounded phone unit-based multilingual models outperformed the equivalent basic phone unit-based models with relative performance improvements of 0.95% to 4.98%. Overall, we discovered that multilingual DNN modeling methods are profoundly effective to develop Chaha speech recognizers. Both the basic and rounded phone acoustic units are convenient to build Chaha ASR system. However, the rounded phone unit-based models are superior in performance and faster in recognition speed over the corresponding basic phone unit-based models. Hence, the rounded phone units are the most suitable acoustic units to develop Chaha ASR systems.

Analyzing rhythmic convergence between two Swiss German dialects

With respect to existing evidence of rhythmic adjustments in response to the interlocutor’s idyosincratic characteristics, in the present study, we test whether interlocutors are likely to mutually adapt their rhythmic characteristics over the course of a conversation or after increased exposure to a dialogue partner. To study rhythmic accommodation, we used a corpus of read speech recorded by 18 speakers of two Swiss German dialects—Grison and Zurich German—before and after performing diapix tasks. The two dialects present crucial differences in the durational characteristics of intervocalic sonorants, vowel in open syllable and vowel in final position, thus creating different rhythm in these two dialects. To determine whether Grison and Zurich German speakers produce the rhythmic contrasts more similarly after the diapix tasks, we measured the Euclidean distance within a pair (ddpair) and within an individual (ddspeak) in three ratio measures devised to capture the durational differences between the two dialects. Preliminary results on ddpair have shown that certain durational contrasts are produced more similarly by certain pairs but more differently by other ones. Ratio measure related to the timing properties of vowels in final position does not change drastically before and after the interaction.

Partial Greedy Algorithm to Extract a Minimum Phonetically-and-Prosodically Rich Sentence Set

A phonetically-and-prosodically rich sentence set is so important in collecting a read-speech corpus for developing phoneme-based speech recognition. The sentence set is usually searched from a huge text corpus of million sentences using the optimization methods. One of the commonly used optimization methods for this case is a Least-to-Most Greedy (LTMG) algo-rithm. It is effective in minimizing the number of phoneme-units. Unfortunately, it does not distribute their frequencies. In this paper, a new method called Partial LTMG algorithm (PLTMG) is proposed to search an optimum set containing triphones and prosodies those are distributed in a near-uniform fashion. Testing on an Indonesian text corpus of ten million sentences crawled from some websites of newspapers and novels shows that the proposed method is not only capable of minimizing both phoneme-units and prosodies but also effective in distributing their frequencies.

Phonation and laryngeal specification in American English voiceless obstruents

This study presents a detailed acoustic analysis of phonation in voiceless obstruents in American English (AE) to investigate the acoustic consequences of the laryngeal timing that has been reported in the literature. The current study examines the appearance of phonation in voiceless obstruents in a corpus of read speech with 37 AE speakers. Linguistic factors such as phrase and word position, stress, and the preceding phoneme are examined and are shown to condition the presence and degree of phonation during the constriction period of stops and fricatives. The amount of phonation present is further analyzed by characterizing where in the constriction interval phonation appears. Carryover phonation (or bleed) from a preceding sonorant is most common for stops, while a trough pattern (phonation that dies out and then begins again before the end of the closure) is more prevalent for fricatives. These acoustic patterns, together with previous reports of laryngeal articulation and air pressure measures, have implications for the representation of laryngeal timing in a gestural phonology framework.

The distribution of the retroflex lateral allophone of the liquid phoneme in Korean

The liquid phoneme in Korean is known to have an optional retroflex allophone in syllable-final positions (Lee 1999). However, there lacks literature evaluating when and how this retroflex allophone is realized. To define the distribution of the retroflex allophone of the liquid phoneme in Korean, this study uses a read speech corpus of Seoul Korean to track the rhotacization of the liquid phoneme in coda positions. Rhotacization of the liquid is detected and measured by the lowering of the third formant. Preliminary results show that the liquid phoneme in coda position is realized as retroflex when preceded by a back vowel, and when followed by another liquid phoneme (gemination). The results suggest that the liquid phoneme is not simply optionally rhotacized in coda positions; instead, the retroflex allophone is realized in contexts defined by the preceding vowel and the following consonant. Future research remains to further examine the articulatory and perceptual aspects of the rhotacization.

A corpus of read and conversational Austrian German

This paper presents GRASS (Graz corpus of Read and Spontaneous Speech), the first large scale speech database for Austrian German with both read and conversational speech. In total, the corpus contains approximately 1900 min of speech in which 38 speakers produced more than 220,000 word tokens from 14,593 different word types. The corpus consists of three components. First, the Conversational Speech Component contains free conversations of one hour length between friends, colleagues, couples, or family members. Second, the Commands Component contains commands and keywords which were either read or elicited by pictures. Third, the Read Speech Component contains phonetically balanced sentences and digits. The speech of all components has been recorded at fullband quality in a soundproof recording-studio with head-mounted microphones, large-diaphragm microphones, a laryngograph, and with a video camera. The corpus was fully annotated at the orthographic level, and partly also at the segmental, sub-segmental and prosodic level. Our analysis of conversational speech characteristics such as overlapping speech, laughter, repetitions, hesitations and the use of colloquial and dialectal words allows us to conclude that the conversational speech material is highly casual in nature. The collected corpus provides conversational material for phoneticians and linguists interested in topics specific for Austrian German (e.g., pronunciation variability, prosody, syntax of spoken Austrian German), and for those studying talk in interaction in general (turn-taking, grounding, entrainment, extra-linguistic factors etc.). Furthermore, it is a valuable resource for speech technologists interested in the development of ASR and dialogue systems for different speaking styles of Austrian German.

Rethinking classification results based on read speech, or: why improvements do not always transfer to other speaking styles

With the growing interest among speech scientists in working with natural conversations also the popularity for using articulatory–acoustic features as basic unit increased. They showed to be more suitable than purely phone-based approaches. Even though the motivation for AF classification is driven by the properties of conversational speech, most of the new methods continue to be developed on read speech corpora (e.g., TIMIT). In this paper, we show in two studies that the improvements obtained on read speech do not always transfer to conversational speech. The first study compares four different variants of acoustic parameters for AF classification of both read and conversational speech using support vector machines. Our experiments show that the proposed set of acoustic parameters substantially improves AF classification for read speech, but only marginally for conversational speech. The second study investigates whether labeling inaccuracies can be compensated for by a data selection approach. Again, although an substantial improvement was found with the data selection approach for read speech, this was not the case for conversational speech. Overall, these results suggest that we cannot continue to develop methods for one speech style and expect that improvements transfer to other styles. Instead, the nature of the application data (here: read vs. conversational) should be taken into account already when defining the basic assumptions of a method (here: segmentation in phones), and not only when applying the method to the application data

Anomaly-based annotation error detection in speech-synthesis corpora

We investigate the problem of automatic detection of annotation errors in single-speaker read-speech corpora used for speech synthesis. For the purpose of annotation error detection, we adopt an anomaly detection framework in which correctly annotated words are considered as normal examples on which the detection methods are trained. Misannotated words are then taken as anomalous examples which do not conform to normal patterns of the trained detection models. We propose and evaluate several anomaly detection models – Gaussian distribution based detectors, Grubbs’ test based detector, and one-class support vector machine based detector. Word-level feature sets including basic features derived from forced alignment and various acoustic, spectral, phonetic, and positional features are examined to find an optimal set of features for each anomaly detector. The results with F1 score being almost 89% show that anomaly detection could help detecting annotation errors in read-speech corpora for speech synthesis. Furthermore, dimensionality reduction techniques are also examined to automatically reduce the number of features used to describe the annotated words. We show that the automatically reduced feature sets achieve statistically similar results as the hand-crafted feature sets. We also conducted additional experiments to investigate both robustness of the proposed anomaly detection framework with respect to particular data sets used for development and evaluation and the influence of the number of examples needed for anomaly detection. We show that a reasonably good detection performance could be reached with using significantly fewer examples during the detector development phase. We also propose a concept of a voting detector – a combination of anomaly detectors in which each “single” detector “votes” on whether or not a testing word is annotated correctly, and the final decision is then made by aggregating the votes. Our results show that the voting detector has a potential to overcome each of the single anomaly detectors. Furthermore, we compare the proposed anomaly detection framework to a classification-based approach (which, unlike anomaly detection, needs to use anomalous examples during training) and we show that both approaches lead to statistically comparable results when all available anomalous examples are utilized during detector/classifier development. However, when a smaller number of anomalous examples are used, the proposed anomaly detection framework clearly outperforms the classification-based approach. A final listening test showed the effectiveness of the proposed anomaly-based annotation error detection for improving the quality of synthetic speech.

Acoustic effects of phonetic conditions and laryngeal specification on phonation in English voiceless obstruents

A detailed acoustic analysis of phonation in voiceless obstruents in American English (AE) is presented to investigate potential acoustic outcomes of the laryngeal coordination relationships that have been reported in the literature. The current study examines the realization of phonation in voiceless obstruents in a corpus of read speech (37 AE speakers). Similar to phonation in voiced obstruents (Davidson 2016), linguistic factors such as phrase and word position, stress, and the preceding phoneme condition the presence and degree of phonation during the constriction period of stops and fricatives. The amount of phonation present is further analyzed by characterizing where in the constriction interval phonation appears. Carryover phonation (or “bleed”) from a preceding sonorant is most common for stops, while a “trough” pattern (phonation that dies out and then begins again before the end of the closure) is more prevalent for fricatives. The linguistic conditioning factors, and the overall lower rate of partial voicing and proportion of phonation in the constriction as compared to voiced obstruents, illustrate the acoustic and aerodynamic consequences of the various laryngeal coordination patterns that have been proposed for English voiceless obstruents in different prosodic environments.

The Importance of a Distributional Approach to Categoriality in Autosegmental-Metrical Accounts of Intonation

When annotating a speech signal using an autosegmental-metrical model of intonation, transcribers associate portions of the F0 contour with labels from a finite inventory of tonal categories. In the models we are concerned with here, these categories have the status of phonological units (phonological form), bridging the intrinsic variability of the speech signal (substance) with the intrinsic fuzziness of post-lexical function (meaning). This, together with the relatively small size of the label inventory, precludes a one-to-one relationship between form and substance, and/or between form and function. A Neapolitan Italian corpus of read speech is used to investigate the distributional properties of two pitch accents that have been studied extensively with respect to substance (the alignment of F0 peaks) and meaning (sentence modality). Although there is a general consensus that peaks in this variety are aligned earlier in declaratives than in interrogatives, evidence is provided of contexts in which the converse is true, i.e., in which interrogative peaks are even earlier than their declarative counterparts. In this respect, interrogatives have a richer internal structure than declaratives. We argue that differences in how variably a prosodic category is encoded can be dealt with in an intonation transcription system, as long as this system relates phonological form (the choice of pitch accent in this case) both to phonetic substance and to meaning in a transparent way.

Semantic language models with deep neural networks

In this paper we explore the use of semantics in training language models for automatic speech recognition and spoken language understanding. Traditional language models (LMs) do not consider the semantic constraints and train models based on fixed-sized word histories. The theory of frame semantics analyzes word meanings and their constructs by using “semantic frames”. Semantic frames represent a linguistic scene with its relevant participants and their relations. They are triggered by target words and include slots which are filled by frame elements. We present semantic LMs (SELMs), which use recurrent neural network architectures and the linguistic scene of frame semantics as context. SELMs incorporate semantic features which are extracted from semantic frames and target words. In this way, long-range and “latent” dependencies, i.e. the implicit semantic dependencies between words, are incorporated into LMs. This is crucial especially when the main aim of spoken language systems is understanding what the user means. Semantic features consist of low-level features, where frame and target information is directly used; and deep semantic encodings, where deep autoencoders are used to extract semantic features. We evaluate the performance of SELMs on publicly available corpora: the Wall Street Journal read-speech corpus and the LUNA human–human conversational corpus. The encoding of semantic frames into SELMs improves the word recognition performance and especially the recognition performance of the target words, the meaning bearing elements of semantic frames. We assess the performance of SELMs for the understanding tasks and we show that SELMs yield better semantic frame identification performance compared to recurrent neural network LMs.

Articulatory and excitation source features for speech recognition in read, extempore and conversation modes

In our previous works, we have explored articulatory and excitation source features to improve the performance of phone recognition systems (PRSs) using read speech corpora. In this work, we have extended the use of articulatory and excitation source features for developing PRSs of extempore and conversation modes of speech, in addition to the read speech. It is well known that the overall performance of speech recognition system heavily depends on accuracy of phone recognition. Therefore, the objective of this paper is to enhance the accuracy of phone recognition systems using articulatory and excitation source features in addition to conventional spectral features. The articulatory features (AFs) are derived from the spectral features using feedforward neural networks (FFNNs). We have considered five AF groups, namely: manner, place, roundness, frontness and height. Five different AF-based tandem PRSs are developed using the combination of Mel frequency cepstral coefficients (MFCCs) and AFs derived from FFNNs. Hybrid PRSs are developed by combining the evidences from AF-based tandem PRSs using weighted combination approach. The excitation source information is derived by processing the linear prediction residual of the speech signal. The vocal tract information is captured using MFCCs. The combination of vocal tract and excitation source features is used for developing PRSs. The PRSs are developed using hidden Markov models. Bengali speech database is used for developing PRSs of read, extempore and conversation modes of speech. The results are analyzed and the performance is compared across different modes of speech. From the results, it is observed that the use of either articulatory or excitation source features along-with to MFCCs will improve the performance of PRSs in all three modes of speech. The improvement in the performance using AFs is much higher compared to the improvement obtained using excitation source features.

Speaker-individuality in Fujisaki model f0 features: implications for forensic voice comparison

Fundamental frequency (f0) is a highly speaker-specific feature. Consequently, practitioners often use f0 information in forensic casework. Current research principally examines the use of long-term f0 statistics such as f0 means and standard deviations for forensic voice comparison. The present study investigates how short-term f0 features such as measured by the Fujisaki intonation model capture speaker-individuality. Based on data of a homogeneous group of Zurich German speakers, we conducted an experiment on a large corpus of read speech and on a subset of sentences that included speaking style variability (spontaneous vs. read). The latter is characteristic of forensic casework. Speakers demonstrated high between-speaker variability and low within-speaker variability across the two speaking styles for a number of f0 features. Given this evidence of speaker-individuality, we discuss Fujisaki f0 features’ potential for forensic voice comparison.

Frequency effects on the vowel length contrast merger in Seoul Korean

AbstractThis paper presents an apparent-time study of the vowel length contrast merger in Seoul Korean based on duration measurements of over 370,000 vowels in word-initial syllables in a read-speech corpus. The effects of word frequency on vowel duration and the lexical diffusion of long-vowel shortening are also examined. The findings confirm the observation made in the previous literature that the vowel length contrast is on its way out in the language, and that this sound change is nearing completion. We also find a significant effect of frequency on long-vowel duration: other things being equal, these vowels are shorter in high-frequency words than in low-frequency words. The rate of change does not differ significantly depending on the frequency of words apart from the high-frequency words reaching the endpoint of change and bottoming out in the change earlier than mid- and low-frequency words. The observed frequency effect is compatible with a model in which the frequency effect on duration comes from on-line factors that affect phonetic implementation of speech sounds, along with an across-the-board lenition bias that drives the sound change, not from stored tokens of word-specific variants.

Burst spectrum as a cue for the stop voicing contrast in American English.

Voicing contrasts in stop consonants are expressed by a constellation of acoustic cues. This study focused on a spectral cue present at burst onset in American English labial and coronal stops. Spectral shape was examined for word-initial, prevocalic stops of all three places of articulation in a laboratory production study and a large corpus of continuous read speech. Voiceless labial and coronal stops were found to have greater energy at higher frequencies in comparison to homorganic voiced stops, a difference that could not be attributed to aspiration in the voiceless stops or modal phonation in the voiced, while no consistent effect was found for dorsal stops. This pattern was found with various methods of spectral estimation (time-averaged and multitaper spectra) and measures of spectral energy concentration (center of gravity and spectral peak) for both linear and auditorily based frequency scales. Perceptual relevance of the spectral cue was tested in laboratory and online experiments with continua created by crossing burst shape and voice onset time. A trading relation was observed such that voiceless identifications were more likely for tokens with higher frequency bursts. Goodness ratings indicated that burst spectrum influences category typicality for voiceless stops even when voice onset time is unambiguous.