Large Speech Corpus Research Articles

Modeling Phone Duration of Lithuanian by Classification and Regression Trees, using Very Large Speech Corpus

Classification and regression tree approach was used in this research to model phone duration of Lithuanian. 300 thousand samples of vowels and 400 thousand samples of consonants extracted from VDU-AB20 corpus were used in experimental part of research. Set of 15 parameters characterizing phone and its context were selected for duration prediction. The most significant of them were: identifier (ID) of phone being predicted, adjacent phones IDs and number of phones in syllable. Models were built using two different data sets: one speaker and 20 speakers. The influence of cost complexity pruning and different values of pre pruning were investigated. Prediction by average leaf duration vs. prediction by median leaf duration was also compared. Investigation of most vivid errors was performed, speech rate normalization and trivial noise reduction were applied and influence on models evaluation parameters discussed. The achieved results, correlation 0.8 and 0.75 respectively for vowels and consonants, and RMSE of ~18 ms are comparable with those reported for Check, Hindi and Telugu, Korean.

Automatic phonetic transcription of large speech corpora

Most large speech corpora are delivered with a lexicon that contains a canonical transcription of every word in the orthographic transcription. Such a lexicon can be used for generating a hypothetical ‘canonical’ phonetic transcription from the orthography. In addition, time and money permitting, some speech corpora are provided with a manually verified broad phonetic transcription of at least part of the material. Since the manual verification of phonetic transcriptions is time-consuming and expensive, we investigated whether existing automatic transcription procedures and combinations of such procedures can offer a quick and cheap alternative for the generation of phonetic transcriptions like the manually verified transcriptions delivered with large speech corpora. In our study, we used 10 automatic transcription procedures to generate a broad phonetic transcription of well-prepared speech (read-aloud texts) and spontaneous speech (telephone dialogues) from the Spoken Dutch Corpus. The performance was assessed in terms of the number and the nature of the discrepancies between the emerging phonetic transcriptions and the corresponding manually verified phonetic transcriptions delivered with the Spoken Dutch Corpus. Some of the resulting automatic transcriptions appeared to be comparable to the manually verified transcriptions.

Methods and apparatus for rapid acoustic unit selection from a large speech corpus

A speech synthesis system can select recorded speech fragments, or acoustic units, from a very large database of acoustic units to produce artificial speech. The selected acoustic units are chosen to minimize a combination of target and concatenation costs for a given sentence. However, as concatenation costs, which are measures of the mismatch between sequential pairs of acoustic units, are expensive to compute, processing can be greatly reduced by pre-computing and caching the concatenation costs. Unfortunately, the number of possible sequential pairs of acoustic units makes such caching prohibitive. However, statistical experiments reveal that while about 85% of the acoustic units are typically used in common speech, less than 1% of the possible sequential pairs of acoustic units occur in practice. A method for constructing an efficient concatenation cost database is provided by synthesizing a large body of speech, identifying the acoustic unit sequential pairs generated and their respective concatenation costs, and storing those concatenation costs likely to occur. By constructing a concatenation cost database in this fashion, the processing power required at run-time is greatly reduced with negligible effect on speech quality.

Incorporating Phonetic Knowledge Into an Evolutionary Subspace Approach for Robust Speech Recognition

The reliability of automatic speech recognition (ASR) systems is closely related to the parameterization process which is expected to accurately characterize the phonetic, dynamic and static components in speech. For this purpose, ASR methods build speech sound models based on large speech corpora that attempt to include common sources of variability that may occur in real-life conditions. Nevertheless, not all variabilities can reasonably be covered. For that reason, the performance of current ASR systems, whose designs are predicated on relatively noise-free conditions, degrades rapidly in the presence of high-level adverse conditions. To cope with mismatched (adverse) conditions and to achieve noise robustness, we present in this paper an original approach that operates in two steps. The first one consists of integrating in the front-end process, besides mean-subtracted mel-frequency cepstral coefficients, acoustic distinctive features that provides a more convenient interface to higher-level components of ASR systems. The second step consists of combining subspace filtering and Genetic Algorithms to get less-variant parameters. The advantages of this approach include that no estimation of noise is required and the recognition system is not modified. The effectiveness of the method is assessed in high interfering car noise by using a noisy subset of the TIMIT database. Obtained results show that the proposed method reduces drastically the word error rate for a wide range of signal-to-noise ratios.

MLP-based phone boundary refining for a TTS database

The automatic labeling of a large speech corpus plays an important role in the development of a high-quality Text-To-Speech (TTS) synthesis system. This paper describes a method for the automatic labeling of speech signals, which mainly involves the construction of a large database for a TTS synthesis system. The main objective of the work involves the refinement of an initial estimation of phone boundaries which are provided by an alignment, based on a Hidden Markov Model. A multilayer perceptron (MLP) was employed to refine the phone boundaries. To increase the accuracy of phoneme segmentation, several specialized MLPs were individually trained based on phonetic transition. The optimum partitioning of the entire phonetic transition space and the corresponding MLPs were constructed from the standpoint of minimizing the overall deviation from the hand-labeling position. The experimental results showed that more than 93% of all phone boundaries have a boundary deviation from a reference position smaller than 20 ms. We also confirmed that the database constructed using the proposed method produced results that were perceptually comparable to a hand-labeled database, based on subjective listening tests.

A novel visualization tool for manual annotation when building large speech corpora

A novel visualized sound description, called sound dendrogram is proposed to make manual annotation easier whenbuilding large speech corpora. It is a lattice structure built from a group of “seed regions” and through an iteractive procedure of mergence. A simple but reliable extraction method of “seed regions” and advanced distance metric are adopted to construct the sound dendrogram, so that it can present speech’s structure character ranging from coarse to fine in a visualized way. Tests show that all phonemic boundaries are contained in the lattice structure of sound dendrogram and very easy to identify. Sound dendrogram can be a powerful assistant tool during the process of speech corpora’s manual annotation.

On temporal aspects of turn taking in conversational dialogues

In this short communication we show how shallow annotations in large speech corpora can be used to derive data about the temporal aspects of turn taking. Within the limitations of such a speech corpus, we show that the average durations of between-turn pauses made by speakers in a dyad are statistically related, and our data suggest the existence of gender effects in the temporal aspects of turn taking. Also, clear differences in turn taking behaviour between face-to-face and telephone dialogues can be detected using shallow analyses. We discuss the most important limitations imposed by the shallowness of the annotations in large corpora, and the possibility for enriching those annotations in a semi-automatic iterative manner.

Statistical modeling of phonological rules through linguistic hierarchies

This paper describes our research aimed at acquiring a generalized probability model for alternative phonetic realizations in conversational speech. For all of our experiments, we utilize the summit landmark-based speech recognition framework. The approach begins with a set of formal context-dependent phonological rules, applied to the baseforms in the recognizer’s lexicon. A large speech corpus is phonetically aligned using a forced recognition procedure. The probability model is acquired by observing specific realizations expressed in these alignments. A set of context-free rules is used to parse words into substructure, in order to generalize context-dependent probabilities to other words that share the same sub-word context. The model maps phones to sub-word units probabilistically in a finite state transducer framework, capturing phonetic predictions based on local phonemic, morphologic, and syllabic contexts. We experimented within two domains: the mercury flight reservation domain and the jupiter weather domain. The baseline system used the same set of phonological rules for lexical expansion, but with no probabilities for the alternates. We achieved 14.4% relative reduction in concept error rate for jupiter and 16.5% for mercury.

A set of corpus-based text-to-speech synthesis technologies for Mandarin Chinese

This paper presents a set of corpus-based text-to-speech synthesis technologies for Mandarin Chinese. A large speech corpus produced by a single speaker is used, and the speech output is. synthesized from waveform units of variable lengths, with desired linguistic properties, retrieved from this corpus. Detailed methodologies were developed for designing phonetically rich and prosodically rich corpora by automatically selecting sentences from a large text corpus to include as many desired phonetic combinations and prosodic features as possible. Automatic phonetic labeling with iterative correction rules and automatic prosodic labeling with a multi-pass top-down procedure were also developed such that the labeling process for the corpora can be completely automatic. A hierarchical prosodic structure for an arbitrary desired text sentence is then generated based on the identification of different levels of break indices, and the prosodic feature sets and appropriate waveform units are finally selected and retrieved from the corpus, modified if necessary, and concatenated to produce the output speech. The special structure of Mandarin Chinese has been carefully considered in all these technologies, and preliminary assessments indicated very encouraging synthesized speech quality.

A corpus-based speech synthesis system with emotion

We propose a new approach to synthesizing emotional speech by a corpus-based concatenative speech synthesis system (ATR CHATR) using speech corpora of emotional speech. In this study, neither emotional-dependent prosody prediction nor signal processing per se is performed for emotional speech. Instead, a large speech corpus is created per emotion to synthesize speech with the appropriate emotion by simple switching between the emotional corpora. This is made possible by the normalization procedure incorporated in CHATR that transforms its standard predicted prosody range according to the source database in use. We evaluate our approach by creating three kinds of emotional speech corpus (anger, joy, and sadness) from recordings of a male and a female speaker of Japanese. The acoustic characteristics of each corpus are different and the emotions identifiable. The acoustic characteristics of each emotional utterance synthesized by our method show clear correlations to those of each corpus. Perceptual experiments using synthesized speech confirmed that our method can synthesize recognizably emotional speech. We further evaluated the method’s intelligibility and the overall impression it gives to the listeners. The results show that the proposed method can synthesize speech with a high intelligibility and gives a favorable impression. With these encouraging results, we have developed a workable text-to-speech system with emotion to support the immediate needs of nonspeaking individuals. This paper describes the proposed method, the design and acoustic characteristics of the corpora, and the results of the perceptual evaluations.

A New Korean Corpus-Based Text-to-Speech System

This paper describes a new Korean Text-to-Speech (TTS) system based on a large speech corpus. Conventional concatenative TTS systems still produce machine-like synthetic speech. The poor naturalness is caused by excessive prosodic modification using a small speech database. To cope with this problem, we utilized a dynamic unit selection method based on a large speech database without prosodic modification. The proposed TTS system adopts triphones as synthesis units. We designed a new sentence set maximizing phonetic or prosodic coverage of Korean triphones. All the utterances were segmented automatically into phonemes using a speech recognizer. With the segmented phonemes, we achieved a synthesis unit cost of zero if two synthesis units were placed consecutively in an utterance. This reduces the number of concatenating points that may occur due to concatenating mismatches. In this paper, we present data concerning the realization of major prosodic variations through a consideration of prosodic phrase break strength. The phrase break was divided into four kinds of strength based on pause length. Using phrase break strength, triphones were further classified to reflect major prosodic variations. To predict phrase break strength on texts, we adopted an HMM-like Part-of-Speech (POS) sequence model. The performance of the model showed 73.5% accuracy for 4-level break strength prediction. For unit selection, a Viterbi beam search was performed to find the most appropriate triphone sequence, which has the minimum continuation cost of prosody and spectrum at concatenating boundaries. From the informal listening test, we found that the proposed Korean corpus-based TTS system showed better naturalness than the conventional demisyllable-based one.

Representational issues in annotation: Using the Australian map task corpus to relate prosody and discourse structure

This paper reports part of an ongoing investigation of the interaction of prosody and discourse structure. A digital speech corpus (4 dialogues from the ANDOSL Australian map task corpus) was coded for prosodic structure (ToBI). Independently, two different coding systems for dialogue micro-structure were applied to the same corpus: the HCRC map task coding scheme (Carletta et al., 1996, 1997b) and the `Switchboard' version of the DRI/DAMSL scheme (Jurafsky et al., 1997). We investigated whether silent pause location and duration, intonational boundaries associated with Break Indices 3 and 4, as well as pitch range reset were significantly correlated with dialogue act boundaries as has been found for other varieties of English (e.g., Lehiste, 1975; Hirschberg and Nakatani, 1996; Silverman, 1987) and Dutch (Swerts, 1997). The dialogue coding systems were systematically evaluated both against one another and in terms of their correlation with the prosodic structure. The paper explores a number of methodological issues which arise in effectively comparing and relating structures from different domains of analysis across a large speech corpus. It also exemplifies the way in which annotated corpora can be used to evaluate theories and systems.

Automatic ToBI prediction and alignment to speed manual labeling of prosody

Tagging of corpora for useful linguistic categories can be a time-consuming process, especially with linguistic categories for which annotation standards are relatively new, such as discourse segment boundaries or the intonational events marked in the Tones and Break Indices (ToBI) system for American English. A ToBI prosodic labeling of speech typically takes even experienced labelers from 100 to 200 times real time. An experiment was conducted to determine (1) whether manual correction of automatically assigned ToBI labels would speed labeling, and (2) whether default labels introduced any bias in label assignment. A large speech corpus of one female speaker reading several types of texts was automatically assigned default labels. Default accent placement and phrase boundary location were predicted from text using machine learning techniques. The most common ToBI labels were assigned to these locations for default tones and break type. Predicted pitch accents were automatically aligned to the mid-point of the word, while breaks and edge tones were aligned to the end of the phrase-final word. The corpus was then labeled by a group of five trained transcribers working over a period of nine months. Half of each set of recordings was labeled in the standard fashion without default labels, and the other half was presented with preassigned default labels for labelers to correct. Results indicate that labeling from defaults was generally faster than standard labeling, and that defaults had relatively little impact on label assignment.

AHUMADA: A large speech corpus in Spanish for speaker characterization and identification

Speaker recognition is an emerging task in both commercial and forensic applications. Nevertheless, while in certain applications we can estimate, adapt or hypothesize about our working conditions, most of the commercial applications and almost the whole of the forensic approaches to speaker recognition are still open problems, due to several reasons. Some of these reasons can be stated: environmental conditions are (usually) rapidly changing or highly degraded, acquisition processes are not always under control, incriminated people exhibit low degree of cooperativeness, etc., inducing a wide range of variability sources on speech utterances. In this sense, real approaches to speaker identification necessarily imply taking into account all these variability factors. In order to isolate, analyze and measure the effect of some of the main variability sources that can be found in real commercial and forensic applications, and their influence in automatic recognition systems, a specific large speech database in Castilian Spanish called AHUMADA (/aumáda/) has been designed and acquired under controlled conditions. In this paper, together with a detailed description of the database, some experimental results including different speech variability factors are also presented.

Automatic ToBI prediction and alignment to speed manual labeling of prosody

ToBI (Tones and Break Indices) prosodic labeling of a speech corpus is a slow, labor-intensive process that typically takes from 100 to 300 times real time, even with experienced labelers. An experiment was conducted to determine: (1) whether manual correction of automatically assigned ToBI labels would speed up the labeling process; and (2) whether default labels introduced any bias in label assignment. A group of four graduate linguistics students previously trained in ToBI labeling were paid participants in the study. A large speech corpus of one female speaker reading several types of texts was labeled over a period of nine months. Half of each recording was labeled in the normal fashion ‘‘from scratch’’ without default labels, and the other half was presented with preassigned default labels for labelers to correct. Default ToBI labels were predicted from text using techniques developed for text-to-speech synthesis. Both labeling methods used standard Entropic waves+ tools developed for ToBI transcription. A log file was created during the labeling of each file that identified labeler, filename, method, and beginning and ending times of each session. Results indicate that labeling from defaults was faster than standard labeling, and that defaults had relatively little impact on label assignment.

The mu + system for corpus based speech research

mu+ is a system for corpus based speech research that can be used to retrieve and analyse segments and their associated signal files from a large speech corpus. The segments can occur at many different levels (acoustic-phonetic, phonemic, intonational, prosodic), while the signal files can include the acoustic speech waveform, analysis parameters derived from the speech waveform (e.g. formant frequencies), and various articulatory measurements (e.g. kinematic parameters from lip and jaw movement). Most combinations of segment types, together with their boundary times and the speech signal files with which they are associated, can be retrieved hierarchically (all phonemes that occur in certain words), sequentially (all phonemes that occur in a particular triphone) or hierarchically and sequentially (e.g. all phonemes that occur in content words which are preceded by an intonational phrase of a particular type). The segments and their associated signal files that are retrieved from the speech database can be analysed subsequently using a wide range of statistical primitives and digital-signal-processing routines. The system has been developed to provide a common environment for experimentation in numerous facets of corpus based speech and language research including: articulatory and acoustic phonetics, prosodic analysis, speech technology research, and linguistic corpus development.