Models For Continuous Speech Recognition Research Articles

Building Acoustic and Language Model for Continuous Speech Recognition in Bahasa Indonesia

Here a development of an Acoustic and Language Model is presented. Low Word Error Rate is an early good sign of a good Language and Acoustic Model. Although there are still parameters other than Words Error Rate, our work focused on building Bahasa Indonesia with approximately 2000 common words and achieved the minimum threshold of 25% Word Error Rate. There were several experiments consist of different cases, training data, and testing data with Word Error Rate and Testing Ratio as the main comparison. The language and acoustic model were built using Sphinx4 from Carnegie Mellon University using Hidden Markov Model for the acoustic model and ARPA Model for the language model. The models configurations, which are Beam Width and Force Alignment, directly correlates with Word Error Rate. The configurations were set to 1e-80 for Beam Width and 1e-60 for Force Alignment to prevent underfitting or overfitting of the acoustic model. The goals of this research are to build continuous speech recognition in Bahasa Indonesia which has low Word Error Rate and to determine the optimum numbers of training and testing data which minimize the Word Error Rate.

Arabic phonemes recognition using hybrid LVQ/HMM model for continuous speech recognition

In attempt to increase the rate of Arabic phonemes recognition, we introduce a novel hybrid recognition algorithm. The algorithm is composed of the learning vector quantization (LVQ) and hidden Markov model (HMM). The hybrid algorithm used to recognizing Arabic phonemes in continuous open-vocabulary speech. A recorded Arabic corpus of different TV news for modern standard Arabic was used for training and testing purposes. We employ a data driven approach to generate the training feature vectors that embed the frame neighboring correlation information. Next, we generate the phonemes codebooks using the K-means splitting algorithm. Then, we trained the generated codebooks using the LVQ algorithm. We achieved a performance of 98.49 % during independent classification training and 90 % during dependent classification training. When using the trained LVQ codebooks in Arabic utterance transcription, the phoneme recognition rate was 72 % using LVQ only. We combined the LVQ codebooks with the single state HMM model using enhanced Viterbi algorithm which includes the phonemes bigrams. We achieved 89 % of Arabic phonemes recognition rate based on the hybrid LVQ/HMM algorithm.

Review on Acoustic Modeling for Continuous Speech Recognition

Review on Acoustic Modeling for Continuous Speech Recognition

Integrated exemplar-based template matching and statistical modeling for continuous speech recognition

We propose a novel approach of integrating exemplar-based template matching with statistical modeling to improve continuous speech recognition. We choose the template unit to be context-dependent phone segments (triphone context) and use multiple Gaussian mixture model (GMM) indices to represent each frame of speech templates. We investigate two different local distances, log likelihood ratio (LLR) and Kullback-Leibler (KL) divergence, for dynamic time warping (DTW)-based template matching. In order to reduce computation and storage complexities, we also propose two methods for template selection: minimum distance template selection (MDTS) and maximum likelihood template selection (MLTS). We further propose to fine tune the MLTS template representatives by using a GMM merging algorithm so that the GMMs can better represent the frames of the selected template representatives. Experimental results on the TIMIT phone recognition task and a large vocabulary continuous speech recognition (LVCSR) task of telehealth captioning demonstrated that the proposed approach of integrating template matching with statistical modeling significantly improved recognition accuracy over the hidden Markov modeling (HMM) baselines for both TIMIT and telehealth tasks. The template selection methods also provided significant accuracy gains over the HMM baseline while largely reducing the computation and storage complexities. When all templates or MDTS were used, using the LLR local distance gave better performance than the KL local distance. For MLTS and template compression, KL local distance gave better performance than the LLR local distance, and template compression further improved the recognition accuracy on top of MLTS while having less computational cost.

Syllable modeling in continuous speech recognition for Tamil language

In automatic speech recognition, the phone has probably been a dominating sub-word unit for more than one decade. Context Dependent phone or triphone modeling accounts for contextual variations between adjacent phones and state tying addresses modeling of triphones that are not seen during training. Recently, syllable is gaining momentum as a new sub-word unit. Syllable being a larger unit than a phone addresses the severe contextual variations between phones within it. Therefore, it is more stable than a phone and models pronunciation variability in a systematic way. Tamil language has challenging features like agglutination and morpho-phonology. In this paper, attempts have been made to provide solutions to these issues by using the syllable as a sub-word unit in an acoustic model. Initially, a small vocabulary context independent word models and a medium vocabulary context dependent phone models are developed. Subsequently, an algorithm based on prosodic syllable is proposed and two experiments have been conducted. First, syllable based context independent models have been trained and tested. Despite large number of syllables, this system has performed reasonably well compared to context independent word models in terms of word error rate and out of vocabulary words. Subsequently, in the second experiment, syllable information is integrated in conventional triphone modeling wherein cross-syllable triphones are replaced with monophones and the number of context dependent phone models is reduced by 22.76% in untied units. In spite of reduction in the number of models, the accuracy of the proposed system is comparable to that of the baseline triphone system.

Framework for Choosing a Set of Syllables and Phonemes for Lithuanian Speech Recognition

This paper describes a framework for making up a set of syllables and phonemes that subsequently is used in the creation of acoustic models for continuous speech recognition of Lithuanian. The target is to discover a set of syllables and phonemes that is of utmost importance in speech recognition. This framework includes operations with lexicon, and transcriptions of records. To facilitate this work, additional programs have been developed that perform word syllabification, lexicon adjustment, etc. Series of experiments were done in order to establish the framework and model syllable- and phoneme-based speech recognition. Dominance of a syllable in lexicon has improved speech recognition results and encouraged us to move away from a strict definition of syllable, i.e., a syllable becomes a simple sub-word unit derived from a syllable. Two sets of syllables and phonemes and two types of lexicons have been developed and tested. The best recognition accuracy achieved 56.67% ±0.33. The speech recognition system is based on Hidden Markov Models (HMM). The continuous speech corpus LRN0 was used for the speech recognition experiments.

A hierarchical Bayesian model for continuous speech recognition

This paper proposes a stochastic model for continuous speech recognition that provides automatic segmentation of a spoken utterance into phonemes and facilitates the quantitative assessment of uncertainty associated with the identified utterance features. The model is specified hierarchically within the Bayesian paradigm. At the lowest level of the hierarchy, a Gibbs distribution is used to specify a probability distribution on all the possible partitions of the utterance. The number of partitioning elements which are phonemes is not specified a priori. At a higher level in the hierarchical specification, random variables representing phoneme durations and acoustic vector values are associated reported about 0.9% word error rate. The new model was experimentally compared to continuous density mixture HMM (CDHMM) on a same recognition task, and gave significantly smaller word error rates.

Audio-visual speech modeling for continuous speech recognition

This paper describes a speech recognition system that uses both acoustic and visual speech information to improve recognition performance in noisy environments. The system consists of three components: a visual module; an acoustic module; and a sensor fusion module. The visual module locates and tracks the lip movements of a given speaker and extracts relevant speech features. This task is performed with an appearance-based lip model that is learned from example images. Visual speech features are represented by contour information of the lips and grey-level information of the mouth area. The acoustic module extracts noise-robust features from the audio signal. Finally the sensor fusion module is responsible for the joint temporal modeling of the acoustic and visual feature streams and is realized using multistream hidden Markov models (HMMs). The multistream method allows the definition of different temporal topologies and levels of stream integration and hence enables the modeling of temporal dependencies more accurately than traditional approaches. We present two different methods to learn the asynchrony between the two modalities and how to incorporate them in the multistream models. The superior performance for the proposed system is demonstrated on a large multispeaker database of continuously spoken digits. On a recognition task at 15 dB acoustic signal-to-noise ratio (SNR), acoustic perceptual linear prediction (PLP) features lead to 56% error rate, noise robust RASTA-PLP (relative spectra) acoustic features to 7.2% error rate and combined noise robust acoustic features and visual features to 2.5% error rate.

A Bayesian approach for building triphone models for continuous speech recognition

This paper introduces a new statistical framework for constructing triphonic models from models of less context-dependency. This composition reduces the number of models to be estimated by higher than an order of magnitude and is therefore of great significance in relieving the data sparsity problem in triphone-based continuous speech recognition. The new framework is derived from Bayesian statistics, and represents an alternative to other triphone-by-composition techniques, particularly to the model-interpolation and quasitriphone approaches. The potential power of this new framework is explored by an implementation based on the hidden Markov modeling technique. It is shown that the new model structure includes the quasitriphone model as a special case, and leads to more efficient parameter estimation than the model-interpolation method. Phone recognition experiments show an increase in the accuracy over that obtained by comparable models.

Phonetic transition modeling for continuous speech recognition

Currently, most speech recognition architectures model the speech signal as a nonoverlapping sequence of phonetic segments. A set of phonetic models is created that attempt to capture the acoustic-phonetic properties of individual phones, but do not explicitly model the transition between phones. It is readily apparent, however, that these transitions contain important information about the identity of neighboring phones. While context-dependent phonetic modeling may capture some of this information, it is likely that more explicit models of phonetic transitions could offer performance improvements. In this talk, the use of phonetic transition models will be discussed within the context of summit, a segment-based continuous speech recognition system [Zue etal., ‘‘Acoustic Segmentation and Phonetic Classification in the summit Speech Recognition System,’’ Proc. ICASSP 89, pp. 389–392, Glasgow, Scotland (1989)]. The transition models use a feature vector based on Mel-frequency spectral coefficients (MFSC’s). The vector is created by concatenating multiple spectral averages on both sides of a transition. For example, in one configuration a total of eight averages were used which spanned a total time interval of 150 ms. In order to reduce the number of dimensions, a principal component analysis was performed. A set of diagonal Gaussian models is used to model the transitions. The models were tested by applying them to the N-Best sentence hypotheses from the recognition system. Each of the N-Best hypotheses is rescored using a linear combination (optimized on training data) of the segment and transition scores. Initial experiments have resulted in 10%–20% reductions in word error rates.

A HYBRID CONTINUOUS SPEECH RECOGNITION SYSTEM USING SEGMENTAL NEURAL NETS WITH HIDDEN MARKOV MODELS

Up till recently, state-of-the-art, large vocabulary, continuous speech recognition (CSR) had employed hidden Markov modeling (HMM) to model speech sounds. In an attempt to improve over HMM we developed a hybrid system that integrates HMM technology with neural networks. We present the concept of a Segmental Neural Net (SNN) for phonetic modeling in CSR. By taking into account all the frames of a phonetic segment simultaneously, the SNN overcomes the well-known conditional-independence limitation of HMMs. We have developed a novel hybrid SNN/HMM system that combines the advantages of SNNs and HMMs using a multiple hypothesis (or N-best) paradigm. In this system, we generate likely phonetic segmentations from the HMM N-best list of word sequences, which are scored by the SNN. The HMM and SNN scores are then combined to optimize performance. In several speaker-independent, 1000-word CSR tests, the error rate for the hybrid system dropped 20% from that of a state-of-the-art HMM system alone.

Context modeling with the stochastic segment model

An approach for context modeling in continuous speech recognition for models based on multivariate Gaussian distributions, specifically, the stochastic segment model, is described. Robust context models are obtained by typing distribution parameters across different classes of context. Experimental results in phoneme and word recognition are comparable to those achieved with discrete hidden Markov models using mixture distributions. >

Development of an acoustic-phonetic hidden Markov model for continuous speech recognition

The techniques used to develop an acoustic-phonetic hidden Markov model, the problems associated with representing the whole acoustic-phonetic structure, the characteristics of the model, and how it performs as a phonetic decoder for recognition of fluent speech are discussed. The continuous variable duration model was trained using 450 sentences of fluent speech, each of which was spoken by a single speaker, and segmented and labeled using a fixed number of phonemes, each of which has a direct correspondence to the states of the matrix. The inherent variability of each phoneme is modeled as the observable random process of the Markov chain, while the phonotactic model of the unobservable phonetic sequence is represented by the state transition matrix of the hidden Markov model. The model assumes that the observed spectral data were generated by a Gaussian source. However, an analysis of the data shows that the spectra for the most of the phonemes are not normally distributed and that an alternative representation would be beneficial.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Interword coarticulation modeling for continuous speech recognition

In large‐vocabulary continuous speech recognition, subword units must be used for practical reasons. Context‐dependent phone models have become a very successful class of subword units. These phone‐sized models take into account the neighboring phonetic contexts, which strongly affect the realization of a phone. However, previous approaches have only considered intraword coarticulation, and have ignored interword coarticulation, which is very important in continuous speech, especially for short function words like “the” and “a.” This study extends triphone‐based modeling to interword coarticulation modeling. A simple extension of triphones is problematic due to the sharply growing number of triphones. In order to contain this growth, a maximum‐likelihood clustering procedure was introduced to reduce 7057 intraword and interword triphones to 1000 generalized triphones. Interword generalized triphones were incorporated into a large‐vocabulary, speaker‐independent, continuous speech recognizer, SPHINX [K. F. Lee and H. W. Hon, Large Vocabulary Speaker‐lndependent Continuous Speech Recognition (ICASSP, 1988)]. This improvement reduced the number of errors by as much as 44% on the 1000‐word DARPA resource management task. This demonstrates the importance of interword coarticulation modeling, and the effectiveness of the methods used.

On the use of triphone models for continuous speech recognition

This paper describes both completed and future work with triphone models for speech recognition. Triphone models are a powerful subword modeling technique because they account for the left and right phonetic contexts. One problem with triphone models is that there are too many models for the amount of training data available. Moreover, many triphone contexts are very similar. In view of this, generalized triphones are introduced, which are created by clustering similar triphones together using an information theoretic criterion. This technique provides a way of finding the “right” number of models for any task. With generalized triphone models, a 96% accuracy with a 1000-word speaker-independent continuous task was achieved. There are only 2500 triphones for the above task. For a different task, there will be new triphones; for a natural English task, the number of triphones will be much larger. In the future, improvement of the applicability of triphone modeling could be done by (1) training a large set of generalized triphone models for English, and (2) exploring techniques to deal with new triphones. [Work supported by DARPA.]