Semi-continuous Hidden Markov Models Research Articles

Gesture Interaction for Coronary Heart Diseases Based on Wavelet Transform and Semi-Continuous Hidden Markov Model in Augmented Reality

Background: Most doctors are used to locate the vascular stenosis position first then clinically estimate vascular stenosis by the CAG images instead of using mouse, keyboard and computer during the preoperative diagnosis. Therefore, it is necessary to have an effective, intuitive and natural method to replace the existing preoperative diagnosis mode for diagnosing the coronary heart disease. Keywords: Augmented reality, coronary heart disease, gesture interaction, gesture recognition, semi-continuous hidden markov model, wavelet transform.

SCHMM-based compensation for the random delays in networked control systems

In order to compensate the network-induced random delays in networked control systems (NCSs), the semi-continuous hidden Markov model (SCHMM) is introduced in this paper to model the controller-to-actuator (CA) delay in the forward network channel. The expectation maximization algorithm is used to obtain the optimal estimation of the model's parameters, and the Viterbi algorithm is used to predict the CA delay in the current sampling period. Thus, the predicted CA delay and the measured sensor-tocontroller (SC) delay in the current sampling period are used to design an optimal controller. Under this controller, the exponentially mean square stability of the NCS is guaranteed, and the SC and CA delays are compensated. Finally, the effectiveness of the method proposed in this paper is demonstrated by a simulation example. Moreover, a comparative example is also given to illustrate the superiority of the SCHMM-based optimal controller over the discrete hidden Markov model (DHMM)-based optimal controller.

Initialization of the HMM-based delay model in networked control systems

When hidden Markov models (HMMs, including discrete HMM and semi-continuous HMM) are used to model and predict the random delays in networked control systems, there are five parameters needed to be estimated. They are the number of different network states, the initial distribution of the network states, the state transition matrix of the hidden Markov chain formed by the network states, the number of different delay observations in the discrete HMM (DHMM) or the number of the Gaussian densities in the semi-continuous HMM (SCHMM), and the delay observation matrix in the DHMM or the combination of the mixture Gaussian distributions in the SCHMM. How to initialize these parameters is very crucial to the precision of the modeling and prediction of random delays. In this paper, the entropy and cluster based initialization methods are proposed to obtain the optimal initialization of these parameters. The effectiveness of the proposed methods is demonstrated by some simulation examples.

Offline Arabic Handwriting Recognition System Based on the Combination of Multiple Semi-Continuous HMMs

In this paper, we present a multi-classifier approach for off-line handwritten Arabic word recognition system. The main objective of this paper is to develop a handwriting recognition system that can be learned and applied to different Arabic writing styles. We propose an approach that combines multiple classifiers based on semi-continuous Hidden Markov Model using different feature extraction methods. The following process consists of several phases: pre-processing, extraction of pertinent characteristics and modeling. The role of the pre-processing phase is to prepare the input image text, i.e, binarization, normalization, segmentation and skeletonization. The obtained images are then used to extract features using a mixture of geometrical and statistical characteristics, namely, the intensities of gray level of the pixel, the densities and the concavities of the pixels, the VH2D projections and the invariants Hu moments. The modeling phase is based on the Hidden Markov Model using the HTK tools for the training and the recognition phase. Each character is modeled by a semi-continuous HMMs, and each set of feature has its own HMM. To improve the performance of our Arabic handwriting recognition system, we propose to combine parallel methods of HMMs having the same architecture, but training phase using four different types of primitive vectors. Our system was evaluated using the ENIT/IFN the base Arabic data. Results Obtained show that the combination of four semi-continuous HMM classifiers gives a significant improvement of our off-line handwriting recognition system compared to results obtained when using individual classifiers.

SCHMM-based modeling and prediction of random delays in networked control systems

This paper is concerned with the modeling and prediction of random delays in networked control systems. Considering the Gaussian mixture distribution followed by the random delays in real networks, the semi-continuous hidden Markov model (SCHMM) is proposed in this paper to model the random delays. The initialization and optimization problems of the model parameters are solved by using the K-mean clustering algorithm and the expectation maximization algorithm. Based on the model, the prediction of the controller-to-actuator (CA) delay in the current sampling period is obtained. The prediction can be used to design a controller to compensate the CA delay in the future research. Two illustrative examples are given to demonstrate the effectiveness and superiority of the proposed modeling and prediction methods.

A Model-Based Sequence Similarity with Application to Handwritten Word Spotting

This paper proposes a novel similarity measure between vector sequences. We work in the framework of model-based approaches, where each sequence is first mapped to a Hidden Markov Model (HMM) and then a measure of similarity is computed between the HMMs. We propose to model sequences with semicontinuous HMMs (SC-HMMs). This is a particular type of HMM whose emission probabilities in each state are mixtures of shared Gaussians. This crucial constraint provides two major benefits. First, the a priori information contained in the common set of Gaussians leads to a more accurate estimate of the HMM parameters. Second, the computation of a similarity between two SC-HMMs can be simplified to a Dynamic Time Warping (DTW) between their mixture weight vectors, which significantly reduces the computational cost. Experiments are carried out on a handwritten word retrieval task in three different datasets-an in-house dataset of real handwritten letters, the George Washington dataset, and the IFN/ENIT dataset of Arabic handwritten words. These experiments show that the proposed similarity outperforms the traditional DTW between the original sequences, and the model-based approach which uses ordinary continuous HMMs. We also show that this increase in accuracy can be traded against a significant reduction of the computational cost.

A scalable architecture for multilingual speech recognition on embedded devices

In-car infotainment and navigation devices are typical examples where speech based interfaces are successfully applied. While classical applications are monolingual, such as voice commands or monolingual destination input, the trend goes towards multilingual applications. Examples are music player control or multilingual destination input. As soon as more languages are considered the training and decoding complexity of the speech recognizer increases. For large multilingual systems, some kind of parameter tying is needed to keep the decoding task feasible on embedded systems with limited resources. A traditional technique for this is to use a semi-continuous Hidden Markov Model as the acoustic model. The monolingual codebook on which such a system relies is not appropriate for multilingual recognition. We introduce Multilingual Weighted Codebooks that give good results with low decoding complexity. These codebooks depend on the actual language combination and increase the training complexity. Therefore an algorithm is needed that can reduce the training complexity. Our first proposal are mathematically motivated projections between Hidden Markov Models defined in Gaussian spaces. Although theoretically optimal, these projections were difficult to employ directly in speech decoders. We found approximated projections to be most effective for practical application, giving good performance without requiring major modifications to the common speech recognizer architecture. With a combination of the Multilingual Weighted Codebooks and Gaussian Mixture Model projections we create an efficient and scalable architecture for non-native speech recognition. Our new architecture offers a solution to the combinatoric problems of training and decoding for multiple languages. It builds new multilingual systems in only 0.002% of the time of a traditional HMM training, and achieves comparable performance on foreign languages.

Handwritten word-spotting using hidden Markov models and universal vocabularies

Handwritten word-spotting is traditionally viewed as an image matching task between one or multiple query word-images and a set of candidate word-images in a database. This is a typical instance of the query-by-example paradigm. In this article, we introduce a statistical framework for the word-spotting problem which employs hidden Markov models (HMMs) to model keywords and a Gaussian mixture model (GMM) for score normalization. We explore the use of two types of HMMs for the word modeling part: continuous HMMs (C-HMMs) and semi-continuous HMMs (SC-HMMs), i.e. HMMs with a shared set of Gaussians. We show on a challenging multi-writer corpus that the proposed statistical framework is always superior to a traditional matching system which uses dynamic time warping (DTW) for word-image distance computation. A very important finding is that the SC-HMM is superior when labeled training data is scarce—as low as one sample per keyword—thanks to the prior information which can be incorporated in the shared set of Gaussians.

Compact Acoustic Models for Embedded Speech Recognition

Speech recognition applications are known to require a significant amount of resources. However, embedded speech recognition only authorizes few KB of memory, few MIPS, and small amount of training data. In order to fit the resource constraints of embedded applications, an approach based on a semicontinuous HMM system using state-independent acoustic modelling is proposed. A transformation is computed and applied to the global model in order to obtain each HMM state-dependent probability density functions, authorizing to store only the transformation parameters. This approach is evaluated on two tasks: digit and voice-command recognition. A fast adaptation technique of acoustic models is also proposed. In order to significantly reduce computational costs, the adaptation is performed only on the global model (using related speaker recognition adaptation techniques) with no need for state-dependent data. The whole approach results in a relative gain of more than 20% compared to a basic HMM-based system fitting the constraints.

Semi-continuous HMMs with explicit state duration for unconstrained Arabic word modeling and recognition

In this paper, we describe an off-line unconstrained handwritten Arabic word recognition system based on segmentation-free approach and semi-continuous hidden Markov models (SCHMMs) with explicit state duration. Character durations play a significant part in the recognition of cursive handwriting. The duration information is still mostly disregarded in HMM-based automatic cursive handwriting recognizers due to the fact that HMMs are deficient in modeling character durations properly. We will show experimentally that explicit state duration modeling in the SCHMM framework can significantly improve the discriminating capacity of the SCHMMs to deal with very difficult pattern recognition tasks such as unconstrained handwritten Arabic recognition. In order to carry out the letter and word model training and recognition more efficiently, we propose a new version of the Viterbi algorithm taking into account explicit state duration modeling. Three distributions (Gamma, Gauss and Poisson) for the explicit state duration modeling have been used and a comparison between them has been reported. To perform word recognition, the described system uses an original sliding window approach based on vertical projection histogram analysis of the word and extracts a new pertinent set of statistical and structural features from the word image. Several experiments have been performed using the IFN/ENIT benchmark database and the best recognition performances achieved by our system outperform those reported recently on the same database.

Robust Remote Homology Detection by Feature Based Profile Hidden Markov Models

The detection of remote homologies is of major importance for molecular biology applications like drug discovery. The problem is still very challenging even for state-of-the-art probabilistic models of protein families, namely Profile HMMs. In order to improve remote homology detection we propose feature based semi-continuous Profile HMMs. Based on a richer sequence representation consisting of features which capture the biochemical properties of residues in their local context, family specific semi-continuous models are estimated completely data-driven. Additionally, for substantially reducing the number of false predictions an explicit rejection model is estimated. Both the family specific semi-continuous Profile HMM and the non-target model are competitively evaluated. In the experimental evaluation of superfamily based screening of the SCOP database we demonstrate that semi-continuous Profile HMMs significantly outperform their discrete counterparts. Using the rejection model the number of false positive predictions could be reduced substantially which is an important prerequisite for target identification applications.

A likelihood measure based on projection-based group delay scheme for Mandarin speech recognition in noise

This paper investigates a projection-based group delay scheme (PGDS) likelihood measure that significantly reduces noise contamination in speech recognition. Because the norm of the cepstral/GDS vector will be shrinked when the speech signals are corrupted by additive noise, the HMM parameters, namely, the mean vector and the covariance matrix, need to be furthermore modified. In this paper, the mean vector compensation, a covariance matrix adaptation function and state duration based upon the projection-based group delay scheme were incorporated with a semi-continuous HMM to improve the recognition rate in noisy environments. The proposed approach compensates the mean vector using a projection-based scale factor and the mean compensation bias, and fits the covariance matrix using a variance adaptive function. The bias and variance adaptive functions estimated from the training and/or testing data were used to balance the mismatch between different environments. Lastly, a state duration method was utilized to deal with the problem that the additive noise segments the error path in Viterbi decoding. Experiments declare that the PGDS presented herein can remarkably elevate the recognition performance in noisy environments.

Discriminative feature weighting for HMM-based continuous speech recognizers

The Discriminative Feature Extraction (DFE) method provides an appropriate formalism for the design of the front-end feature extraction module in pattern classification systems. In the recent years, this formalism has been successfully applied to different speech recognition problems, like classification of vowels, classification of phonemes or isolated word recognition. The DFE formalism can be applied to weight the contribution of the components in the feature vector. This variant of DFE, that we call Discriminative Feature Weighting (DFW), improves the pattern classification systems by enhancing those components more relevant for the discrimination among the different classes. This paper is dedicated to the application of the DFW formalism to Continuous Speech Recognizers (CSR) based on Hidden Markov Models (HMMs). Two different types of HMM-based speech recognizers are considered: recognizers based on Discrete-HMMs (DHMMs) (for which the acoustic evaluation is based on an Euclidean distance measure) and Semi-Continuous-HMMs (SCHMMs) (for which the acoustic evaluation is performed making use of a mixture of multivariated Gaussians). We report how the components can be weighted and how the weights can be discriminatively trained and applied to the speech recognizers. We present recognition results for several continuous speech recognition tasks. The experimental results show the utility of DFW for HMM-based continuous speech recognizers.

Application of the variance compensation likelihood measure for robust hidden Markov model in noise

This letter presents covariance matrix adaptation based upon the incorporation of the projection-based group delay scheme (PGDS) with a semi-continuous hidden Markov model (HMM) for the minimization of undesirable effects in noisy environments. The proposed approach fits the covariance matrix using a variance adaptive function to balance the mismatch between different environments.

Improving continuous speech recognition in Spanish by phone-class semicontinuous HMMs with pausing and multiple pronunciations

This paper presents a comprehensive study of continuous speech recognition in Spanish. It shows the use and optimisation of several well-known techniques together with the application for the first time to Spanish of language specific knowledge to these systems, i.e. the careful selection of the phone inventory, the phone-classes used, and the selection of alternative pronunciation rules. We have developed a semicontinuous phone-class dependent contextual modelling. Using four phone-classes, we have obtained recognition error rate reductions roughly equivalent to the percentage increase of the number of parameters, compared to baseline semicontinuous contextual modelling. We also show that the use of pausing in the training system and multiple pronunciations in the vocabulary help to improve recognition rates significantly. The actual pausing of the training sentences and the application of assimilation effects improve the transcription into context-dependent units. Multiple pronunciation possibilities are generated using general rules that are easily applied to any Spanish vocabulary. With all these ideas we have reduced the recognition errors of the baseline system by more than 30% in a task parallel to DARPA-RM translated into Spanish with a vocabulary of 979 words. Our database contains four speakers with 600 training sentences and 100 testing sentences each. All experiments have been carried out with a perplexity of 979, and even slightly higher in the case of multiple pronunciations, to be able to study the acoustic modelling power of the systems with no grammar constraints.

Mean compensation based on projection-based group delay scheme for noisy speech recognition

A mean vector compensation technique based on the projection-based group delay scheme has been combined with a semi-continuous HMM to improve the recognition rate in noisy environments. The proposed approach compensates the mean vector using a projection-based scale factor and the bias estimated from the training and/or testing data to balance the mismatch between different environments. Experiments show that the significant improvement in speaker-dependent, isolated word recognition was achieved by adding the projection-based scale factor and mean vector bias.

Fast and accurate acoustic modelling with semi-continuous HMMs

In this paper the design of accurate Semi-Continuous Density Hidden Markov Models (SC-HMMs) for acoustic modelling in large vocabulary continuous speech recognition is presented. Two methods are described to improve drastically the efficiency of the observation likelihood calculations for the SC-HMMs. First, reduced SC-HMMs are created, where each state does not share all the – gaussian – probability density functions ( pdfs) but only those which are important for it. It is shown how the average number of gaussians per state can be reduced to 70 for a total set of 10 000 gaussians. Second, a novel scalar selection algorithm is presented reducing to 5% the number of gaussians which have to be calculated on the total set of 10 000, without any degradation in recognition performance. Furthermore, the concept of tied state context-dependent modelling with phonetic decision trees is adapted to SC-HMMs. In fact, a node splitting criterion appropriate for SC-HMMs is introduced: it is based on a distance measure between the mixtures of gaussian pdfs as involved in SC-HMM state modelling. This contrasts with other criteria from literature which are based on simplified pdfs to manage the algorithmic complexity. On the ARPA Resource Management task, a relative reduction in word error rate of 8% was achieved with the proposed criterion, comparing with two known criteria based on simplified pdfs.

On-line adaptation of the SCHMM parameters based on the segmental quasi-Bayes learning for speech recognition

On-line quasi-Bayes adaptation of the mixture coefficients and mean vectors in semicontinuous hidden Markov model (SCHMM) is studied. The viability of the proposed algorithm is confirmed and the related practical issues are addressed in a specific application of on-line speaker adaptation using a 26-word English alphabet vocabulary.

Discriminative codebook design using multiple vector quantization in HMM-based speech recognizers

Research on multiple vector quantization (MVQ) has shown the suitability of such a technique for speech recognition. Basically, MVQ proposes the use of one separate VQ codebook for each recognition unit. Thus, a MVQHMM model is composed of a VQ codebook and a discrete HMM model. This technique allows the incorporation in the recognition dynamics of the input sequence information wasted by discrete HMM models in the VQ process. The use of distinct codebooks also allows one to train them in a discriminative manner. We propose a new VQ codebook design method for MVQ-based systems, obtained from a modified maximum mutual information estimation. This method provides meaningful error reductions and is performed independently from the estimation of the discrete HMM part of the MVQ model. The results show that the proposed discriminative design turns the MVQHMM technique into a powerful acoustic modeling tool in comparison with other classical methods such as discrete or semicontinuous HMMs.

Word recognition by a parallel-branch subunit model based on misrecognized data in semi-continuous HMM

We propose a parallel-branch subunit model in semicontinuous HMM for improved initialization and training in word recognition. We obtain the model by adding a new subunit branch based on misrecognized data in training to the previous parallel branches for that subunit. Simulation results show that this proposed model is efficient and gives good recognition performance in word recognition.