Conventional Hidden Markov Model Research Articles

A New Prosody-Assisted Mandarin ASR System

This paper presents a new prosody-assisted automatic speech recognition (ASR) system for Mandarin speech. It differs from the conventional approach of using simple prosodic cues on employing a sophisticated prosody modeling approach based on a four-layer prosody-hierarchy structure to automatically generate 12 prosodic models from a large unlabeled speech database by the joint prosody labeling and modeling (PLM) algorithm proposed previously. By incorporating these 12 prosodic models into a two-stage ASR system to rescore the word lattice generated in the first stage by the conventional hidden Markov model (HMM) recognizer, we can obtain a better recognized word string. Besides, some other information can also be decoded, including part of speech (POS), punctuation mark (PM), and two types of prosodic tags which can be used to construct the prosody-hierarchy structure of the testing speech. Experimental results on the TCC300 database, which consists of long paragraphic utterances, showed that the proposed system significantly outperformed the baseline scheme using an HMM recognizer with a factored language model which models word, POS, and PM. Performances of 20.7%, 14.4%, and 9.6% in word, character, and base-syllable error rates were obtained. They corresponded to 3.7%, 3.7%, and 2.4% absolute (or 15.2%, 20.4%, and 20% relative) error reductions. By an error analysis, we found that many word segmentation errors and tone recognition errors were corrected.

Dempster–Shafer fusion of multisensor signals in nonstationary Markovian context

The latest developments in Markov models’ theory and their corresponding computational techniques have opened new rooms for image and signal modeling. In particular, the use of Dempster–Shafer theory of evidence within Markov models has brought some keys to several challenging difficulties that the conventional hidden Markov models cannot handle. These difficulties are concerned mainly with two situations: multisensor data, where the use of the Dempster–Shafer fusion is unworkable; and nonstationary data, due to the mismatch between the estimated stationary model and the actual data. For each of the two situations, the Dempster–Shafer combination rule has been applied, thanks to the triplet Markov models’ formalism, to overcome the drawbacks of the standard Bayesian models. However, so far, both situations have not been considered in the same time. In this article, we propose an evidential Markov chain that uses the Dempster–Shafer combination rule to bring the effect of contextual information into segmentation of multisensor nonstationary data. We also provide the Expectation–Maximization parameters’ estimation and the maximum posterior marginal’s restoration procedures. To validate the proposed model, experiments are conducted on some synthetic multisensor data and noised images. The obtained segmentation results are then compared to those obtained with conventional approaches to bring out the efficiency of the present model.

Semi-supervised context adaptation: case study of audience excitement recognition

To recognise just the same human reaction (for example, a strong excitement) in different contexts, customary behaviours in these contexts have to be taken into account; e.g. a happy sport audience may be cheering for long time, while a happy theatrical audience may produce only short bursts of laughter in order to not interrupt the performance. Tailoring recognition algorithms to contexts can be achieved by building either a context-specific or a generic system. The former is individually trained for each context to recognise sets of characteristic responses, whereas the latter—in contrast to the context-specific one—adapts to the context via significantly more lightweight modification of parameters. This paper follows the latter way and proposes a simple modification of a hidden Markov model (HMM) classifier that enables end users to adapt the generic system to a context or a personal perception of an annotator by labelling a fairly small number of data samples of each context. For better adaptability to the limited number of the user’s annotations, the proposed semi-supervised HMM classifier employs the maximum posterior marginal, rather than the more conventional maximum a posteriori decision rule. The proposed user- and context-adaptable semi-supervised HMM classifier was tested on recognising excitement of a show audience in three contexts (a concert hall, a circus, and a sport event), differing in how the excitement is expressed. In our experiments the proposed classifier recognised reactions of a non-neutral audience with 10% higher accuracy than the conventional HMM and support vector machine based classifiers.

Acoustic modeling problem for automatic speech recognition system: conventional methods (Part I)

In automatic speech recognition (ASR) systems, the speech signal is captured and parameterized at front end and evaluated at back end using the statistical framework of hidden Markov model (HMM). The performance of these systems depend critically on both the type of models used and the methods adopted for signal analysis. Researchers have proposed a variety of modifications and extensions for HMM based acoustic models to overcome their limitations. In this review, we summarize most of the research work related to HMM-ASR which has been carried out during the last three decades. We present all these approaches under three categories, namely conventional methods, refinements and advancements of HMM. The review is presented in two parts (papers): (i) An overview of conventional methods for acoustic phonetic modeling, (ii) Refinements and advancements of acoustic models. Part I explores the architecture and working of the standard HMM with its limitations. It also covers different modeling units, language models and decoders. Part II presents a review on the advances and refinements of the conventional HMM techniques along with the current challenges and performance issues related to ASR.

A New Evidence Model for Missing Data Speech Recognition With Applications in Reverberant Multi-Source Environments

Conventional hidden Markov model (HMM) decoders often experience severe performance degradations in practice due to their inability to cope with uncertain data in time-varying environments. In order to address this issue, we propose the bounded-Gauss-Uniform mixture probability density function (pdf) as a new class of evidence model for missing data speech recognition. Exemplary for a hands-free speech recognition scenario, we illustrate how the parameters of the new mixture pdf can be estimated with the help of a multi-channel source separation front-end. In comparison with other models the new evidence pdf retains a fuller description of the available data and provides a more effective link between source separation and recognition. The superiority of the bounded-Gauss-Uniform mixture pdf over conventional approaches is demonstrated for a connected digits recognition task under varying test conditions.

Unsupervised Segmentation of Human Motion Data Using a Sticky Hierarchical Dirichlet Process-Hidden Markov Model and Minimal Description Length-Based Chunking Method for Imitation Learning

We propose an unsupervised motion segmentation method for enabling a robot to imitate and perform various unit motions by observing unsegmented human motion. Natural unsegmented human motion data contain various types of unit motions, such as 'waving good-bye', 'walking' and 'throwing a ball'. A robot has to segment the data and extract unit motions from the data to imitate the motions. In previous work, an ergodic hidden Markov model (HMM) was used to model unsegmented human motion. However, there are two main problems with the classical use of this model. (i) Setting an appropriate number of hidden states is difficult because how complex the motions contained in the learning data are and how many there are unknown. (ii) We did not have an effective chunking method that could chunk elemental motions into meaningful unit motions without being captured by local minima. To overcome these problems, we developed an unsupervised motion segmentation method for imitation learning using a sticky hierarchical Dirichlet process (HDP)-HMM, a nonparametric Bayesian model, and an unsupervised chunking method based on a Gibbs sampler and the minimal description length (MDL) principle of imitation learning of unsegmented human motion. We developed this chunking method to work with the sticky HDP-HMM and extract unit human motions. We conducted several experiments to evaluate this method. The proposed method could extract unit motions from unsegmented human motion data. The sticky HDP-HMM can be used to model unsegmented human motion more accurately than with a conventional HMM and simultaneously estimate the number of hidden states. We also evaluated the dependency of the HDP-HMM on the hyperparameters of the model.

Malay Language Speech Recogniser with Hybrid Hidden Markov Model and Artificial Neural Network(HMM/ANN)

There are many artificial intelligence approaches used in the development of Automatic Speech Recognition (ASR), and hybrid approach is one of them. The common hybrid method in speech recognition is the combination of Hidden Markov Model (HMM) and Artificial Neural Network (ANN). The hybrid HMM/ANN is able to combine the strength of HMM in sequential modeling structure and ANN in pattern classification. Thus, this paper proposed a speaker independent and continuous Malay language speech recogniser by using the hybrid HMM/ANN method. In addition to that, this paper presents a study on Standard Malay's phonetic and phonology to help in the recognition of Malay words. The CSLU toolkit is utilized for building the recogniser, and the experimental results showed that the proposed HMM/ANN model outperformed the conventional HMM model. The performances of the recognisers are measured in term of word accuracy and sentence accuracy.

A Sequential Pattern Classifier Based on Hidden Markov Kernel Machine and Its Application to Phoneme Classification

This paper describes a novel classifier for sequential data based on nonlinear classification derived from kernel methods. In the proposed method, kernel methods are used for enhancing the emission probability density functions (pdfs) of hidden Markov models (HMMs). Because the emission pdfs enhanced by kernel methods have sufficient nonlinear classification performance, mixture models such as Gaussian mixture models (GMMs), which might cause problems of overfitting and local optima, are not necessary in the proposed method. Unlike the methods used in earlier studies on sequential pattern classification using kernel methods, our method can be regarded as an extension of conventional HMMs, and therefore, it can completely model the transition of hidden states with the observed vectors. Therefore, our method can be applied to many applications developed with conventional HMMs, especially for speech recognition. In this paper, we carried out an isolated phoneme classification as a preliminary experiment in order to evaluate the efficiency of the proposed sequential pattern classifier. We confirmed that the proposed method achieved steady improvements as compared to conventional HMMs with Gaussian-mixture emission pdfs trained by the maximum likelihood and the maximum mutual information procedures.

Optimisation of HMM Topologies Enhances DNA and Protein Sequence Modelling

Hidden Markov models (HMMs) play a major role in applications to unravel biomolecular functionality. Though HMMs are technically mature and widely applied in computational biology, there is a potential of methodical optimisation concerning its modelling of biological data sources with varying sequence lengths. Single building blocks of these models, the states, are associated with a certain holding time, being the link to the length distribution of represented sequence motifs. An adaptation of regular HMM topologies to bell-shaped sequence lengths is achieved by a serial chain-linking of hidden states, while residing in the class of conventional hidden Markov models. The factor of the repetition of states (r) and the parameter for state-specific duration of stay (p) are determined by fitting the distribution of sequence lengths with the method of moments (MM) and maximum likelihood (ML). Performance evaluations of differently adjusted HMM topologies underline the impact of an optimisation for HMMs based on sequence lengths. Secondary structure prediction on internal transcribed spacer 2 sequences demonstrates exemplarily the general impact of topological optimisations. In summary, we propose a general methodology to improve the modelling behaviour of HMMs by topological optimisation with ML and a fast and easily implementable moment estimator.

Discriminative transformation for speech features based on genetic algorithm and HMM likelihoods

Hidden Markov Model (HMM) is a well-known classification approach which its parameters are conventionally learned using maximum likelihood (ML) criterion based on expectation maximization algorithm. Improving of parameter learning beyond ML has been performed based on the concept of discrimination among classes in contrast to maximizing likelihood of each individual class. In this paper, we propose a discriminative feature transformation method based on genetic algorithm, to increase Hidden Markov Model likelihoods in its training phase for a better class discrimination. The method is evaluated for phoneme recognition on clean and noisy TIMIT. Experimental results demonstrate that the proposed transformation method results in higher phone recognition rate than well-known feature transformations methods and conventional HMM learning algorithm based on ML criterion.

Video tracking and behaviour segmentation of laboratory rodents

In this paper a system for laboratory rodent video tracking and behavior segmentation is proposed. A new real-time mouse pose estimation method is proposed based on semi-automatically generated animal shape model. Behavior segmentation into separate behavior acts is considered as a signal segmentation problem using hidden Markov models (HMM). Conventional first order HMM supposes a geometric prior distribution on segment’s length, which is inadequate for behavior segmentation. We propose a modification of conventional first order HMM that allows any prior distribution on segment’s length. Experiments show that the developed approach can lead to more adequate results comparing to conventional HMM.

Fuzzy Qualitative Human Motion Analysis

This paper proposes a fuzzy qualitative approach to vision-based human motion analysis with an emphasis on human motion recognition. It achieves feasible computational cost for human motion recognition by combining fuzzy qualitative robot kinematics with human motion tracking and recognition algorithms. First, a data-quantization process is proposed to relax the computational complexity suffered from visual tracking algorithms. Second, a novel human motion representation, i.e., qualitative normalized template, is developed in terms of the fuzzy qualitative robot kinematics framework to effectively represent human motion. The human skeleton is modeled as a complex kinematic chain, and its motion is represented by a series of such models in terms of time. Finally, experiment results are provided to demonstrate the effectiveness of the proposed method. An empirical comparison with conventional hidden Markov model (HMM) and fuzzy HMM (FHMM) shows that the proposed approach consistently outperforms both HMMs in human motion recognition.

Efficient backward decoding of high-order hidden Markov models

The forward–backward search (FBS) algorithm [S. Austin, R. Schwartz, P. Placeway, The forward–backward search algorithm, in: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 1991, pp. 697–700] has resulted in increases in speed of up to 40 in expensive time-synchronous beam searches in hidden Markov model (HMM) based speech recognition [R. Schwartz, S. Austin, Efficient, high-performance algorithms for N-best search, in: Proceedings of the Workshop on Speech and Natural Language, 1990, pp. 6–11; L. Nguyen, R. Schwartz, F. Kubala, P. Placeway, Search algorithms for software-only real-time recognition with very large vocabularies, in: Proceedings of the Workshop on Human Language Technology, 1993, pp. 91–95; A. Sixtus, S. Ortmanns, High-quality word graphs using forward–backward pruning, in: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 1999, pp. 593–596]. This is typically achieved by using a simplified forward search to decrease computation in the following detailed backward search. FBS implicitly assumes that forward and backward searches of HMMs are computationally equivalent. In this paper we present experimental results, obtained on the CallFriend database, that show that this assumption is incorrect for conventional high-order HMMs. Therefore, any improvement in computational efficiency that is gained by using conventional low-order HMMs in the simplified backward search of FBS is lost. This problem is solved by presenting a new definition of HMMs termed a right-context HMM, which is equivalent to conventional HMMs. We show that the computational expense of backward Viterbi-beam decoding right-context HMMs is similar to that of forward decoding conventional HMMs. Though not the subject of this paper, this allows us to more efficiently decode high-order HMMs, by capitalising on the improvements in computational efficiency that is obtained by using the FBS algorithm.

A simple and efficient hidden Markov model scheme for host-based anomaly intrusion detection

Extensive research activities have been observed on network-based intrusion detection systems (IDSs). However, there are always some attacks that penetrate traffic-profiling-based network IDSs. These attacks often cause very serious damages such as modifying host critical files. A host-based anomaly IDS is an effective complement to the network IDS in addressing this issue. This article proposes a simple data preprocessing approach to speed up a hidden Markov model (HMM) training for system-call-based anomaly intrusion detection. Experiments based on a public database demonstrate that this data preprocessing approach can reduce training time by up to 50 percent with unnoticeable intrusion detection performance degradation, compared to a conventional batch HMM training scheme. More than 58 percent data reduction has been observed compared to our prior incremental HMM training scheme. Although this maximum gain incurs more degradation of false alarm rate performance, the resulting performance is still reasonable.

An ICA Mixture Hidden Markov Model for Video Content Analysis

In this paper, a new theoretical framework based on hidden Markov model (HMM) and independent component analysis (ICA) mixture model is presented for content analysis of video, namely ICAMHMM. Unlike the Gaussian mixture observation model commonly used in conventional HMM applications, the observations in the new ICAMHMM are modeled as a mixture of non-Gaussian components. Each non-Gaussian component is formulated by an ICA mixture, reflecting the independence of different components across video frames. In addition, to construct a compact feature space to represent a video frame, ICA is applied on video frames and the ICA coefficients are used to form a compact 2-D feature subspace that makes the subsequent modeling computationally efficient. The model parameters can be identified using supervised learning by the training sequences. The new re-estimation learning formulae of iterative ICAMHMM parameter estimation are derived based on a maximum likelihood function. Employing the identified model, maximum likelihood algorithms are developed to detect and recognize video events. As a case study, golf video sequences are used to test the effectiveness of the proposed algorithm. Experimental results show that the presented method can effectively detect and recognize the recurrent event patterns in video data. The presented new ICAMHMM is generic and can be applied to sequential data analysis in other applications.

Automatic Detection System for Cough Sounds as a Symptom of Abnormal Health Condition

The problem of attending to the health of the aged who live alone has became an important issue in developed countries. One way of solving the problem is to check their health condition by a remote-monitoring technique and support them with well-timed treatment. The purpose of this study is to develop an automatic system that can monitor a health condition in real time using acoustical information and detect an abnormal symptom. In this study, cough sound was chosen as a representative acoustical symptom of abnormal health conditions. For the development of the system distinguishing a cough sound from other environmental sounds, a hybrid model was proposed that consists of an artificial neural network (ANN) model and a hidden Markov model (HMM). The ANN model used energy cepstral coefficients obtained by filter banks based on human auditory characteristics as input parameters representing a spectral feature of a sound signal. Subsequently, an output of this ANN model and a filtered envelope of the signal were used for making an input sequence for the HMM that deals with the temporal variation of the sound signal. Compared with the conventional HMM using Mel-frequency cepstral coefficients, the proposed hybrid model improved recognition rates on low SNR from 5 dB down to -10 dB. Finally, a preliminary prototype of the automatic detection system was simply illustrated.

Acoustic-Articulatory Modeling With the Trajectory HMM

In this letter, we introduce an hidden Markov model (HMM)-based inversion system to recovery articulatory movements from speech acoustics. Trajectory HMMs are used as generative models for modelling articulatory data. Experiments on the MOCHA-TIMIT corpus indicate that the jointly trained acoustic-articulatory models are more accurate (lower RMS error) than the separately trained ones, and that trajectory HMM training results in greater accuracy compared with conventional maximum likelihood HMM training. Moreover, the system has the ability to synthesize articulatory movements directly from a textual representation.

Comparative annotation of viral genomes with non-conserved gene structure

Detecting genes in viral genomes is a complex task. Due to the biological necessity of them being constrained in length, RNA viruses in particular tend to code in overlapping reading frames. Since one amino acid is encoded by a triplet of nucleic acids, up to three genes may be coded for simultaneously in one direction. Conventional hidden Markov model (HMM)-based gene-finding algorithms may typically find it difficult to identify multiple coding regions, since in general their topologies do not allow for the presence of overlapping or nested genes. Comparative methods have therefore been restricted to likelihood ratio tests on potential regions as to being double or single coding, using the fact that the constrictions forced upon multiple-coding nucleotides will result in atypical sequence evolution. Exploiting these same constraints, we present an HMM based gene-finding program, which allows for coding in unidirectional nested and overlapping reading frames, to annotate two homologous aligned viral genomes. Our method does not insist on conserved gene structure between the two sequences, thus making it applicable for the pairwise comparison of more distantly related sequences. We apply our method to 15 pairwise alignments of six different HIV2 genomes. Given sufficient evolutionary distance between the two sequences, we achieve sensitivity of approximately 84-89% and specificity of approximately 97-99.9%. We additionally annotate three pairwise alignments of the more distantly related HIV1 and HIV2, as well as of two different hepatitis viruses, attaining results of approximately 87% sensitivity and approximately 98.5% specificity. We subsequently incorporate prior knowledge by 'knowing' the gene structure of one sequence and annotating the other conditional on it. Boosting accuracy close to perfect we demonstrate that conservation of gene structure on top of nucleotide sequence is a valuable source of information, especially in distantly related genomes. The Java code is available from the authors.

Classification of musical patterns using variable duration hidden Markov models

This paper presents a new extension to the variable duration hidden Markov model (HMM), capable of classifying musical pattens that have been extracted from raw audio data into a set of predefined classes. Each musical pattern is converted into a sequence of music intervals by means of a fundamental frequency tracking procedure. This sequence is subsequently presented as input to a set of variable-duration HMMs. Each one of these models has been trained to recognize patterns of a corresponding predefined class. Classification is determined based on the highest recognition probability. The new type of variable-duration hidden Markov modeling proposed in this paper results in enhanced performance because 1) it deals effectively with errors that commonly originate during the feature extraction stage, and 2) it accounts for variations due to the individual expressive performance of different instrument players. To demonstrate its effectiveness, the novel classification scheme has been employed in the context of Greek traditional music, to monophonic musical patterns of a popular instrument, the Greek traditional clarinet. Although the method is also appropriate for western-style music, Greek traditional music poses extra difficulties and makes music pattern recognition a harder task. The classification results demonstrate that the new approach outperforms previous work based on conventional HMMs

A Two-Channel Training Algorithm for Hidden Markov Model and Its Application to Lip Reading

Hidden Markov model (HMM) has been a popular mathematical approach for sequence classification such as speech recognition since 1980s. In this paper, a novel two-channel training strategy is proposed for discriminative training of HMM. For the proposed training strategy, a novel separable-distance function that measures the difference between a pair of training samples is adopted as the criterion function. The symbol emission matrix of an HMM is split into two channels: a static channel to maintain the validity of the HMM and a dynamic channel that is modified to maximize the separable distance. The parameters of the two-channel HMM are estimated by iterative application of expectation-maximization (EM) operations. As an example of the application of the novel approach, a hierarchical speaker-dependent visual speech recognition system is trained using the two-channel HMMs. Results of experiments on identifying a group of confusable visemes indicate that the proposed approach is able to increase the recognition accuracy by an average of 20% compared with the conventional HMMs that are trained with the Baum-Welch estimation.