Hidden Markov Model Estimation Research Articles

Order selection for heterogeneous semiparametric hidden Markov models.

Hidden Markov models (HMMs), which can characterize dynamic heterogeneity, are valuable tools for analyzing longitudinal data. The order of HMMs (ie, the number of hidden states) is typically assumed to be known or predetermined by some model selection criterion in conventional analysis. As prior information about the order frequently lacks, pairwise comparisons under criterion-based methods become computationally expensive with the model space growing. A few studies have conducted order selection and parameter estimation simultaneously, but they only considered homogeneous parametric instances. This study proposes a Bayesian double penalization (BDP) procedure for simultaneous order selection and parameter estimation of heterogeneous semiparametric HMMs. To overcome the difficulties in updating the order, we create a brand-new Markov chain Monte Carlo algorithm coupled with an effective adjust-bound reversible jump strategy. Simulation results reveal that the proposed BDP procedure performs well in estimation and works noticeably better than the conventional criterion-based approaches. Application of the suggested method to the Alzheimer's Disease Neuroimaging Initiative research further supports its usefulness.

A comparison between marginal likelihood and data augmented MCMC algorithms for Gaussian hidden Markov models

We provide a comparison between marginal likelihood and data augmented Markov chain Monte Carlo (MCMC) algorithms for Bayesian estimation of hidden Markov models. In particular, we focus on a specification of these models based on a multivariate Gaussian distribution for the response variables. We evaluate the performance of both methods on the basis of simulated samples and an empirical application, where we compare these approaches in terms of standard measures of estimation and time efficiency. The results show that the augmented algorithm is preferable, obtaining optimal convergence properties and producing larger values of the effective sample size. Instead of using variable dimensional techniques for model selection, we suggest a method based on the so-called parallel sampling for this aim.

Parallel EM algorithm on Hadoop for large-scale hidden Markov model parameter estimation

This paper aims to propose a parallel EM algorithm based on Hadoop for parameter estimation of large-scale hidden Markov models (HMM). HMM is a commonly used statistical model. However, since the parameter estimation of HMM involves the storage and processing of large-scale data sets, traditional serial algorithms have certain limitations in efficiency. This paper introduces the Hadoop parallel computing framework, divides the task into multiple subtasks through the MapReduce programming model and assigns them to different machines for parallel computing, which improves the efficiency and scalability of parameter estimation. The results show that using parallel EM algorithm for large-scale hidden Markov model parameter estimation on Hadoop is feasible and effective.

Parametric estimation of hidden Markov models by least squares type estimation and deconvolution

This paper develops a simple and computationally efficient parametric approach to the estimation of general hidden Markov models (HMMs). For non-Gaussian HMMs, the computation of the maximum likelihood estimator (MLE) involves a high-dimensional integral that has no analytical solution and can be difficult to approach accurately. We develop a new alternative method based on the theory of estimating functions and a deconvolution strategy. Our procedure requires the same assumptions as the MLE and deconvolution estimators. We provide theoretical guarantees about the performance of the resulting estimator; its consistency and asymptotic normality are established. This leads to the construction of confidence intervals. Monte Carlo experiments are investigated and compared with the MLE. Finally, we illustrate our approach using real data for ex-ante interest rate forecasts.

Hidden mover-stayer model for disease progression accounting for misclassified and partially observed diagnostic tests: Application to the natural history of human papillomavirus and cervical precancer.

Hidden Markov models (HMMs) have been proposed to model the natural history of diseases while accounting for misclassification in state identification. We introduce a discrete time HMM for human papillomavirus (HPV) and cervical precancer/cancer where the hidden and observed state spaces are defined by all possible combinations of HPV, cytology, and colposcopy results. Because the population of women undergoing cervical cancer screening is heterogeneous with respect to sexual behavior, and therefore risk of HPV acquisition and subsequent precancers, we use a mover-stayer mixture model that assumes a proportion of the population will stay in the healthy state and are not subject to disease progression. As each state is a combination of three distinct tests that characterize the cervix, partially observed data arise when at least one but not every test is observed. The standard forward-backward algorithm, used for evaluating the E-step within the E-M algorithm for maximum-likelihood estimation of HMMs, cannot incorporate time points with partially observed data. We propose a new forward-backward algorithm that considers all possible fully observed states that could have occurred across a participant's follow-up visits. We apply our method to data from a large management trial for women with low-grade cervical abnormalities. Our simulation study found that our method has relatively little bias and out preforms simpler methods that resulted in larger bias.

Efficient recursive distributed state estimation of hidden Markov models over unreliable networks

We consider a scenario in which a process of interest, evolving within an environment occupied by several agents, is well-described probablistically via a Markov model. The agents each have local views and observe only some limited partial aspects of the world, but their overall task is to fuse their data to construct an integrated, global portrayal. The problem, however, is that their communications are unreliable: network links may fail, packets can be dropped, and generally the network might be partitioned for protracted periods. The fundamental problem then becomes one of consistency as agents in different parts of the network gain new information from their observations but can only share this with those with whom they are able to communicate. As the communication network changes, different views may be at odds; the challenge is to reconcile these differences. The issue is that correlations must be accounted for, lest some sensor data be double counted, inducing overconfidence or bias. As a means to address these problems, a new recursive consensus filter for distributed state estimation on hidden Markov models is presented. It is shown to be well-suited to multi-agent settings and associated applications since the algorithm is scalable, robust to network failure, capable of handling non-Gaussian transition and observation models, and is, therefore, quite general. Crucially, no global knowledge of the communication network is ever assumed. We have dubbed the algorithm a Hybrid method because two existing pieces are used in concert: the first, iterative conservative fusion is used to reach consensus over potentially correlated priors, while consensus over likelihoods, the second, is handled using weights based on a Metropolis Hastings Markov chain. To attain a detailed understanding of the theoretical upper limit for estimator performance modulo imperfect communication, we introduce an idealized distributed estimator. It is shown that under certain general conditions, the proposed Hybrid method converges exponentially to the ideal distributed estimator, despite the latter being purely conceptual and unrealizable in practice. An extensive evaluation of the Hybrid method, through a series of simulated experiments, shows that its performance surpasses competing algorithms.

A novel algorithm for parameter estimation of Hidden Markov Model inspired by Ant Colony Optimization

HMM is a powerful method to model data in various fields. Estimation of Hidden Markov Model parameters is an NP-Hard problem. We propose a heuristic algorithm called “AntMarkov” to improve the efficiency of estimating HMM parameters. We compared our method with four algorithms. The comparison was conducted on 5 different simulated datasets with different features. For further evaluation, we analyzed the performance of algorithms on the prediction of protein secondary structures problem. The results demonstrate that our algorithm obtains better results with respect to the results of the other algorithms in terms of time efficiency and the amount of similarity of estimated parameters to the original parameters and log-likelihood.The source code of our algorithm is available in https://github.com/emdadi/HMMPE.

Source Estimation in Time Series and the Surprising Resilience of HMMs

Suppose that we are given a time series where consecutive samples are believed to come from a probabilistic source, that the source changes from time to time and that the total number of sources is fixed. Our objective is to estimate the distributions of the sources. A standard approach to this problem is to model the data as a hidden Markov model (HMM). However, since the data often lacks the Markov or the stationarity properties of an HMM, one can ask whether this approach is still suitable or perhaps another approach is required. In this paper, we show that a maximum likelihood HMM estimator can be used to approximate the source distributions in a much larger class of models than HMMs. Specifically, we propose a natural and fairly general non-stationary model of the data, where the only restriction is that the sources do not change too often. Our main result shows that for this model, a maximum-likelihood HMM estimator produces the correct second moment of the data, and the results can be extended to higher moments.

Hidden Markov model analysis of extreme behaviors of foreign exchange rates

We examine the extreme behaviors at both the lower 5% and 1% quantile levels of the three exchange rates series (Japanese yen, Korean won, and New Taiwan dollar) against the US dollar between 2002 and 2017. We employ two econometric tools for specific purposes: (1) to determine the bootstrap goodness of fit test for the Generalized Pareto distribution (GPD) of the tail behaviors proposed by Villaseñor-Alva and González-Estrada (2009); and (2) to identify, estimate, and test the hidden Markov model (HMM). The testing outcomes mostly reject the GPD assumption for this study. HMM estimation outcomes provide the detailed pictures. They evidence the multiple structural breaks in the returns and the gaps between the qualified extremes. The respective properties of the exchange markets are revealed in the comparisons of the estimation outcomes in terms of transition matrix, response parameter, and bivariate analysis. In general, Japanese yen is evaluated as a resilient safe haven currency. Korean won exhibits longer distress duration. New Taiwan dollar occurs with heavier losses with longer gaps.

Hidden Markov Models revealing the household thermal profiling from smart meter data

This work describes a methodology based on Hidden Markov Models (HMMs) that are applied for revealing household thermal load profiles which are not available to direct observation. This research is motivated by the necessity of reducing the energy consumption for cooling and heating in residential buildings. Our methodology uses data that is becoming readily available at households – hourly energy consumption records collected from smart electricity meters, as well as hourly outdoor air temperature records. The heat transfer regime, namely the states corresponding to lower or higher building hourly thermal loads related to the outdoor air temperatures, will be considered as the underlying mechanism affecting the generation of observations. We aggregate the observed data to obtain a certain number of clusters. The problem of HMM estimation is addressed and the subsequent HMMs are compared on the basis of information criteria, like Akaike and Bayesian Information Criteria. Our goal is to reveal the dynamic of building thermal load (heating/cooling) under the uncertainties induced by the residents’ behavior. Consequently, we present examples of thermal load profiles generated using our best HMM on a testing facility located in the Polytechnic University of Bucharest campus, namely the UPB's passive building house.

Event-Based State Estimation of Hidden Markov Models Through a Gilbert–Elliott Channel

In this note, the problem of event-based state estimation for a finite-state hidden Markov model under a generic stochastic event-triggering condition and an unreliable communication channel is investigated. The effect of packet dropout is characterized with a Gilbert–Elliott process. Utilizing the change of probability measure approach, the packet dropout model and the event-triggered measurement information available to the estimator, analytical expressions for the conditional probability distributions of the states are obtained, based on which the optimal event-based state estimates can be further calculated, together with a closed-form expression of the average sensor-to-estimator communication rate. The effectiveness of the proposed results is illustrated by an application to a wireless automated machine health monitoring problem.

Single microphone speech separation by diffusion-based HMM estimation

We present a novel non-iterative and rigorously motivated approach for estimating hidden Markov models (HMMs) and factorial hidden Markov models (FHMMs) of high-dimensional signals. Our approach utilizes the asymptotic properties of a spectral, graph-based approach for dimensionality reduction and manifold learning, namely the diffusion framework. We exemplify our approach by applying it to the problem of single microphone speech separation, where the log-spectra of two unmixed speakers are modeled as HMMs, while their mixture is modeled as an FHMM. We derive two diffusion-based FHMM estimation schemes. One of which is experimentally shown to provide separation results that compare with contemporary speech separation approaches based on HMM. The second scheme allows a reduced computational burden.

Theory of segmented particle filters

AbstractWe study the asymptotic behavior of a new particle filter approach for the estimation of hidden Markov models. In particular, we develop an algorithm where the latent-state sequence is segmented into multiple shorter portions, with an estimation technique based upon a separate particle filter in each portion. The partitioning facilitates the use of parallel processing, which reduces the wall-clock computational time. Based upon this approach, we introduce new estimators of the latent states and likelihood which have similar or better variance properties compared to estimators derived from standard particle filters. We show that the likelihood function estimator is unbiased, and show asymptotic normality of the underlying estimators.

Hybrid Metaheuristic Approaches to the Expectation Maximization for Estimation of the Hidden Markov Model for Signal Modeling

The expectation maximization (EM) is the standard training algorithm for hidden Markov model (HMM). However, EM faces a local convergence problem in HMM estimation. This paper attempts to overcome this problem of EM and proposes hybrid metaheuristic approaches to EM for HMM. In our earlier research, a hybrid of a constraint-based evolutionary learning approach to EM (CEL-EM) improved HMM estimation. In this paper, we propose a hybrid simulated annealing stochastic version of EM (SASEM) that combines simulated annealing (SA) with EM. The novelty of our approach is that we develop a mathematical reformulation of HMM estimation by introducing a stochastic step between the EM steps and combine SA with EM to provide better control over the acceptance of stochastic and EM steps for better HMM estimation. We also extend our earlier work and propose a second hybrid which is a combination of an EA and the proposed SASEM, (EA-SASEM). The proposed EA-SASEM uses the best constraint-based EA strategies from CEL-EM and stochastic reformulation of HMM. The complementary properties of EA and SA and stochastic reformulation of HMM of SASEM provide EA-SASEM with sufficient potential to find better estimation for HMM. To the best of our knowledge, this type of hybridization and mathematical reformulation have not been explored in the context of EM and HMM training. The proposed approaches have been evaluated through comprehensive experiments to justify their effectiveness in signal modeling using the speech corpus: TIMIT. Experimental results show that proposed approaches obtain higher recognition accuracies than the EM algorithm and CEL-EM as well.

Comment on the paper “On the memory complexity of the forward–backward algorithm,” by Khreich W., Granger E., Miri A., Sabourin, R.

We illustrate the Efficient Forward Filtering Backward Smoothing (EFFBS) algorithm proposed by Khreich et al. (2010) for estimation of hidden Markov models. The algorithm is aimed at reducing the amount of memory required by the Baum-Welch recursions, while having the same complexity in terms of number of operations. In implementing the EFFBS algorithm we found a numerical problem that limits its applicability. We discuss this problem in detail, providing some possible explanations of the causes of the error, together with two illustrative examples.

A Hybrid HMM/SVM Classifier for Wavelet Front End Robust Automatic Speech Recognition

Noisy ambient conditions pose a challenge to speech recognition, increasing the acoustic confusability, thereby looking for powerful acoustic models to improve the generalization ability of the machine learning and improve the recognition accuracy. This paper discusses a hybrid classifier that harness the power of hidden markov models (HMM) and the discriminative support vector machines (SVM) applied to a wavelet front end based automatic speech recognition (ASR) system. The experiments are performed on speaker independent TIMIT database which are trained in a clean environment and later tested in the presence of additive white gaussian noise (AWGN) for various SNR levels using the HTK toolkit, SVMLib and SVMLight software tool. Experiments indicate that for large vocabulary the classification power of SVMs and the elegant iterative training algorithms for the estimation of HMMs together as a hybrid classifier with the wavelet front end performs better than the conventional classifiers.

An HMM-Based Algorithm for Content Ranking and Coherence-Feature Extraction

In this paper, we propose an algorithm called coherence hidden Markov model (HMM) to extract coherence features and rank content. Coherence HMM is a variant of HMM and is used to model the stochastic process of essay writing and identify topics as hidden states, given sequenced clauses as observations. This study uses probabilistic latent semantic analysis for parameter estimation of coherence HMM. In coherence-feature extraction, support vector regression (SVR) with surface features and coherence features is used for essay grading. The experimental results indicate that SVR can benefit from coherence features. The adjacent agreement rate and the exact agreement rate are 95.24% and 59.80%, respectively. Moreover, this study submits high-scoring essays to the same experiment and finds that the adjacent agreement rate and exact agreement rate are 98.33% and 64.50%, respectively. In content ranking, we design and implement an intelligent assisted blog writing system based on the coherence-HMM ranking model. Several corpora are employed to help users efficiently compose blog articles. When users finish composing a clause or sentence, the system provides candidate texts for their reference based on current clause or sentence content. The experimental results demonstrate that all participants can benefit from the system and save considerable time on writing articles.

Dynamic Quantization and Power Allocation for Multisensor Estimation of Hidden Markov Models

This paper investigates an optimal quantizer design problem for multisensor estimation of a hidden Markov model (HMMs) whose description depends on unknown parameters. The sensor measurements are simply binary quantized and transmitted to a remote fusion center over noisy flat fading wireless channels under an average sum transmit power constraint. The objective is to determine a set of optimal quantization thresholds and sensor transmit powers, called an optimal policy, which minimizes the long run average of a weighted combination of the expected state estimation error and sum transmit power. We analyze the problem by formulating an adaptive Markov decision process (MDP) problem. In this framework, adaptive optimal control policies are obtained using a nonstationary value iteration (NVI) scheme and are termed as NVI-adaptive policies. These NVI-adaptive policies are adapted to the HMM parameter estimates obtained via a strongly consistent maximum likelihood estimator. In particular, HMM parameter estimation is performed by a recursive expectation-maximization (EM) algorithm which computes estimates of the HMM parameters by maximizing a relative entropy information measure using the received quantized observations and the trajectory of the MDP. Under some regularity assumptions on the observation probability distributions and a geometric ergodicity condition on an extended Markov chain, the maximum-likelihood estimator is shown to be strongly consistent. It is shown that the NVI-adaptive policy based on this sequence of strongly consistent HMM parameter estimates is (asymptotically, under appropriate assumptions) average-optimal. Essentially, it minimizes the long run average cost of the weighted combination of the expected state estimation error and sum transmit power across the sensors for the HMM with true parameters in a time-asymptotic sense. The advantage of this scheme is that the policies are obtained recursively without the need to solve the Bellman equation at each time step, which can be computationally prohibitive. As is usual with value iteration schemes, practical implementation of the NVI-adaptive policy requires discretization of the state and action space, which results in some loss of optimality. Nevertheless, numerical results illustrate the asymptotic convergence properties of the parameter estimates and the asymptotically close to optimal performance of the adaptive MDP algorithm compared to the performance of an MDP based dynamic quantization and power allocation algorithm designed with perfect knowledge of the true parameters.

Trust Region-Based Optimization for Maximum Mutual Information Estimation of HMMs in Speech Recognition

In this paper, we have proposed two novel optimization methods for discriminative training (DT) of hidden Markov models (HMMs) in speech recognition based on an efficient global optimization algorithm used to solve the so-called trust region (TR) problem, where a quadratic function is minimized under a spherical constraint. In the first method, maximum mutual information estimation (MMIE) of Gaussian mixture HMMs is formulated as a standard TR problem so that the efficient global optimization method can be used in each iteration to maximize the auxiliary function of discriminative training for speech recognition. In the second method, we propose to construct a new auxiliary function for DT of HMMs by adding a quadratic penalty term. The new auxiliary function is constructed to serve as first-order approximation as well as lower bound of the original discriminative objective function within a locality constraint. Due to the lower-bound property, the found optimal point of the new auxiliary function is guaranteed to improve the original discriminative objective function until it converges to a local optimum or stationary point of the objective function. Both TR-based optimization methods have been investigated on two standard large-vocabulary continuous speech recognition tasks, using the WSJ0 and Switchboard databases. Experimental results have shown that the proposed TR methods outperform the conventional EBW method in terms of convergence behavior as well as recognition performance.

Large-Margin Estimation of Hidden Markov Models With Second-Order Cone Programming for Speech Recognition

Large-margin estimation (LME) holds a property of good generalization on unseen test data. In our previous work, LME of HMMs has been successfully applied to some small-scale speech recognition tasks, using the SDP (semi-definite programming) technique. In this paper, we further extend the previous work by exploring a more efficient convex optimization method with the technique of second-order cone programming (SOCP). More specifically, we have studied and proposed several SOCP relaxation techniques to convert LME of HMMs in speech recognition into a standard SOCP problem so that LME can be solved with more efficient SOCP methods. The formulation is general enough to deal with various types of competing hypothesis space, such as N-best lists and word graphs. The proposed LME/SOCP approaches have been evaluated on two standard speech recognition tasks. The experimental results on the TIDIGITS task show that the SOCP method significantly outperforms the gradient descent method, and achieve comparable performance with SDP, but with 20-200 times faster speed, requiring less memory and computing resources. Furthermore, the proposed LME/SOCP method has also been successfully applied to a large vocabulary task using the Wall Street Journals (WSJ0) database. The WSJ-5k recognition results show that the proposed method yields better performance than the conventional approaches including maximum-likelihood estimation (MLE), maximum mutual information estimation (MMIE), and more recent boosted MMIE methods.