Inference In Hidden Markov Models Research Articles

Divide-and-conquer Bayesian inference in hidden Markov models

Divide-and-conquer Bayesian methods consist of three steps: dividing the data into smaller computationally manageable subsets, running a sampling algorithm in parallel on all the subsets, and combining parameter draws from all the subsets. These methods use the combined parameter draws for efficient posterior inference in massive data settings. Existing divide-and-conquer methods have a major limitation in that their first two steps assume that the observations are independent. We address this problem by developing a divide-and-conquer method for Bayesian inference in parametric hidden Markov models, where the state space is known and finite. Our main contributions are two-fold. First, we partition the data into smaller blocks of consecutive observations and modify the likelihood on every time block. For any time block, the posterior distribution computed using the modified likelihood is such that its variance has the same asymptotic order as that of the true posterior. Second, suppose the number of subsets is chosen appropriately depending on the mixing properties of the hidden Markov chain. In that case, we show that the subset posterior distributions defined using the modified likelihood are asymptotically normal as the subset sample size tends to infinity. This result facilitates using any existing combination algorithm in the third step. We show that the combined posterior distribution obtained using one such algorithm is close to the true posterior distribution in 1-Wasserstein distance under widely used regularity assumptions. Our numerical results show that the proposed method provides an accurate approximation of the true posterior distribution than its competitors in simulation studies and a real data analysis.

Full Bayesian inference in hidden Markov models of plant growth

Accurately modeling the growth process of plants in interaction with their environment is important for predicting their biophysical characteristics, referred to as phenotype prediction. Most models are described by discrete dynamic systems in general state-space representation with important domain-specific characteristics: First, plant model parameters have usually clear functional meanings and may be of genetic origins, thus necessitating a precise estimation. Second, critical growth variables, specifically biomass production and dynamic allocation to organs, are hidden variables not accessible to measure. Finally, the difficulty to assess the local plant environment may imply the introduction of process noises in models. Therefore, a precise understanding of the system’s behavior requires the joint estimation of functional parameters, hidden states, and noise parameters. In this paper we describe how a full Bayesian method of estimation can accurately estimate all these key model variables using Markov chain Monte Carlo (MCMC) techniques. In the presence of both process and observation noises, it requires to use adequate particle MCMC (PMCMC) algorithms to efficiently sample the hidden states which, consequently, allows for a precise estimation of all noise parameters involved. Thanks to the Bayesian framework, appropriate choices of prior distributions for the noise parameters have enabled analytical posterior distributions and only simple updates are required. Furthermore, this estimation strategy can be easily generalized and adapted to different types of plant growth models, such as organ-scale or compartmental, provided that they are formulated as hidden Markov models. Our estimation method improves on those classically used in plant growth modeling in several aspects: First, by building upon a general probabilistic framework the estimation results allow proper statistical analyses. It is useful in prediction, no only for uncertainty and risk analysis (e.g., for crop yield prediction) but also to analyze the results of experimental trials, for example, to compare genotypes in breeding. Moreover, the care taken in the estimation of hidden variables opens new perspectives in the understanding of inner growth processes, notably the balance and interaction between biomass production and allocation (referred to as source-sink dynamics). Applications of this estimation procedure are demonstrated on the GreenLab model for Arabidopsis thaliana and the Log-Normal Allocation and Senescence (LNAS) model for sugar beet, on both synthetic and real data.

Temporal Parallelization of Inference in Hidden Markov Models

This paper presents algorithms for parallelization of inference in hidden Markov models (HMMs). In particular, we propose parallel backward-forward type of filtering and smoothing algorithm as well as parallel Viterbi-type maximum-a-posteriori (MAP) algorithm. We define associative elements and operators to pose these inference problems as parallel-prefix-sum computations in sum-product and max-product algorithms and parallelize them using parallel-scan algorithms. The advantage of the proposed algorithms is that they are computationally efficient in HMM inference problems with long time horizons. We empirically compare the performance of the proposed methods to classical methods on a highly parallel graphical processing unit (GPU).

Exact inference for a class of hidden Markov models on general state spaces

Exact inference for hidden Markov models requires the evaluation of all distributions of interest – filtering, prediction, smoothing and likelihood – with a finite computational effort. This article provides sufficient conditions for exact inference for a class of hidden Markov models on general state spaces given a set of discretely collected indirect observations linked non linearly to the signal, and a set of practical algorithms for inference. The conditions we obtain are concerned with the existence of a certain type of dual process, which is an auxiliary process embedded in the time reversal of the signal, that in turn allows to represent the distributions and functions of interest as finite mixtures of elementary densities or products thereof. We describe explicitly how to update recursively the parameters involved, yielding qualitatively similar results to those obtained with Baum–Welch filters on finite state spaces. We then provide practical algorithms for implementing the recursions, as well as approximations thereof via an informed pruning of the mixtures, and we show superior performance to particle filters both in accuracy and computational efficiency. The code for optimal filtering, smoothing and parameter inference is made available in the Julia package DualOptimalFiltering.

Using HaMMLET for Bayesian Segmentation of WGS Read-Depth Data.

CNV detection requires a high-quality segmentation of genomic data. In many WGS experiments, sample and control are sequenced together in a multiplexed fashion using DNA barcoding for economic reasons. Using the differential read depth of these two conditions cancels out systematic additive errors. Due to this detrending, the resulting data is appropriate for inference using a hidden Markov model (HMM), arguably one of the principal models for labeled segmentation. However, while the usual frequentist approaches such as Baum-Welch are problematic for several reasons, they are often preferred to Bayesian HMM inference, which normally requires prohibitively long running times and exceeds a typical user's computational resources on a genome scale data. HaMMLET solves this problem using a dynamic wavelet compression scheme, which makes Bayesian segmentation of WGS data feasible on standard consumer hardware.

Stochastic dynamics: Models for intrinsic and extrinsic noises and their applications

Mathematical modeling for complex systems and processes requires concepts from and techniques for stochastic dynamics. The theory of stochastic dynamics has two different mathematical representations: Stochastic processes and random dynamical systems. The latter is a more refined mathematical description of reality; it provides not only a stochastic trajectory following one initial condition, but also describes how the entire phase space, with all initial conditions, changes with time. The former represents the stochastic motion of individual systems with intrinsic noise while the latter describes many systems experiencing a common deterministic law of motion which is changing with time due to environmental fluctuations. We call these two situations with intrinsic and extrinsic noises; both have wide applications in chemistry and biology. The recently developed, graph $G(\mathscr{V},\mathscr{E})$ based probabilistic Boolean networks is precisely a class of random dynamical systems (RDS) with discrete state space $\{0,1\}^{\mathscr{V}}$. This paper introduces discrete-time RDS with discrete state space as well as discusses its applications in estimating a rate of convergence in hidden Markov model inference.

Pharmacometrics models with hidden Markovian dynamics.

The aim of this paper is to provide an overview of pharmacometric models that involve some latent process with Markovian dynamics. Such models include hidden Markov models which may be useful for describing the dynamics of a disease state that jumps from one state to another at discrete times. On the contrary, diffusion models are continuous-time and continuous-state Markov models that are relevant for modelling non observed phenomena that fluctuate continuously and randomly over time. We show that an extension of these models to mixed effects models is straightforward in a population context. We then show how the forward-backward algorithm used for inference in hidden Markov models and the extended Kalman filter used for inference in diffusion models can be combined with standard inference algorithms in mixed effects models for estimating the parameters of the model. The use of these models is illustrated with two applications: a hidden Markov model for describing the epileptic activity of a large number of patients and a stochastic differential equation based model for describing the pharmacokinetics of theophyllin.

A Hidden Markov Model inference approach to testing the Random Walk Hypothesis: Empirical evidence from the Nigerian Stock Market

The movement of stock prices, in capital markets across the world, has been found to be both random and non-random. Basically, for a stock price to follow a random walk, its future price changes randomly based on all currently available information in the stock market, its price history inclusive. Some research findings have shown that the existing traditional unit root tests have low statistical power and hence cannot capture gradual changes over successive observations. Consequently, there is a need to revisit the random walk theory in stock prices using other tests. This study employs a Hidden Markov Model (HMM) with time-varying parameters to assess whether the stock price movements of the Nigerian Stock Exchange (NSE) follow a random walk process, or otherwise. Via hidden states, the HMM allows for periods with different volatility levels characterised by the hidden states. By simply accounting for the non-constant variance of the data with a two-state Hidden Markov Model and taking estimation into account via the Sequential Monte Carlo Expectation Maximisation (SMCEM) technique, this study finds no support of randomness. In conclusion, the stock price movements of the NSE do not follow the random walk process.

Motion retrieval based on Motion Semantic Dictionary and HMM inference

A novel motion retrieval method which combines semantic analysis with graph model is proposed. The method includes 2 main stages: (1) in stage of learning, firstly, we can get the Motion Semantic Dictionary (MSD) and the Motion Code Table (MCT) by clustering and handmade based on motion training data learning. Next, the MSD and the MCT are used to calculate system parameters, and the Hidden Markov Model (HMM) is built. For each motion in testing data, aligned cluster analysis (ACA) is used to get key frames, and semantic code is got based on HMM inference. All semantic codes of testing data are combined to construct the Semantic Code Book (SCB). (2) In stage of motion retrieval, according to the above steps, query motion code is got, and the query motion is recognized based on motion code matching. Our method has lesser time and cost than existing algorithms. The experimental results show that the proposed method is effectiveness.

Localizing the latent structure canonical uncertainty: entropy profiles for hidden Markov models

This report addresses state inference for hidden Markov models. These models rely on unobserved states, which often have a meaningful interpretation. This makes it necessary to develop diagnostic tools for quantification of state uncertainty. The entropy of the state sequence that explains an observed sequence for a given hidden Markov chain model can be considered as the canonical measure of state sequence uncertainty. This canonical measure of state sequence uncertainty is not reflected by the classic multivariate state profiles computed by the smoothing algorithm, which summarizes the possible state sequences. Here, we introduce a new type of profiles which have the following properties: (i) these profiles of conditional entropies are a decomposition of the canonical measure of state sequence uncertainty along the sequence and makes it possible to localize this uncertainty, (ii) these profiles are univariate and thus remain easily interpretable on tree structures. We show how to extend the smoothing algorithms for hidden Markov chain and tree models to compute these entropy profiles efficiently.

Multiple LacI-mediated loops revealed by Bayesian statistics and tethered particle motion.

The bacterial transcription factor LacI loops DNA by binding to two separate locations on the DNA simultaneously. Despite being one of the best-studied model systems for transcriptional regulation, the number and conformations of loop structures accessible to LacI remain unclear, though the importance of multiple coexisting loops has been implicated in interactions between LacI and other cellular regulators of gene expression. To probe this issue, we have developed a new analysis method for tethered particle motion, a versatile and commonly used in vitro single-molecule technique. Our method, vbTPM, performs variational Bayesian inference in hidden Markov models. It learns the number of distinct states (i.e. DNA–protein conformations) directly from tethered particle motion data with better resolution than existing methods, while easily correcting for common experimental artifacts. Studying short (roughly 100 bp) LacI-mediated loops, we provide evidence for three distinct loop structures, more than previously reported in single-molecule studies. Moreover, our results confirm that changes in LacI conformation and DNA-binding topology both contribute to the repertoire of LacI-mediated loops formed in vitro, and provide qualitatively new input for models of looping and transcriptional regulation. We expect vbTPM to be broadly useful for probing complex protein–nucleic acid interactions.

In-Home Activity Recognition: Bayesian Inference for Hidden Markov Models

Activity recognition in a home setting is being widely explored as a means to support elderly people living alone. Probabilistic models using classical, maximum-likelihood estimation methods are known to work well in this domain, but they are prone to overfitting and require labeled activity data for every new site. This limitation has important practical implications, because labeling activities is expensive, time-consuming, and intrusive to the monitored person. In this article, the authors use Markov Chain Monte Carlo techniques to estimate the parameters of activity recognition models in a Bayesian framework. They evaluate their approach by comparing it to a state-of-the-art maximum-likelihood method on three publicly available real-world datasets. Their approach achieves significantly better recognition performance (p less than or equal to 0.05).

Approximate Bayesian Computation for Smoothing

We consider a method for approximate inference in hidden Markov models (HMMs). The method circumvents the need to evaluate conditional densities of observations given the hidden states. It may be considered an instance of Approximate Bayesian Computation (ABC) and it involves the introduction of auxiliary variables valued in the same space as the observations. The quality of the approximation may be controlled to arbitrary precision through a parameter ε > 0. We provide theoretical results which quantify, in terms of ε, the ABC error in approximation of expectations of additive functionals with respect to the smoothing distributions. Under regularity assumptions, this error is , where n is the number of time steps over which smoothing is performed. For numerical implementation, we adopt the forward-only sequential Monte Carlo (SMC) scheme of [14] and quantify the combined error from the ABC and SMC approximations. This forms some of the first quantitative results for ABC methods which jointly treat the ABC and simulation errors, with a finite number of data and simulated samples.

Dynamic state allocation for MEG source reconstruction.

Our understanding of the dynamics of neuronal activity in the human brain remains limited, due in part to a lack of adequate methods for reconstructing neuronal activity from noninvasive electrophysiological data. Here, we present a novel adaptive time-varying approach to source reconstruction that can be applied to magnetoencephalography (MEG) and electroencephalography (EEG) data. The method is underpinned by a Hidden Markov Model (HMM), which infers the points in time when particular states re-occur in the sensor space data. HMM inference finds short-lived states on the scale of 100 ms. Intriguingly, this is on the same timescale as EEG microstates. The resulting state time courses can be used to intelligently pool data over these distinct and short-lived periods in time. This is used to compute time-varying data covariance matrices for use in beamforming, resulting in a source reconstruction approach that can tune its spatial filtering properties to those required at different points in time. Proof of principle is demonstrated with simulated data, and we demonstrate improvements when the method is applied to MEG.

Online Expectation Maximization based algorithms for inference in Hidden Markov Models

The Expectation Maximization (EM) algorithm is a versatile tool for model parameter estimation in latent data models. When processing large data sets or data stream however, EM becomes intractable since it requires the whole data set to be available at each iteration of the algorithm. In this contribution, a new generic online EM algorithm for model parameter inference in general Hidden Markov Model is proposed. This new algorithm updates the parameter estimate after a block of observations is processed (online). The convergence of this new algorithm is established, and the rate of convergence is studied showing the impact of the block-size sequence. An averaging procedure is also proposed to improve the rate of convergence. Finally, practical illustrations are presented to highlight the performance of these algorithms in comparison to other online maximum likelihood procedures.

Inference in Hidden Markov Models with Explicit State Duration Distributions

In this letter we borrow from the inference techniques developed for unbounded state-cardinality (nonparametric) variants of the HMM and use them to develop a tuning-parameter free, black-box inference procedure for Explicit-state-duration hidden Markov models (EDHMM). EDHMMs are HMMs that have latent states consisting of both discrete state-indicator and discrete state-duration random variables. In contrast to the implicit geometric state duration distribution possessed by the standard HMM, EDHMMs allow the direct parameterisation and estimation of per-state duration distributions. As most duration distributions are defined over the positive integers, truncation or other approximations are usually required to perform EDHMM inference.

LARGE DEVIATIONS FOR INHOMOGENEOUS SYSTEMS

In this paper large deviations for mixture of large deviation systems are investigated. Some general results are presented, with conditions either less restrictive or more practical for verification. Applications are illustrated in a concrete case-mixture of products of probability measures (homogeneous or inhomogeneous). We also discussed potential applications to problems of statistic inference for hidden Markov models.

EM versus Markov chain Monte Carlo for estimation of hidden Markov models: a computational perspective

Hidden Markov models (HMMs) and related models have become standard in statistics during the last 15--20 years, with applications in diverse areas like speech and other statistical signal processing, hydrology, financial statistics and econometrics, bioinformatics etc. Inference in HMMs is traditionally often carried out using the EM algorithm, but examples of Bayesian estimation, in general implemented through Markov chain Monte Carlo (MCMC) sampling are also frequent in the HMM literature. The purpose of this paper is to compare the EM and MCMC approaches in three cases of different complexity; the examples include model order selection, continuous-time HMMs and variants of HMMs in which the observed data depends on many hidden variables in an overlapping fashion. All these examples in some way or another originate from real-data applications. Neither EM nor MCMC analysis of HMMs is a black-box methodology without need for user-interaction, and we will illustrate some of the problems, like poor mixing and long computation times, one may expect to encounter.

Reversible jump and the label switching problem in hidden Markov models

Reversible jump Markov chain Monte Carlo (RJMCMC) algorithms can be efficiently applied in Bayesian inference for hidden Markov models (HMMs), when the number of latent regimes is unknown. As for finite mixture models, when priors are invariant to the relabelling of the regimes, HMMs are unidentifiable in data fitting, because multiple ways to label the regimes can alternate during the MCMC iterations; this is the so-called label switching problem. HMMs with an unknown number of regimes are considered here and the goal of this paper is the comparison, both applied and theoretical, of five methods used for tackling label switching within a RJMCMC algorithm; they are: post-processing, partial reordering, permutation sampling, sampling from a Markov prior and rejection sampling. The five strategies we compare have been proposed mostly in the literature of finite mixture models and only two of them, i.e. rejection sampling and partial reordering, have been presented in RJMCMC algorithms for HMMs. We consider RJMCMC algorithms in which the parameters are updated by Gibbs sampling and the dimension of the model changes in split-and-merge and birth-and-death moves. Finally, an example illustrates and compares the five different methodologies.

Approximate inference in hidden Markov models using iterative active state selection

The inferential task of computing the marginal posterior probability mass functions of state variables and pairs of consecutive state variables of a hidden Markov model is considered. This can be exactly and efficiently performed using a message passing scheme such as the Bahl-Cocke-Jelinek-Raviv (BCJR) algorithm. We present a novel iterative reduced complexity variation of the BCJR algorithm that uses reduced support approximations for the forward and backward messages, as in the M-BCJR algorithm. Forward/backward message computation is based on the concept of expectation propagation, which results in an algorithm similar to the M-BCJR algorithm with the active state selection criterion being changed from the filtered distribution of state variables to beliefs of state variables. By allowing possibly different supports for the forward and backward messages, we derive identical forward and backward recursions that can be iterated. Simulation results of application for trellis-based equalization of a wireless communication system confirm the improved performance over the M-BCJR algorithm.