Discrete-time Hidden Markov Model Research Articles

Estimating Hidden Markov Models (HMMs) of the cognitive process in strategic thinking using eye-tracking

Hidden Markov Models (HMMs) are used to study language, sleep, macroeconomic states, and other processes that reflect probabilistic transitions between states that can't be observed directly. This paper applies HMMs to data from location-based game theory experiments. In these location games, players choose a pixel location from an image. These players either have a common goal (choose a matching location), or competing goals, to mismatch (hide) or match (seek) in hider-seeker games. We use eye-tracking to record where players look throughout the experimental decision. Each location's numerical salience is predicted using an accurate, specialized vision science-based neural network [the Saliency Attentive Model (SAM)]. The HMM shows the pattern of transitioning from hidden states corresponding to either high or low-salience locations, combining the eye-tracking and salience data. The transitions vary based on the player's strategic goal. For example, hiders transition more often to low-salience states than seekers do. The estimated HMM is then used to do two useful things. First, a continuous-time HMM (cHMM) predicts the salience level of each player's looking over several seconds. The cHMM can then be used to predict what would happen if the same process was truncated by time pressure: This calculation makes a specific numerical prediction about how often seekers will win, and it predicts an increase in win rate but underestimates the size of the change. Second, a discrete-time HMM (dHMM) can be used to infer levels of strategic thinking from high-to-low salience eye-tracking transitions. The resulting estimates are more plausible than some maximum-likelihood models, which underestimate strategic sophistication in these games. Other applications of HMM in experimental economics are suggested.

A Resilient Adaptive Event-triggered H\(\infty\) Tracking Control of T-S Fuzzy Systems with Markovian Jump Parameters under DOS Attacks

In this paper, the problem of H\(\infty\) output-tracking control for T-S fuzzy markov jump systems(MJSs) with imperfect premise matching is investigated. The conditional probability information between the system and the controller is imperfectly known. Firstly, due to the in influence of network-induce delay, the system mode information can not be transmitted to the controller synchronously, thus, a discrete-time hidden markov model(HMM) is established to describe the asynchronous phenomenon between the system and the controller; Secondly, form the prospectively of practical applications, packet loss in two channels is taken into consideration, which is caused by denial-of-service(DOS) attacks; Thirdly, in order to improve the data transmission efficiency and save network bandwidth resources, a novel resilient adaptive event-triggered(AET) mechanism is proposed, in which the event-based threshold parameter is dynamically regulated over the processing. Additionally, resorted the compensation principle and the lyapunov-krasovski(L-K) functions methods, some sufficient conditions in form of liner matrix inequality(LMI) are achieved to co-guarantee the stability and H\(\infty\) tracking property of the closed-loop system; Finally, two examples are given to verify the effectiveness of our design mentality.

Statistical modelling of individual animal movement: an overview of key methods and a discussion of practical challenges

With the influx of complex and detailed tracking data gathered from electronic tracking devices, the analysis of animal movement data has recently emerged as a cottage industry among biostatisticians. New approaches of ever greater complexity are continue to be added to the literature. In this paper, we review what we believe to be some of the most popular and most useful classes of statistical models used to analyse individual animal movement data. Specifically, we consider discrete-time hidden Markov models, more general state-space models and diffusion processes. We argue that these models should be core components in the toolbox for quantitative researchers working on stochastic modelling of individual animal movement. The paper concludes by offering some general observations on the direction of statistical analysis of animal movement. There is a trend in movement ecology towards what are arguably overly complex modelling approaches which are inaccessible to ecologists, unwieldy with large data sets or not based on mainstream statistical practice. Additionally, some analysis methods developed within the ecological community ignore fundamental properties of movement data, potentially leading to misleading conclusions about animal movement. Corresponding approaches, e.g. based on Levy walk-type models, continue to be popular despite having been largely discredited. We contend that there is a need for an appropriate balance between the extremes of either being overly complex or being overly simplistic, whereby the discipline relies on models of intermediate complexity that are usable by general ecologists, but grounded in well-developed statistical practice and efficient to fit to large data sets.

Incorporating Telemetry Error into Hidden Markov Models of Animal Movement Using Multiple Imputation

When data streams are observed without error and at regular time intervals, discrete-time hidden Markov models (HMMs) have become immensely popular for the analysis of animal location and auxiliary biotelemetry data. However, measurement error and temporally irregular data are often pervasive in telemetry studies, particularly in marine systems. While relatively small amounts of missing data that are missing-completely-at-random are not typically problematic in HMMs, temporal irregularity can result in few (if any) observations aligning with the regular time steps required by HMMs. Fitting HMMs that explicitly account for uncertainty attributable to location measurement error, temporally irregular observations, or other forms of missing data typically requires computationally demanding techniques, such as Markov chain Monte Carlo (MCMC). Using simulation and a real-world bearded seal (Erignathus barbatus) example, I investigate a practical alternative to incorporating measurement error and temporally irregular observations into HMMs based on multiple imputation of the position process drawn from a single-state continuous-time movement model. This two-stage approach is relatively simple, performed with existing software using efficient maximum likelihood methods, and completely parallelizable. I generally found the approach to perform well across a broad range of simulated measurement error and irregular sampling rates, with latent states and locations reliably recovered in nearly all simulated scenarios. However, high measurement error coupled with low sampling rates often induced bias in both the estimated probability distributions of data streams derived from the imputed position process and the estimated effects of spatial covariates on state transition probabilities. Results from the two-stage analysis of the bearded seal data were similar to a more computationally intensive single-stage MCMC analysis, but the two-stage analysis required much less computation time and no custom model-fitting algorithms. I thus found the two-stage multiple-imputation approach to be promising in terms of its ease of implementation, computation time, and performance. Code for implementing the approach using the R package “momentuHMM” is provided.Supplementary materials accompanying this paper appear online.

Bayesian nonhomogeneous Markov models via Pólya-Gamma data augmentation with applications to rainfall modeling

Discrete-time hidden Markov models are a broadly useful class of latent-variable models with applications in areas such as speech recognition, bioinformatics, and climate data analysis. It is common in practice to introduce temporal nonhomogeneity into such models by making the transition probabilities dependent on time-varying exogenous input variables via a multinomial logistic parametrization. We extend such models to introduce additional nonhomogeneity into the emission distribution using a generalized linear model (GLM), with data augmentation for sampling-based inference. However, the presence of the logistic function in the state transition model significantly complicates parameter inference for the overall model, particularly in a Bayesian context. To address this, we extend the recently-proposed Polya-Gamma data augmentation approach to handle nonhomogeneous hidden Markov models (NHMMs), allowing the development of an efficient Markov chain Monte Carlo (MCMC) sampling scheme. We apply our model and inference scheme to 30 years of daily rainfall in India, leading to a number of insights into rainfall-related phenomena in the region. Our proposed approach allows for fully Bayesian analysis of relatively complex NHMMs on a scale that was not possible with previous methods. Software implementing the methods described in the paper is available via the R package NHMM.

-Filtering for discrete-time hidden Markov jump systems

ABSTRACTIn this paper, we consider the filtering problem for a discrete-time Markov jump linear systemin which the Markov parameter is not available. In the spirit of active fault-tolerant control systems, it is supposed that there is a discrete-time hidden Markov model in which the observable part represents the information coming from a detector and available to the filter while the hidden part θ(k) of the process represents the dynamics of the real system. Several models found in the literature are encompassed by this framework, like the complete observation case, the clustering information case, the mode-independent case, and Markov models in fault-tolerant control. We start by analysing an auxiliary filtering problem in which it is assumed that both are available to the filter, so that necessary and sufficient conditions in terms of LMI (linear matrix inequalities) for the existence of the optimal filter can be obtained. In the sequel, we consider the realistic case in which only is available for the design of the filter. A sufficient condition based on an LMI optimisation problem to design a guaranteed cost filter that depends only on is presented. The results are strengthened for two particular cases, named the cluster case and the Bernoulli case. The paper is concluded with some numerical examples to illustrate the obtained results.

A continuous-time HMM approach to modeling the magnitude-frequency distribution of earthquakes

ABSTRACTThe magnitude-frequency distribution (MFD) of earthquake is a fundamental statistic in seismology. The so-called b-value in the MFD is of particular interest in geophysics. A continuous time hidden Markov model (HMM) is proposed for characterizing the variability of b-values. The HMM-based approach to modeling the MFD has some appealing properties over the widely used sliding-window approach. Often, large variability appears in the estimation of b-value due to window size tuning, which may cause difficulties in interpretation of b-value heterogeneities. Continuous-time hidden Markov models (CT-HMMs) are widely applied in various fields. It bears some advantages over its discrete time counterpart in that it can characterize heterogeneities appearing in time series in a finer time scale, particularly for highly irregularly-spaced time series, such as earthquake occurrences. We demonstrate an expectation–maximization algorithm for the estimation of general exponential family CT-HMM. In parallel with discrete-time hidden Markov models, we develop a continuous time version of Viterbi algorithm to retrieve the overall optimal path of the latent Markov chain. The methods are applied to New Zealand deep earthquakes. Before the analysis, we first assess the completeness of catalogue events to assure the analysis is not biased by missing data. The estimation of b-value is stable over the selection of magnitude thresholds, which is ideal for the interpretation of b-value variability.

Inferring Traffic Signal Phases From Turning Movement Counters Using Hidden Markov Models

This work poses the problem of estimating traffic signal phases from a sequence of maneuvers. We model the problem as an inference problem on a discrete-time hidden Markov model (HMM) in which maneuvers are observations and signal phases are hidden states. The model is calibrated from maneuver observations using either the classical Baum-Welch algorithm or a Bayesian learning algorithm. The trained model is then used to infer the traffic signal phases on the data set via the Viterbi algorithm. When training with the Bayesian learning algorithm, we set the prior distribution as a Dirichlet distribution. We identify the best parameters of the prior distribution for both fixed-time and sensor-actuated signals using numerical simulations and employ them in the field experiments. It is shown that when the model is trained by the Bayesian learning method with appropriate prior parameters from the Dirichlet distribution, the inferred phases are more accurate in both numerical and field experiments. Because the best set of prior parameters for a fixed-time intersection is different from those for sensor-actuated signals, a classification strategy to distinguish between these two types of signals is proposed. The supporting source code and data are available for download at https://github.com/reisiga2/TrafficSignalPhaseEstimation.

Variational mean-field algorithm for efficient inference in large systems of stochastic differential equations

This work introduces a Gaussian variational mean-field approximation for inference in dynamical systems which can be modeled by ordinary stochastic differential equations. This new approach allows one to express the variational free energy as a functional of the marginal moments of the approximating Gaussian process. A restriction of the moment equations to piecewise polynomial functions, over time, dramatically reduces the complexity of approximate inference for stochastic differential equation models and makes it comparable to that of discrete time hidden Markov models. The algorithm is demonstrated on state and parameter estimation for nonlinear problems with up to 1000 dimensional state vectors and compares the results empirically with various well-known inference methodologies.

A Coupled Hidden Markov Model for Disease Interactions

To investigate interactions between parasite species in a host, a population of field voles was studied longitudinally, with presence or absence of six different parasites measured repeatedly. Although trapping sessions were regular, a different set of voles was caught at each session, leading to incomplete profiles for all subjects. We use a discrete time hidden Markov model for each disease with transition probabilities dependent on covariates via a set of logistic regressions. For each disease the hidden states for each of the other diseases at a given time point form part of the covariate set for the Markov transition probabilities from that time point. This allows us to gauge the influence of each parasite species on the transition probabilities for each of the other parasite species. Inference is performed via a Gibbs sampler, which cycles through each of the diseases, first using an adaptive Metropolis–Hastings step to sample from the conditional posterior of the covariate parameters for that particular disease given the hidden states for all other diseases and then sampling from the hidden states for that disease given the parameters. We find evidence for interactions between several pairs of parasites and of an acquired immune response for two of the parasites.

Flexible and practical modeling of animal telemetry data: hidden Markov models and extensions

We discuss hidden Markov-type models for fitting a variety of multistate random walks to wildlife movement data. Discrete-time hidden Markov models (HMMs) achieve considerable computational gains by focusing on observations that are regularly spaced in time, and for which the measurement error is negligible. These conditions are often met, in particular for data related to terrestrial animals, so that a likelihood-based HMM approach is feasible. We describe a number of extensions of HMMs for animal movement modeling, including more flexible state transition models and individual random effects (fitted in a non-Bayesian framework). In particular we consider so-called hidden semi-Markov models, which may substantially improve the goodness of fit and provide important insights into the behavioral state switching dynamics. To showcase the expediency of these methods, we consider an application of a hierarchical hidden semi-Markov model to multiple bison movement paths.

DTHMM based delay modeling and prediction for networked control systems

In the forward channel of a networked control system (NCS), by defining the network states as a hidden Markov chain and quantizing the network-induced delays to a discrete sequence distributing over a finite time interval, the relation between the network states and the network-induced delays is modelled as a discrete-time hidden Markov model (DTHMM). The expectation maximization (EM) algorithm is introduced to derive the maximum- likelihood estimation (MLE) of the parameters of the DTHMM. Based on the derived DTHMM, the Viterbi algorithm is introduced to predict the controller-to-actuator (C-A) delay during the current sampling period. The simulation experiments demonstrate the ef- fectiveness of the modelling and predicting methods proposed.

Optimal Hedging in Discrete and Continuous Time

In this article we find the optimal solution of the hedging problem in discrete time by minimizing the mean square hedging error, when the underlying assets are multidimensional, extending the results of Schweizer (1995). We also find explicit expressions for the optimal hedging problem in continuous time when the underlying assets are modeled by a regime-switching geometric L'evy process. It is also shown that the continuous time solution can be approximated by discrete time Hidden Markov models processes. In addition, in the case of the regime-switching geometric Brownian motion, the optimal prices are the same as the prices under an equivalent martingale measure, making that measure a natural choice. However, the optimal hedging strategy is not the usual delta hedging but it can be easily computed by Monte Carlo methods. These results presented here are different from those of Follmer & Schweizer (1991) on hedging and Prigent (2003) on weak convergence, who both considered the problem of local quadratic risk minimizing strategies instead of the global quadratic risk minimizing strategies. Even in the discrete case, these two settings were shown to lead, in general, to different solutions by Schweizer (1995).

The stability of conditional Markov processes and Markov chains in random environments

We consider a discrete time hidden Markov model where the signal is a stationary Markov chain. When conditioned on the observations, the signal is a Markov chain in a random environment under the conditional measure. It is shown that this conditional signal is weakly ergodic when the signal is ergodic and the observations are nondegenerate. This permits a delicate exchange of the intersection and supremum of σ-fields, which is key for the stability of the nonlinear filter and partially resolves a long-standing gap in the proof of a result of Kunita [J. Multivariate Anal. 1 (1971) 365–393]. A similar result is obtained also in the continuous time setting. The proofs are based on an ergodic theorem for Markov chains in random environments in a general state space.

Estimating rate constants in hidden Markov models by the EM algorithm

The EM algorithm, e.g., the Baum-Welch (1970) re-estimation, is an important tool for parameter estimation in discrete-time hidden Markov models. We present a direct re-estimation of rate constants for applications in which the underlying Markov process is continuous in time. Previous estimation of discrete-time transition probabilities is not necessary.

Hidden Markov model state estimation with randomly delayed observations

This paper considers state estimation for a discrete-time hidden Markov model (HMM) when the observations are delayed by a random time. The delay process is itself modeled as a finite state Markov chain that allows an augmented state HMM to model the overall system. State estimation algorithms for the resulting HMM are then presented, and their performance is studied in simulations. The motivation for the model stems from the situation when distributed sensors transmit measurements over a connectionless packet switched communications network.