Abstract Hidden Markov Models Research Articles

Know what you don't know: Embracing state uncertainty in disease‐structured multistate models

Abstract Hidden Markov models (HMMs) are broadly applicable hierarchical models that derive their utility from separating state processes from observation processes yielding the data. Multistate models such as mark–recapture and dynamic multistate occupancy models are HMMs frequently used in ecology. In their early formulations, states, such as pathogen infection status, were assumed to be perfectly observed without ambiguity. However, state uncertainty is a pervasive feature of many ecological studies, and multievent models were developed to explicitly account for it. We developed a novel extended multievent mark–recapture model that incorporates state uncertainty at multiple levels of detection. Using a disease‐structured example, both false negative and false positive state assignment errors are modelled at two levels of state assignment—the pathogen sampling process and the diagnostic process that samples are subjected to. We additionally describe methods to jointly model infection intensity to integrate heterogeneity in ecological parameters, such as mortality and infection dynamics, and the pathogen detection processes. We provide code to simulate and analyse datasets with various underlying ecological processes and fit our model to a mark–recapture dataset of Mixophyes fleayi (Fleay's barred frog) infected with the amphibian chytrid fungus (Batrachochytrium dendrobatidis, Bd). In our case study, we found evidence for various state assignment errors: the sampling protocol performed poorly in detecting Bd, pathogen detection was highly dependent on infection intensity and false positives were non‐negligible. Incorporating state uncertainty yielded significantly higher estimates of infection prevalence and 4–5 times lower rates of infection state transitions compared to those obtained from a traditional multistate model. Our results highlight that incorporating state assignment errors improves inference on the ecological process, especially when sensitivity and specificity of the state assignment processes are low. The general model structure can be applied to other HMMs, providing a foundation for modelling state uncertainty in related models. For disease‐structured multistate models, we recommend conducting robust design surveys and collecting samples during each capture event to facilitate incorporating pathogen detection errors.

Hidden Markov models: Pitfalls and opportunities in ecology

Abstract Hidden Markov models (HMMs) and their extensions are attractive methods for analysing ecological data where noisy, multivariate measurements are made of a hidden, ecological process, and where this hidden process is represented by a sequence of discrete states. Yet, as these models become more complex and challenging to understand, it is important to consider what pitfalls these methods have and what opportunities there are for future research to address these pitfalls. In this paper, we review five lesser known pitfalls one can encounter when using HMMs or their extensions to solve ecological problems: (a) violation of the snapshot property in continuous‐time HMMs; (b) biased inference from hierarchical HMMs when applied to temporally misaligned processes; (c) sensitive inference from using random effects to partially pool across heterogeneous individuals; (d) computational burden when using HMMs to approximate models with continuous state spaces; and (e) difficulty linking the hidden process to space or environment. This review is for ecologists and ecological statisticians familiar with HMMs, but who may be less aware of the problems that arise in more specialised applications. We demonstrate how each pitfall arises, by simulation or example, and discuss why this pitfall is important to consider. Along with identifying the problems, we highlight potential research opportunities and offer ideas that may help alleviate these pitfalls. Each of the methods we review are solutions to current ecological research problems. We intend for this paper to heighten awareness of the pitfalls ecologists may encounter when applying these more advanced methods, but we also hope that by highlighting future research opportunities, we can inspire ecological statisticians to weaken these pitfalls and provide improved methods.

Worth the effort? A practical examination of random effects in hidden Markov models for animal telemetry data

Abstract Hidden Markov models (HMMs) that include individual‐level random effects have recently been promoted for inferring animal movement behaviour from biotelemetry data. These ‘mixed HMMs’ come at significant cost in terms of implementation and computation, and discrete random effects have been advocated as a practical alternative to more computationally intensive continuous random effects. However, the performance of mixed HMMs has not yet been sufficiently explored to justify their widespread adoption, and there is currently little guidance for practitioners weighing the costs and benefits of mixed HMMs for a particular research objective. I performed an extensive simulation study comparing the performance of a suite of fixed and random effect models for individual heterogeneity in the hidden state process of a two‐state HMM. I focused on sampling scenarios more typical of telemetry studies, which often consist of relatively long time series (30–250 observations per animal) for relatively few individuals (5–100 animals). I generally found mixed HMMs did not improve state assignment relative to standard HMMs. Reliable estimation of random effects required larger sample sizes than are often feasible in telemetry studies. Continuous random effect models performed reasonably well with data generated under discrete random effects, but not vice versa. Random effects accounting for unexplained individual variation can improve estimation of state transition probabilities and measurable covariate effects, but discrete random effects can be a relatively poor (and potentially misleading) approximation for continuous variation. When weighing the costs and benefits of mixed HMMs, three important considerations are study objectives, sample size and model complexity. HMM applications often focus on state assignment with little emphasis on heterogeneity in state transition probabilities, in which case random effects in the hidden state process simply may not be worth the additional effort. However, if explaining variation in state transition probabilities is a primary objective and sufficient explanatory covariates are not available, then random effects are worth pursuing as a more parsimonious alternative to individual fixed effects. To help put my findings in context and illustrate some potential challenges that practitioners may encounter when applying mixed HMMs, I revisit a previous analysis of long‐finned pilot whale biotelemetry data.

Generalized hidden Markov models for phylogenetic comparative datasets

Abstract Hidden Markov models (HMMs) have emerged as an important tool for understanding the evolution of characters that take on discrete states. Their flexibility and biological sensibility make them appealing for many phylogenetic comparative applications. Previously available packages placed unnecessary limits on the number of observed and hidden states that can be considered when estimating transition rates and inferring ancestral states on a phylogeny. To address these issues, we expanded the capabilities of the R package corHMM to handle n‐state and n‐character problems and provide users with a streamlined set of functions to create custom HMMs for any biological question of arbitrary complexity. We show that increasing the number of observed states increases the accuracy of ancestral state reconstruction. We also explore the conditions for when an HMM is most effective, finding that an HMM is an appropriate model when the degree of rate heterogeneity is moderate to high. Finally, we demonstrate the importance of these generalizations by reconstructing the phyllotaxy of the ancestral angiosperm flower. Partially contradicting previous results, we find the most likely state to be a whorled perianth, whorled androecium, whorled gynoecium. The difference between our analysis and previous studies was that our modelling explicitly allowed for the correlated evolution of several flower characters.

Mixed location scale hidden Markov model for the analysis of intensive longitudinal data

Hidden Markov models (HMM) presents an attractive analytical framework for capturing the state-switching process for auto-correlated data. These models have been extended to longitudinal data setting where simultaneous multiple processes are observed by including subject specific random effects. However, application of HMMs for intensive longitudinal data, where each subject gets measured intensively over relatively short period of time, has not been widely studied. In this paper, we extend the mixed hidden Markov model and allow subject heterogeneity with respect to the mean and within subject variance by including subject random effects in both perspectives. We focus on the application of this model to intensive longitudinal studies in psychological and behavioral research where individual’s latent states and state-switching process are of interest. Models are estimated using forward–backward algorithm via Bayesian sampling approach. Advantages over regular HMM and mixed HMM that only accounts for the subjects’ mean heterogeneity are illustrated through a series of simulation studies. Finally, models are applied to an adolescent mood study data set and results show that the proposed mixed location scale HMM achieves better model fit and more interpretative mood state identification in terms of state specific covariate effects compared to regular HMM and mixed HMM.

Joint modelling of multi‐scale animal movement data using hierarchical hidden Markov models

Abstract Hidden Markov models are prevalent in animal movement modelling, where they are widely used to infer behavioural modes and their drivers from various types of telemetry data. To allow for meaningful inference, observations need to be equally spaced in time, or otherwise regularly sampled, where the corresponding temporal resolution strongly affects what kind of behaviours can be inferred from the data. Recent advances in biologging technology have led to a variety of novel telemetry sensors which often collect data from the same individual simultaneously at different time‐scales, for example step lengths obtained from GPS tags every hour, dive depths obtained from time‐depth recorders once per dive, or accelerations obtained from accelerometers several times per second. However, to date, statistical machinery to address the corresponding complex multi‐stream and multi‐scale data is lacking. We propose hierarchical hidden Markov models as a versatile statistical framework that naturally accounts for differing temporal resolutions across multiple variables. In these models, the observations are regarded as stemming from multiple connected behavioural processes, each of which operates at the time‐scale at which the corresponding variables were observed. By jointly modelling multiple data streams, collected at different temporal resolutions, corresponding models can be used to infer behavioural modes at multiple time‐scales and in particular, help to draw a much more comprehensive picture of an animal's movement patterns, for example with regard to long‐term versus short‐term movement strategies. The suggested approach is illustrated in two real‐data applications, where we jointly model (a) coarse‐scale horizontal and fine‐scale vertical Atlantic cod Gadus morhua movements throughout the English Channel, and (b) coarse‐scale horizontal movements and corresponding fine‐scale accelerations of a horn shark Heterodontus francisci tagged off the Californian coast.

Parameter redundancy and identifiability in hidden Markov models

Hidden Markov models are a flexible class of models that can be used to describe time series data which depends on an unobservable Markov process. As with any complex model, it is not always obvious whether all the parameters are identifiable, or if the model is parameter redundant; that is, the model can be reparameterised in terms of a smaller number of parameters. This paper considers different methods for detecting parameter redundancy and identifiability in hidden Markov models. We examine both numerical methods and methods that involve symbolic algebra. These symbolic methods require a unique representation of a model, known as an exhaustive summary. We provide an exhaustive summary for hidden Markov models and show how it can be used to investigate identifiability.

How Linkage Error Affects Hidden Markov Model Estimates: A Sensitivity Analysis

Abstract Hidden Markov models (HMMs) are increasingly used to estimate and correct for classification error in categorical, longitudinal data, without the need for a “gold standard,” error-free data source. To accomplish this, HMMs require multiple observations over time on a single indicator and assume that the errors in these indicators are conditionally independent. Unfortunately, this “local independence” assumption is often unrealistic, untestable, and a source of serious bias. Linking independent data sources can solve this problem by making the local independence assumption plausible across sources, while potentially allowing for local dependence within sources. However, record linkage introduces a new problem: the records may be erroneously linked or incorrectly not linked. In this paper, we investigate the effects of linkage error on HMM estimates of transitions between employment contract types. Our data come from linking a labor force survey to administrative employer records; this linkage yields two indicators per time point that are plausibly conditionally independent. Our results indicate that both false-negative and false-positive linkage error turn out to be problematic primarily if the error is large and highly correlated with the dependent variable. Moreover, under certain conditions, false-positive linkage error (mislinkage) in fact acts as another source of misclassification that the HMM can absorb into its error-rate estimates, leaving the latent transition estimates unbiased. In these cases, measurement error modeling already accounts for linkage error. Our results also indicate where these conditions break down and more complex methods would be needed.

A Hidden Markov Model for the Linguistic Analysis of the Voynich Manuscript

Hidden Markov models are a very useful tool in the modeling of time series and any sequence of data. In particular, they have been successfully applied to the field of mathematical linguistics. In this paper, we apply a hidden Markov model to analyze the underlying structure of an ancient and complex manuscript, known as the Voynich manuscript, which remains undeciphered. By assuming a certain number of internal states representations for the symbols of the manuscripts, we train the network by means of the α and β -pass algorithms to optimize the model. By this procedure, we are able to obtain the so-called transition and observation matrices to compare with known languages concerning the frequency of consonant andvowel sounds. From this analysis, we conclude that transitions occur between the two states with similar frequencies to other languages. Moreover, the identification of the vowel and consonant sounds matches some previous tentative bottom-up approaches to decode the manuscript.

Limited Dynamic Forecasting of Hidden Markov Models

Hidden Markov Models (HMMs) have emerged as an empirical “workhorse” in the marketing literature in capturing and forecasting within-customer non-stationary behaviors. Extant research has demonstrated that HMMs typically outperform nested benchmarks when examining fit statistics aggregated over individuals and time, but have remained largely silent on the set of dynamic out-of-sample forecasting paths offered by an HMM at the individual level. We examine the capabilities of a two-state HMM using theory and reveal a surprising result: an HMM’s forecasting paths are generally limited to monotonic mean-reverting trajectories. Specifically, they lack the notable flexibility associated with the in-sample state-switching imputations, which are generally (but, as we show, erroneously) presumed to exist in the holdout sample as well. Further, we find that common HMM extensions such as adding more hidden states, allowing for heterogeneity, allowing for covariates, and using hidden semi-Markov models do not alleviate the limited forecasting flexibility. Using a simulation design, we show how these limitations can affect forecasting performance empirically. We discuss implications of the limited forecasting properties of HMMs for researchers and managers.

Latent subspace modeling of sequential data under the maximum entropy discrimination framework

Abstract Hidden Markov models (HMMs) are a popular approach for modeling continuous sequential data, typically based on the assumption of Gaussian-distributed observations. A significant issue HMMs with Gaussian conditional densities are confronted with concerns effectively modeling high-dimensional observations, without getting prone to overfitting or singularities. To this end, one can resort to extracting lower-dimensional latent variable representations of the observed high-dimensional data, as part of the inference algorithm of the postulated HMM. Factor analysis (FA) is a well-established linear latent variable scheme that can be employed for this purpose; its functionality consists in modeling the covariances between the elements of multivariate observations under a set of linear assumptions. Recently, it has been proposed that FA can be effectively generalized under an efficient large-margin Bayesian inference perspective, namely maximum entropy discrimination (MED). This work capitalizes on these recent findings to derive an effective HMM-driven sequential data modeling framework for high-dimensional data. Our proposed approach extracts lower-dimensional latent variable representations of observed high-dimensional data, taking into account the large-margin principle. On this basis, it postulates that the data temporal dynamics are conditional to the inferred values of these latent variables. We devise efficient mean-field inference algorithms for our model, and exhibit its advantages through a set of experiments.

A Nonstationary Hidden Markov Model with Approximately Infinitely-Long Time-Dependencies

Hidden Markov models (HMMs) are a popular approach for modeling sequential data, typically based on the assumption of a first-order Markov chain. In other words, only one-step back dependencies are modeled which is a rather unrealistic assumption in most applications. In this paper, we propose a method for postulating HMMs with approximately infinitely-long time-dependencies. Our approach considers the whole history of model states in the postulated dependencies, by making use of a recently proposed nonparametric Bayesian method for modeling label sequences with infinitely-long time dependencies, namely the sequence memoizer. We manage to derive training and inference algorithms for our model with computational costs identical to simple first-order HMMs, despite its entailed infinitely-long time-dependencies, by employing a mean-field-like approximation. The efficacy of our proposed model is experimentally demonstrated.

Statistical Inference in Hidden Markov Models Using k-Segment Constraints

Hidden Markov models (HMMs) are one of the most widely used statistical methods for analyzing sequence data. However, the reporting of output from HMMs has largely been restricted to the presentation of the most-probable (MAP) hidden state sequence, found via the Viterbi algorithm, or the sequence of most probable marginals using the forward–backward algorithm. In this article, we expand the amount of information we could obtain from the posterior distribution of an HMM by introducing linear-time dynamic programming recursions that, conditional on a user-specified constraint in the number of segments, allow us to (i) find MAP sequences, (ii) compute posterior probabilities, and (iii) simulate sample paths. We collectively call these recursions k-segment algorithms and illustrate their utility using simulated and real examples. We also highlight the prospective and retrospective use of k-segment constraints for fitting HMMs or exploring existing model fits. Supplementary materials for this article are available online.

Simultaneous characterization of sense and antisense genomic processes by the double-stranded hidden Markov model.

Hidden Markov models (HMMs) have been extensively used to dissect the genome into functionally distinct regions using data such as RNA expression or DNA binding measurements. It is a challenge to disentangle processes occurring on complementary strands of the same genomic region. We present the double-stranded HMM (dsHMM), a model for the strand-specific analysis of genomic processes. We applied dsHMM to yeast using strand specific transcription data, nucleosome data, and protein binding data for a set of 11 factors associated with the regulation of transcription.The resulting annotation recovers the mRNA transcription cycle (initiation, elongation, termination) while correctly predicting strand-specificity and directionality of the transcription process. We find that pre-initiation complex formation is an essentially undirected process, giving rise to a large number of bidirectional promoters and to pervasive antisense transcription. Notably, 12% of all transcriptionally active positions showed simultaneous activity on both strands. Furthermore, dsHMM reveals that antisense transcription is specifically suppressed by Nrd1, a yeast termination factor.

Gesture Recognition with Accelerometers for Game Controllers, Phones and Wearables

AbstractHidden Markov Models have been effectively used in time series based pattern recognition problems in the past. This work explores using Hidden Markov Models (HMM) to do 3D gesture recognition from accelerometer data. Our work differs from much of the previous work in that we examine the use of discreet HMMs rather than continuous HMMs. An interesting side effect of this is that method is therefore theoretically transportable to other devices that have a 3D sensor output system. In essence this brings us a mechanism to use the HMM model across a series of different sensor devices for gesture recognition. We achieve recognition results with accuracy rates approaching 90 percent for users who are not in the training samples. The speed of our system is also of interest as we are able to classify gestures at a rate of several hundred times per second. As long as the sen-sor system is capable of outputting information about the 3 axes of motion, and the outputs can be discretized to volumetrically equivalent cubic sub-spaces; that information can then be used in this generic model for accurate, high speed gesture recognition.

An observed sequence probability estimate in binary linear hidden Markov models with posterior inference in algebraic Bayesian networks

Hidden Markov models (HMM) and algebraic Bayesian networks (ABN) are proba-bilistic graphical models and because of that they are quit similar. HMM has wide application while ABN are not so widespread, but its instruments allow to simulate and solve hidden Markov models problems. The goal of this work is to solve hidden Markov model first problem by means of algebraic Bayesian network posterior inference. An algorithm of estimating probability of observed sequence in binary linear HMM by means of algebraic Bayesian network posterior inference.

Continuous speech recognition using linear dynamic models

Hidden Markov models (HMMs) with Gaussian mixture distributions rely on an assumption that speech features are temporally uncorrelated, and often assume a diagonal covariance matrix where correlati...

A Bayesian HMM with random effects and an unknown number of states for DNA copy number analysis

Hidden Markov models (HMMs) have been shown to be a flexible tool for modelling complex biological processes. However, choosing the number of hidden states remains an open question and the inclusion of random effects also deserves more research, as it is a recent addition to the fixed-effect HMM in many application fields. We present a Bayesian mixed HMM with an unknown number of hidden states and fixed covariates. The model is fitted using reversible-jump Markov chain Monte Carlo, avoiding the need to select the number of hidden states. We show through simulations that the estimations produced are more precise than those from a fixed-effect HMM and illustrate its practical application to the analysis of DNA copy number data, a field where HMMs are widely used.

Semiparametric Hidden Markov Models

Hidden Markov models (HMMs) are widely used for dependent data modeling. Classically, the state-dependent distributions, that is, the distribution of the observation given the hidden state, belong to some parametric family. This article proposes a semiparametric HMM for continuous observations with at least one state-dependent distribution modeled with modern nonparametric techniques, for example, nonparametric maximum likelihood estimation under shape constraints like log-concavity. The resulting model is much more flexible and avoids a model misspecification bias. Concentrating on the special case of two states, the article discusses identifiability of the model. Further, an estimation procedure for semiparametric HMMs is proposed based on the expectation–maximization algorithm, which gives also rise to model validation techniques, as demonstrated in a simulation study and an empirical illustration.

Recurrent hidden Markov models using particle swarm optimisation

Hidden Markov models (HMMs) are well known for their application in temporal pattern recognition. However, they have such problems that easy to converge to local optimal solutions and not fit well with sparsely connected time-series data. It is therefore imperative to have good methods to explore a more suitable choice, which can avoid the problems mentioned above as much as possible. We proposed a recurrent hidden Markov models (RHMM) and particle swarm optimisation (PSO) approach. Under this framework, convergence of local optimal solutions and fit with sparsely connected time-series data are solved by adjusting disturbed extremum. Furthermore, PSO is used to train the RHMM and determine the model parameters. Our approach has been applied to cursive handwriting recognition. The results demonstrate that the RHMM/PSO can provide meaningful insights for pattern recognition like fast lexicon-free recognition. It makes lexicon-free result alternatives extraction efficiently, word hypotheses evaluation quickl...