Hidden Markov Structure Research Articles

Hidden Markov Models of Evidence Accumulation in Speeded Decision Tasks

Speeded decision tasks are usually modeled within the evidence accumulation framework, enabling inferences on latent cognitive parameters, and capturing dependencies between the observed response times and accuracy. An example is the speed-accuracy trade-off, where people sacrifice speed for accuracy (or vice versa). Different views on this phenomenon lead to the idea that participants may not be able to control this trade-off on a continuum, but rather switch between distinct states (Dutilh et al., Cognitive Science 35(2):211–250, 2010). Hidden Markov models are used to account for switching between distinct states. However, combining evidence accumulation models with a hidden Markov structure is a challenging problem, as evidence accumulation models typically come with identification and computational issues that make them challenging on their own. Thus, an integration of hidden Markov models with evidence accumulation models has still remained elusive, even though such models would allow researchers to capture potential dependencies between response times and accuracy within the states, while concomitantly capturing different behavioral modes during cognitive processing. This article presents a model that uses an evidence accumulation model as part of a hidden Markov structure. This model is considered as a proof of principle that evidence accumulation models can be combined with Markov switching models. As such, the article considers a very simple case of a simplified Linear Ballistic Accumulation. An extensive simulation study was conducted to validate the model’s implementation according to principles of robust Bayesian workflow. Example reanalysis of data from Dutilh et al. (Cognitive Science 35(2):211–250, 2010) demonstrates the application of the new model. The article concludes with limitations and future extensions or alternatives to the model and its application.

Hierarchical Hidden Markov Models for Response Time Data

Psychological data, particularly measurements obtained sequentially in experiments designed to test theories of human cognition, are often treated as independent and identically distributed samples from a single distribution that describes the cognitive process. This assumption is made for mathematical and analytic convenience; it is widely appreciated that such data are in fact mixtures from two or more processes, only a subset of which are associated with the cognitive process of interest. Our modeling framework describes response times (RTs) as arising from a mixture of three distinct distributions. Transitions across the distributions are governed by a hidden Markov structure whose states produce either fast, average, or slow RTs. This process is nested within a second hidden Markov structure, producing an “environment” process that allows the distribution of the response modes to evolve due to both internal factors (such as fatigue and distractions) and external factors (such as changing task demands). We performed a detection experiment designed to elicit responses under three environments that mimic external conditions that influence latent response modes. We present our hierarchical model and demonstrate its fit on the experimental data. We also demonstrate the model’s fit in the case when external conditions were not manipulated as part of the experimental process.

Trip Purpose Identification of Docked Bike-Sharing From IC Card Data Using a Continuous Hidden Markov Model

It is different from the previous supervised learning algorithm based on personal travel questionnaire, the aim of this study is to develop an unsupervised learning methodology to estimate the docked bike-sharing users’ trip purposes using IC card data, which trip purposes were unknown from the dataset. The present study is able to extract the trip-chains, which is used to understand the complete individual trip process. A rigorous method is then proposed to interpret the purpose of each leg of the trip-chain using a continuous hidden Markov model (CHMM). This method effectively combines the Gaussian mixture model and the hidden Markov model, and realizes the inference based on trip-chains. It is intended to enhance the understanding of docked bike-sharing users’ transfer intention, which is different from most trip motivation recognition methods. The Gaussian mixture layer uses the feature space constructed by the spatial and temporal information on trip-chains from the IC card data, as well as the land-use characteristics of the docked bike-sharing docking stations to complete the transfer of the trip-chains to the trip modes. The hidden Markov structure can realize the process from the trip modes to the trip purposes. The IC card data of docked bike-sharing usage in Nanjing, China is used to interpret the specific steps of the proposed model. A questionnaire survey is conducted to obtain the real trip purposes, which is compared with the estimated results from the model to verify the effectiveness of the model. The results show that the accuracies of single trip recognition and chain trip recognition are 0.770 and 0.756, respectively. Compared with the baseline algorithm, the model also shows good performance. Therefore, the proposed approach can be used to discover and interpret the trip purpose using the IC card data.

Composite Likelihood Inference in a Discrete Latent Variable Model for Two-Way “Clustering-by-Segmentation” Problems

ABSTRACTWe consider a discrete latent variable model for two-way data arrays, which allows one to simultaneously produce clusters along one of the data dimensions (e.g., exchangeable observational units or features) and contiguous groups, or segments, along the other (e.g., consecutively ordered times or locations). The model relies on a hidden Markov structure but, given its complexity, cannot be estimated by full maximum likelihood. Therefore, we introduce a composite likelihood methodology based on considering different subsets of the data. The proposed approach is illustrated by simulation, and with an application to genomic data.

Bayesian parameter inference for individual-based models using a Particle Markov Chain Monte Carlo method

Parameter estimation for agent-based and individual-based models (ABMs/IBMs) is often performed by manual tuning and model uncertainty assessment is often ignored. Bayesian inference can jointly address these issues. However, due to high computational requirements of these models and technical difficulties in applying Bayesian inference to stochastic models, the exploration of its application to ABMs/IBMs has just started. We demonstrate the feasibility of Bayesian inference for ABMs/IBMs with a Particle Markov Chain Monte Carlo (PMCMC) algorithm developed for state-space models. The algorithm profits from the model's hidden Markov structure by jointly estimating system states and the marginal likelihood of the parameters using time-series observations. The PMCMC algorithm performed well when tested on a simple predator-prey IBM using artificial observation data. Hence, it offers the possibility for Bayesian inference for ABMs/IBMs. This can yield additional insights into model behaviour and uncertainty and extend the usefulness of ABMs/IBMs in ecological and environmental research.

Gradient-based simulated maximum likelihood estimation for stochastic volatility models using characteristic functions

Parameter estimation and statistical inference are challenging problems for stochastic volatility (SV) models, especially those driven by pure jump Lévy processes. Maximum likelihood estimation (MLE) is usually preferred when a parametric statistical model is correctly specified, but traditional MLE implementation for SV models is computationally infeasible due to high dimensionality of the integral involved. To overcome this difficulty, we propose a gradient-based simulated MLE method under the hidden Markov structure for SV models, which covers those driven by pure jump Lévy processes. Gradient estimation using characteristic functions and sequential Monte Carlo in the simulation of the hidden states are implemented. Numerical experiments illustrate the efficiency of the proposed method.

Bayesian Spatiotemporal Multitask Learning for Radar HRRP Target Recognition

A Bayesian dynamic model based on multitask learning (MTL) is developed for radar automatic target recognition (RATR) using high-resolution range profile (HRRP). The aspect-dependent HRRP sequence is modeled using a truncated stick-breaking hidden Markov model (TSB-HMM) with time-evolving transition probabilities, in which the spatial structure across range cells is described by the hidden Markov structure and the temporal dependence between HRRP samples is described by the time evolution of the transition probabilities. This framework imposes the belief that temporally proximate HRRPs are more likely to be drawn from similar HMMs, while also allowing for possible distant repetition or “innovation”. In addition, as formulated the stick-breaking prior and MTL mechanism are employed to infer the number of hidden states in an HMM and learn the target-dependent states collectively for all targets. The form of the proposed hierarchical model allows efficient variational Bayesian (VB) inference, of interest for large-scale problems. To validate the formulation, example results are presented for an illustrative synthesized dataset and our main application-RATR, for which we consider the measured HRRP data. For the latter, we also make comparisons to the model with the independent state-transition statistics and some other existing statistical models for radar HRRP data.

Statistical Alignment with a Sequence Evolution Model Allowing Rate Heterogeneity along the Sequence

We present a stochastic sequence evolution model to obtain alignments and estimate mutation rates between two homologous sequences. The model allows two possible evolutionary behaviors along a DNA sequence in order to determine conserved regions and take its heterogeneity into account. In our model, the sequence is divided into slow and fast evolution regions. The boundaries between these sections are not known. It is our aim to detect them. The evolution model is based on a fragment insertion and deletion process working on fast regions only and on a substitution process working on fast and slow regions with different rates. This model induces a pair hidden Markov structure at the level of alignments, thus making efficient statistical alignment algorithms possible. We propose two complementary estimation methods, namely, a Gibbs sampler for Bayesian estimation and a stochastic version of the EM algorithm for maximum likelihood estimation. Both algorithms involve the sampling of alignments. We propose a partial alignment sampler, which is computationally less expensive than the typical whole alignment sampler. We show the convergence of the two estimation algorithms when used with this partial sampler. Our algorithms provide consistent estimates for the mutation rates and plausible alignments and sequence segmentations on both simulated and real data.

Stochastic Modeling of Early Hematopoiesis

Abstract Hematopoiesis is the body's way of making the cellular constituents of blood. Oxygen transport, response to infections, and control of bleeding are among the functions of different mature blood cells. These specific functions are acquired as cells mature in the bone marrow. Stem cells are the “master cells” at the top of this pedigree, having within them the capacity to reconstitute the entire system. Although the latter stages of hematopoiesis are fairly well understood, the functioning of stem cells and other multipotential cells is currently a matter of intense research. This article presents a statistical analysis providing support for the clonal succession model of early hematopoiesis. J. L. Abkowitz and colleagues at the University of Washington have developed an experimental method for studying the kinetics of early hematopoiesis in a hybrid cat. The essence of the method is to analyze G6PD, an enzyme linked to the X chromosome. The G6PD type of a cell forms a binary marker that is passed down to all its descendant cells. Data record time series of proportions of one G6PD type in cells from the bone marrow, providing information about the number and lifetime of unobservable stem cells. Studies were performed after the autologous transplantation of G6PD heterozygous cats with limited numbers of hematopoietic stem cells. Preliminary analysis of the observed proportions indicates that under these circumstances, the proportion of cells with one type of G6PD is not constant over time. A simple stochastic model is used to quantify the relationship between observed proportions and unobserved stem cell populations. The model has a hidden Markov structure. We develop parameter estimates, confidence sets, and goodness-of-fit tests for this model. For our simple model, a recursive updating algorithm allows computation of the multimodal likelihood functions. A similar algorithm produces estimates of the realized Markov process. The parametric bootstrap is used to calibrate likelihood-based confidence sets and to perform simple goodness-of-fit tests. We address the question of whether stem cells have a constant proliferative potential between cats, and we discuss criticisms of the simple model.

Stochastic Modeling of Early Hematopoiesis

Abstract Hematopoiesis is the body's way of making the cellular constituents of blood. Oxygen transport, response to infections, and control of bleeding are among the functions of different mature blood cells. These specific functions are acquired as cells mature in the bone marrow. Stem cells are the “master cells” at the top of this pedigree, having within them the capacity to reconstitute the entire system. Although the latter stages of hematopoiesis are fairly well understood, the functioning of stem cells and other multipotential cells is currently a matter of intense research. This article presents a statistical analysis providing support for the clonal succession model of early hematopoiesis. J. L. Abkowitz and colleagues at the University of Washington have developed an experimental method for studying the kinetics of early hematopoiesis in a hybrid cat. The essence of the method is to analyze G6PD, an enzyme linked to the X chromosome. The G6PD type of a cell forms a binary marker that is passed down to all its descendant cells. Data record time series of proportions of one G6PD type in cells from the bone marrow, providing information about the number and lifetime of unobservable stem cells. Studies were performed after the autologous transplantation of G6PD heterozygous cats with limited numbers of hematopoietic stem cells. Preliminary analysis of the observed proportions indicates that under these circumstances, the proportion of cells with one type of G6PD is not constant over time. A simple stochastic model is used to quantify the relationship between observed proportions and unobserved stem cell populations. The model has a hidden Markov structure. We develop parameter estimates, confidence sets, and goodness-of-fit tests for this model. For our simple model, a recursive updating algorithm allows computation of the multimodal likelihood functions. A similar algorithm produces estimates of the realized Markov process. The parametric bootstrap is used to calibrate likelihood-based confidence sets and to perform simple goodness-of-fit tests. We address the question of whether stem cells have a constant proliferative potential between cats, and we discuss criticisms of the simple model.