State-dependent Distributions Research Articles

Hidden Markov and Semi-Markov Models When and Why are These Models Useful for Classifying States in Time Series Data?

Hidden Markov models (HMMs) and their extensions have proven to be powerful tools for classification of observations that stem from systems with temporal dependence as they take into account that observations close in time are likely generated from the same state (i.e., class). When information on the classes of the observations is available in advanced, supervised methods can be applied. In this paper, we provide details for the implementation of four models for classification in a supervised learning context: HMMs, hidden semi-Markov models (HSMMs), autoregressive-HMMs, and autoregressive-HSMMs. Using simulations, we study the classification performance under various degrees of model misspecification to characterize when it would be important to extend a basic HMM to an HSMM. As an application of these techniques we use the models to classify accelerometer data from Merino sheep to distinguish between four different behaviors of interest. In particular in the field of movement ecology, collection of fine-scale animal movement data over time to identify behavioral states has become ubiquitous, necessitating models that can account for the dependence structure in the data. We demonstrate that when the aim is to conduct classification, various degrees of model misspecification of the proposed model may not impede good classification performance unless there is high overlap between the state-dependent distributions, that is, unless the observation distributions of the different states are difficult to differentiate. Supplementary materials accompanying this paper appear on-line.

A copula-based multivariate hidden Markov model for modelling momentum in football

We investigate the potential occurrence of change points—commonly referred to as “momentum shifts”—in the dynamics of football matches. For that purpose, we model minute-by-minute in-game statistics of Bundesliga matches using hidden Markov models (HMMs). To allow for within-state dependence of the variables, we formulate multivariate state-dependent distributions using copulas. For the Bundesliga data considered, we find that the fitted HMMs comprise states which can be interpreted as a team showing different levels of control over a match. Our modelling framework enables inference related to causes of momentum shifts and team tactics, which is of much interest to managers, bookmakers, and sports fans.

Penalized estimation of flexible hidden Markov models for time series of counts

We propose an effectively nonparametric approach to fitting hidden Markov models to time series of counts, where the state-dependent distributions are estimated in a completely data-driven way without the need to specify a parametric family of distributions. To avoid overfitting, a roughness penalty based on higher-order differences between adjacent count probabilities is added to the likelihood, which is demonstrated to produce smooth state-dependent probability mass functions. The feasibility of the suggested approach is assessed in simulation experiments, and further illustrated in two real-data applications, where we model the distributions of (i) major earthquake counts and (ii) acceleration counts of an oceanic whitetip shark (Carcharhinus longimanus) over time. The proposed methodology is implemented in the accompanying R package countHMM, which is available on CRAN.

A Time-Series Model for Underdispersed or Overdispersed Counts

ABSTRACTIt is common for time series of unbounded counts (that is, nonnegative integers) to display overdispersion relative to the Poisson. Such an overdispersed series can be modeled by a hidden Markov model with Poisson state-dependent distributions (a “Poisson–HMM”), since a Poisson–HMM allows for both overdispersion and serial dependence. Time series of underdispersed counts seems less common, but more awkward to model; a Poisson–HMM cannot cope with underdispersion. But if in a Poisson–HMM one replaces the Poisson distributions by Conway–Maxwell–Poisson distributions, one gets a class of models which can allow for under- or overdispersion (and serial dependence). In addition, this class can cope with the combination of slight overdispersion and substantial serial dependence, a combination that is apparently difficult for a Poisson–HMM to represent. We discuss the properties of this class of models, and use direct numerical maximization of likelihood to fit a range of models to three published series of counts which display underdispersion, and to a series which displays slight overdispersion plus substantial serial dependence. In addition, we illustrate how such models can be fitted without imputation when some observations are missing from the series, and how approximate standard errors of the parameter estimates can be found. Supplementary materials for this article are available online.

Hidden Markov models reveal complexity in the diving behaviour of short-finned pilot whales

Diving behaviour of short-finned pilot whales is often described by two states; deep foraging and shallow, non-foraging dives. However, this simple classification system ignores much of the variation that occurs during subsurface periods. We used multi-state hidden Markov models (HMM) to characterize states of diving behaviour and the transitions between states in short-finned pilot whales. We used three parameters (number of buzzes, maximum dive depth and duration) measured in 259 dives by digital acoustic recording tags (DTAGs) deployed on 20 individual whales off Cape Hatteras, North Carolina, USA. The HMM identified a four-state model as the best descriptor of diving behaviour. The state-dependent distributions for the diving parameters showed variation between states, indicative of different diving behaviours. Transition probabilities were considerably higher for state persistence than state switching, indicating that dive types occurred in bouts. Our results indicate that subsurface behaviour in short-finned pilot whales is more complex than a simple dichotomy of deep and shallow diving states, and labelling all subsurface behaviour as deep dives or shallow dives discounts a significant amount of important variation. We discuss potential drivers of these patterns, including variation in foraging success, prey availability and selection, bathymetry, physiological constraints and socially mediated behaviour.

Nonparametric identification and maximum likelihood estimation for hidden Markov models.

Nonparametric identification and maximum likelihood estimation for finite-state hidden Markov models are investigated. We obtain identification of the parameters as well as the order of the Markov chain if the transition probability matrices have full-rank and are ergodic, and if the state-dependent distributions are all distinct, but not necessarily linearly independent. Based on this identification result, we develop a nonparametric maximum likelihood estimation theory. First, we show that the asymptotic contrast, the Kullback-Leibler divergence of the hidden Markov model, also identifies the true parameter vector nonparametrically. Second, for classes of state-dependent densities which are arbitrary mixtures of a parametric family, we establish the consistency of the nonparametric maximum likelihood estimator. Here, identification of the mixing distributions need not be assumed. Numerical properties of the estimates and of nonparametric goodness of fit tests are investigated in a simulation study.

Nonparametric inference in hidden Markov models using P-splines.

Hidden Markov models (HMMs) are flexible time series models in which the distribution of the observations depends on unobserved serially correlated states. The state-dependent distributions in HMMs are usually taken from some class of parametrically specified distributions. The choice of this class can be difficult, and an unfortunate choice can have serious consequences for example on state estimates, and more generally on the resulting model complexity and interpretation. We demonstrate these practical issues in a real data application concerned with vertical speeds of a diving beaked whale, where we demonstrate that parametric approaches can easily lead to overly complex state processes, impeding meaningful biological inference. In contrast, for the dive data, HMMs with nonparametrically estimated state-dependent distributions are much more parsimonious in terms of the number of states and easier to interpret, while fitting the data equally well. Our nonparametric estimation approach is based on the idea of representing the densities of the state-dependent distributions as linear combinations of a large number of standardized B-spline basis functions, imposing a penalty term on non-smoothness in order to maintain a good balance between goodness-of-fit and smoothness.

Hidden Markov models with state-dependent mixtures: minimal representation, model testing and applications to clustering

Finite-state hidden Markov models (HMMs), also called Markov-dependent finite mixtures, form a popular, frequently used model class for serially dependent observations with unobserved heterogeneity. We consider HMMs in which the state-dependent distributions are themselves finite mixtures. In such models, the parametrization is not unique, since components from the state-dependent mixtures may also be represented as states in the underlying Markov chain. We determine a unique (up to label switching) representation for the HMM in which the Markov chain has a minimal number of states. Further, we propose a likelihood-ratio test for the hypothesis that the number of states in the Markov chain can be reduced without changing the distribution of the time-series model. Our method has important applications in cluster analysis and model selection. After highlighting the relevance of serial dependence for clustering, we propose two-step clustering algorithms. Starting with a BIC choice for a standard HMM (with simple state-dependent distributions), in the first step we determine the minimal representation of the HMM by testing, and in the second step we merge components in the resulting state-dependent finite mixtures by using either a local entropy or a modality-based criterion. The states in the resulting Markov chain, potentially split according to the remaining state-dependent components, are then interpreted as clusters. For model selection, we illustrate our method on a series of logarithmic returns of gold prices using normal HMMs. The AIC choice is a six-state HMM, while the BIC choice has four states. When starting with the AIC choice, successive testing results in a four-state Markov chain, with two state-dependent distributions consisting of two-component normal mixtures.

Understanding Fiscal Limits and Debt in the Developing Economies of Central America and the Caribbean.

This study uses simulations of state dependent distributions of fiscal limits for eighteen (18) economies within Central America and the Caribbean to better understand governments’ ability to service its debt, arising from endogenously determined dynamic Laffer curves. Using a small open economy model to simulate macroeconomic fundamentals and fiscal policy interactions, the empirical findings produced results not previous available for these economies, showing varying and wider distribution of fiscal limits for the open economy model subject to term of trade shocks, indicating that terms of trade volatility impacted the ability of state economies in servicing their debt. It is therefore prudent that policymakers and central bankers consider models that incorporate the use of trade and its volatility, as a robust way of more accurately determining fiscal limits which are a critical component in better understanding governments’ ability to service its debt.

Fiscal Limits, External Debt, and Fiscal Policy in Developing Countries

This paper studies fiscal policy effects in developing countries with external debt and sovereign default risks. State-dependent distributions of fiscal limits are simulated based on macroeconomic uncertainty and fiscal policy specifications. The analysis shows that expected future revenue plays an important role in the low fiscal limits of developing countries, relative to those of developed countries. External debt carries additional risks since large devaluation of the real exchange rate can suddenly raise default probabilities. Consistent with majority views, fiscal consolidations are counterproductive in the short and medium runs. When an economy approaches its fiscal limits, government spending can be less expansionary than in a low-debt state. As more revenue is required to service debt in a high-debt state, higher tax rates raise the economic cost of increasing consumption, reducing the fiscal multiplier.

Fiscal Limits, External Debt, and Fiscal Policy in Developing Countries

This paper studies fiscal policy effects in developing countries with external debt and sovereign default risks. State-dependent distributions of fiscal limits are simulated based on macroeconomic uncertainty and fiscal policy specifications. The analysis shows that expected future revenue plays an important role in the low fiscal limits of developing countries, relative to those of developed countries. External debt carries additional risks since large devaluation of the real exchange rate can suddenly raise default probabilities. Consistent with majority views, fiscal consolidations are counterproductive in the short and medium runs. When an economy approaches its fiscal limits, government spending can be less expansionary than in a low-debt state. As more revenue is required to service debt in a high-debt state, higher tax rates raise the economic cost of increasing consumption, reducing the fiscal multiplier.

PC-SAFT parameters from ab initio calculations

We use highly accurate ab initio calculations of binding enthalpies and entropies of gas phase clusters of alcohols to demonstrate how they can be used to obtain association parameters for PC-SAFT. The thermochemical results demonstrate that cooperativity effects and state dependent cluster distributions cause a strongly varying average enthalpy and entropy per bond as function of temperature and density for alcohols. In contrast to this, the two association parameters of PC-SAFT lead to density independent bond enthalpy and entropy and are thus effective parameters. Therefore, we choose to compute the cluster distribution at a universal state point and show that the thus obtained association parameters can be used to reduce the number of adjustable parameters from 5 to 3 with only a marginal loss of accuracy for most of the studied systems, and even an estimation of thermodynamic properties without adjusted parameters is possible. The ab initio calculations suggest that the 2B association scheme is more appropriate for 1-alkanols than the 3B one.

Mixture distributions for modelling demand during lead time

A mixture distribution approach to modelling demand during lead time in a continuous-review inventory model is described. Using this approach, both lead time and demand per unit time can follow state-dependent distributions. By using mixtures of truncated exponentials functions to approximate these distributions, mixture distributions that can be easily manipulated in closed form can be constructed as the marginal distributions for lead time and demand per unit time. These are then used to approximate the mixture of compound distributions for demand during lead time. The technique is illustrated by first applying it to a ‘normal-gamma’ inventory problem, then by modelling a problem with empirical distributions for lead time and demand per unit time.

Semiparametric Hidden Markov Models

Hidden Markov models (HMMs) are widely used for dependent data modeling. Classically, the state-dependent distributions, that is, the distribution of the observation given the hidden state, belong to some parametric family. This article proposes a semiparametric HMM for continuous observations with at least one state-dependent distribution modeled with modern nonparametric techniques, for example, nonparametric maximum likelihood estimation under shape constraints like log-concavity. The resulting model is much more flexible and avoids a model misspecification bias. Concentrating on the special case of two states, the article discusses identifiability of the model. Further, an estimation procedure for semiparametric HMMs is proposed based on the expectation–maximization algorithm, which gives also rise to model validation techniques, as demonstrated in a simulation study and an empirical illustration.

Sovereign default risk premia, fiscal limits, and fiscal policy

We develop a closed economy model to study the interactions among sovereign risk premia, fiscal limits, and fiscal policy. The fiscal limits, which measure the government's ability to service its debt, arise endogenously from dynamic Laffer curves. The state-dependent distributions of fiscal limits depend on the growth of lump-sum transfers, the size of the government, the degree of countercyclical policy responses, and economic diversity. The country-specific fiscal limits imply that the market perceives the riskiness of sovereign debt issued by different countries to be different, which is consistent with the observation that developed countries are downgraded at different levels of debt. A nonlinear relationship between sovereign risk premia and the level of government debt emerges in equilibrium, which is in line with the empirical evidence that once risk premia begin to rise, they do so rapidly. Nonlinear simulations show that fiscal austerity measures that aim to balance the government budget in the short run fail to contain the default risk premium, even with sizeable cuts in government purchases; but a long-term plan for fiscal reform, if it credibly changes the market's expectation about future fiscal policies, can alleviate the rising risk premium.

Hitting Times and the Running Maximum of Markovian Growth-Collapse Processes

We consider the level hitting times τy= inf{t≥ 0 |Xt=y} and the running maximum processMt= sup{Xs| 0 ≤s≤t} of a growth-collapse process (Xt)t≥0, defined as a [0, ∞)-valued Markov process that grows linearly between random ‘collapse’ times at which downward jumps with state-dependent distributions occur. We show how the moments and the Laplace transform of τycan be determined in terms of the extended generator ofXtand give a power series expansion of the reciprocal of Ee−sτy. We prove asymptotic results for τyandMt: for example, ifm(y) = Eτyis of rapid variation thenMt/m-1(t) →w1 ast→ ∞, wherem-1is the inverse function ofm, while ifm(y) is of regular variation with indexa∈ (0, ∞) andXtis ergodic, thenMt/m-1(t) converges weakly to a Fréchet distribution with exponenta. In several special cases we provide explicit formulae.

Hitting Times and the Running Maximum of Markovian Growth-Collapse Processes

We consider the level hitting times τy = inf{t ≥ 0 | Xt = y} and the running maximum process Mt = sup{Xs | 0 ≤ s ≤ t} of a growth-collapse process (Xt)t≥0, defined as a [0, ∞)-valued Markov process that grows linearly between random ‘collapse’ times at which downward jumps with state-dependent distributions occur. We show how the moments and the Laplace transform of τy can be determined in terms of the extended generator of Xt and give a power series expansion of the reciprocal of Ee−sτy. We prove asymptotic results for τy and Mt: for example, if m(y) = Eτy is of rapid variation then Mt / m-1(t) →w 1 as t → ∞, where m-1 is the inverse function of m, while if m(y) is of regular variation with index a ∈ (0, ∞) and Xt is ergodic, then Mt / m-1(t) converges weakly to a Fréchet distribution with exponent a. In several special cases we provide explicit formulae.

The likelihood ratio test for hidden Markov models in two-sample problems

The asymptotic distribution of the likelihood ratio test statistic in two-sample testing problems for hidden Markov models is derived when allowing for unequal sample sizes as well as for different families of state-dependent distributions. In both cases under regularity conditions the limit distribution is a standard χ 2 -distribution, and in particular does not depend on the ratio of the distinct sample sizes. In a simulation study, the finite sample properties are investigated, and the methodology is illustrated in an application to modeling the movement of Drosophila larvae.

Subspace constrained Gaussian mixture models for speech recognition

A standard approach to automatic speech recognition uses hidden Markov models whose state dependent distributions are Gaussian mixture models. Each Gaussian can be viewed as an exponential model whose features are linear and quadratic monomials in the acoustic vector. We consider here models in which the weight vectors of these exponential models are constrained to lie in an affine subspace shared by all the Gaussians. This class of models includes Gaussian models with linear constraints placed on the precision (inverse covariance) matrices (such as diagonal covariance, maximum likelihood linear transformation, or extended maximum likelihood linear transformation), as well as the LDA/HLDA models used for feature selection which tie the part of the Gaussians in the directions not used for discrimination. In this paper, we present algorithms for training these models using a maximum likelihood criterion. We present experiments on both small vocabulary, resource constrained, grammar-based tasks, as well as large vocabulary, unconstrained resource tasks to explore the rather large parameter space of models that fit within our framework. In particular, we demonstrate significant improvements can be obtained in both word error rate and computational complexity.

Recurrent Inhibitory Dynamics: The Role of State-Dependent Distributions of Conduction Delay Times

We have formulated and analysed a dynamic model for recurrent inhibition that takes into account the state dependence of the delayed feedback signal (due to the variation in threshold of fibres with their size) and the distribution of these delays (due to the distribution of fibre diameters in the feedback pathway). Using a combination of analytic and numerical tools, we have analysed the behaviour of this model. Depending on the parameter values chosen, as well as the initial preparation of the system, there may be a spectrum of post-synaptic firing dynamics ranging from stable constant values through periodic bursting (limit cycle) behaviour and chaotic firing as well as bistable behaviours. Using detailed parameter estimation for a physiologically motivated example (the CA3-basket cell-mossy fibre system in the hippocampus), we present some of these numerical behaviours. The numerical results corroborate the results of the analytic characterization of the solutions. Namely, for some parameter values the model has a single stable steady state while for the others there is a bistability in which the eventual behaviour depends on the magnitude of stimulation (the initial function).