Mixture Transition Distribution Model Research Articles

General framework and model building in the class of Hidden Mixture Transition Distribution models

Modeling time series that present non-Gaussian features plays as central role in many fields, including finance, seismology, psychological, and life course studies. The Hidden Mixture Transition Distribution model is an answer to the complexity of such series. The observed heterogeneity can be induced by one or several latent factors, and each level of these factors is related to a different component of the observed process. The time series is then treated as a mixture and the relation between the components is governed by a Markovian latent transition process. This framework generalizes several specifications that appear separately in related literature. Both the expectation and the standard deviation of each component are allowed to be functions of the past of the process. The latent process can be of any order, and can be modeled using a discrete Mixture Transition Distribution. The effects of covariates at the visible and hidden levels are also investigated. One of the main difficulties lies in correctly specifying the structure of the model. Therefore, we propose a hierarchical model selection procedure that exploits the multilevel structure of our approach. Finally, we illustrate the model and the model selection procedure through a real application in social science.

A New Model for Multivariate Markov Chains

ABSTRACTWe propose a new model for multivariate Markov chains of order one or higher on the basis of the mixture transition distribution (MTD) model. We call it the MTD‐Probit. The proposed model presents two attractive features: it is completely free of constraints, thereby facilitating the estimation procedure, and it is more precise at estimating the transition probabilities of a multivariate or higher‐order Markov chain than the standard MTD model.

MTD models for aggregate data from higher order Markov chains

We consider two higher order models for aggregate data on a finite state space. In the first model, aggregate data are obtained from N i.i.d. individuals who follow Mixture Transition Distribution (MTD) Markov model of lag l . In the second model, aggregate data are modeled as a MTD Markov model based on multinomial thinning. In both the cases, it is shown that Conditional Least Square Estimators are CAN for a fixed N .

Flexible Latent‐State Modelling of Old Faithful's Eruption Inter‐Arrival Times in 2009

SummaryThis paper is concerned with the analysis of a time series comprising the eruption inter‐arrival times of the Old Faithful geyser in 2009. The series is much longer than other well‐documented ones and thus gives a more comprehensive insight into the dynamics of the geyser. Basic hidden Markov models with gamma state‐dependent distributions and several extensions are implemented. In order to better capture the stochastic dynamics exhibited by Old Faithful, the different non‐standard models under consideration seek to increase the flexibility of the basic models in various ways: (i) by allowing non‐geometric distributions for the times spent in the different states; (ii) by increasing the memory of the underlying Markov chain, with or without assuming additional structure implied by mixture transition distribution models; and (iii) by incorporating feedback from the observation process on the latent process. In each case it is shown how the likelihood can be formulated as a matrix product which can be conveniently maximized numerically.

Geometry of Higher-Order Markov Chains

We determine an explicit Grobner basis, consisting of linear forms and determinantalquadrics, for the prime ideal of Raftery's mixture transition distribution model for Markov chains.When the states are binary, the corresponding projective variety is a linear space, the model itselfconsists of two simplices in a cross-polytope, and the likelihood function typically has two localmaxima. In the general non-binary case, the model corresponds to a cone over a Segre variety.

A note on the mixture transition distribution and hidden Markov models

We discuss an interpretation of the mixture transition distribution (MTD) for discrete-valued time series which is based on a sequence of independent latent variables which are occasion-specific. We show that, by assuming that this latent process follows a first order Markov Chain, MTD can be generalized in a sensible way. A class of models results which also includes the hidden Markov model (HMM). For these models we outline an EM algorithm for the maximum likelihood estimation which exploits recursions developed within the HMM literature. As an illustration, we provide an example based on the analysis of stock market data referred to different American countries.

Parameter estimation of the WMTD model

The MTD (mixture transition distribution) model based on Weibull distribution (WMTD model) is proposed in this paper, which is aimed at its parameter estimation. An EM algorithm for estimation is given and shown to work well by some simulations. And bootstrap method is used to obtain confidence regions for the parameters. Finally, the results of a real example—predicting stock prices—show that the WMTD model proposed is able to capture the features of the data from thick-tailed distribution better than GMTD (mixture transition distribution) model.

$\boldsymbol{\mathcal{G}-}$ Inhomogeneous Markov Systems of High Order

In the present, we introduce and study the \(\mathcal{G-}\)inhomogeneous Markov system of high order, which is a more general in many respects stochastic process than the known inhomogeneous Markov system. We define the inhomogeneous superficial razor cut mixture transition distribution model extending for the homogeneous case the idea of the mixture transition model. With the introduction of the appropriate vector stochastic process and the establishment of relationships among them, we study the asymptotic behaviour of the \(\mathcal{G-}\)inhomogeneous Markov system of high order. In the form of two theorems, the asymptotic behaviour of the inherent \(\mathcal{G-}\)inhomogeneous Markov chain and the expected and relative expected population structure of the \(\mathcal{G-}\)inhomogeneous Markov system of high order, are provided under assumptions easily met in practice. Finally, we provide an illustration of the present results in a manpower system.

A Novel Estimation Approach for Mixture Transition Distribution Model in High-Order Markov Chains

A transformation is proposed to convert the nonlinear constraints of the parameters in the mixture transition distribution (MTD) model into box-constraints. The proposed transformation removes the difficulties associated with the maximum likelihood estimation (MLE) process in the MTD modeling so that the MLEs of the parameters can be easily obtained via a hybrid algorithm from the evolutionary algorithms and/or quasi-Newton algorithms for global optimization. Simulation studies are conducted to demonstrate MTD modeling by the proposed novel approach through a global search algorithm in R environment. Finally, the proposed approach is used for the MTD modelings of three real data sets.

An EM algorithm for estimation in the mixture transition distribution model

The mixture transition distribution (MTD) model was introduced by Raftery to face the need for parsimony in the modeling of high-order Markov chains in discrete time. The particularity of this model comes from the fact that the effect of each lag upon the present is considered separately and additively, so that the number of parameters required is drastically reduced. However, the efficiency for the MTD parameter estimations proposed up to date still remains problematic on account of the large number of constraints on the parameters. In this article, an iterative procedure, commonly known as expectation–maximization (EM) algorithm, is developed cooperating with the principle of maximum likelihood estimation (MLE) to estimate the MTD parameters. Some applications of modeling MTD show the proposed EM algorithm is easier to be used than the algorithm developed by Berchtold. Moreover, the EM estimations of parameters for high-order MTD models led on DNA sequences outperform the corresponding fully parametrized Markov chain in terms of Bayesian information criterion. A software implementation of our algorithm is available in the library seq++at http://stat.genopole.cnrs.fr/seqpp.

Heteroscedastic mixture transition distribution (HMTD) model

The paper further studies the heteroscedastic mixture transition distribution (HMTD) model introduced by Berchtold. Both the expectation and the standard deviation of each component are written as functions of the past of the process. The stationarity conditions are derived. An expectation conditional maximization (ECM) algorithm is used and shown to work well for estimation, the model selection problem is addressed, and the formulaes for computing the observed information matrix are derived. The shape changing feature of conditional distributions makes the model capable of modelling time series with asymmetric or multimodal distribution. The model is applied to several simulated and real datasets with satisfactory results.

Stationary mixture transition distribution (MTD) models via predictive distributions

This paper combines two ideas to construct autoregressive processes of arbitrary order. The first idea is the construction of first order stationary processes described in Pitt et al. [(2002). Constructing first order autoregressive models via latent processes. Scand. J. Statist. 29, 657–663] and the second idea is the construction of higher order processes described in Raftery [(1985). A model for high order Markov chains. J. Roy. Statist. Soc. B. 47, 528–539]. The resulting models provide appealing alternatives to model non-linear and non-Gaussian time series.

Modeling Marked Point Processes via Bivariate Mixture Transition Distribution Models

We propose new probability models for the analysis of marked point processes. These models deal with the type of data that arrive or are observed in possibly unequal time intervals, such as financial transactions and earthquakes, among others. The models treat both the time between event arrivals and the observed marks as stochastic processes. We adopt a class of bivariate distributions to form the bivariate mixture transition distribution. In these models the conditional bivariate distribution of the next observation given the past is a mixture of conditional distributions given each one of the last p observations or a selection of past p events. The identifiability of the model is investigated, and an EM algorithm is developed to obtain estimates of the model parameters. Simulation and real data examples are used to demonstrate the utility of these models.

Investigating purchasing-sequence patterns for financial services using Markov, MTD and MTDg models

In the past, several authors have found evidence for the existence of a priority pattern of acquisition for durable goods, as well as for financial services. Its usefulness lies in the fact that if the position of a particular customer in this acquisition sequence is known, one can predict what service will be acquired next by that customer. In this paper, we analyse purchase sequences of financial services to identify cross-buying patterns, which might be used to discover cross-selling opportunities as part of customer relationship management (CRM). Hereby, special attention is paid to transitions, which might encourage bank-only or insurance-only customers to become financial-services customers. We introduce the mixture transition distribution (MTD) model as a parsimonious alternative to the Markov model for use in the analysis of marketing problems. An interesting extension on the MTD model is the MTDg model, which is able to represent situations where the relationship between each lag and the current state differs. We illustrate the MTD and MTDg model on acquisition sequences of customers of a major financial-services company and compare the fit of these models with that of the corresponding Markov model. Our results are in favor of the MTD and MTDg models. Therefore, the MTD as well as the MTDg transition matrices are investigated to reveal cross-buying patterns. The results are valuable to product managers as they clarify the customer flows among product groups. In some cases, the lag-specific transition matrices of the MTDg model give better insight into the acquisition patterns than the general transition matrix of the MTD model.

YAKUSA: A fast structural database scanning method

YAKUSA is a program designed for rapid scanning of a structural database with a query protein structure. It searches for the longest common substructures called SHSPs (structural high-scoring pairs) existing between a query structure and every structure in the structural database. It makes use of protein backbone internal coordinates (alpha angles) in order to describe protein structures as sequences of symbols. The structural similarities are established in 5 steps, the first 3 being analogous to those used in BLAST: (1) building up a deterministic finite automaton describing all patterns identical or similar to those in the query structure; (2) searching for all these patterns in every structure in the database; (3) extending the patterns to longer matching substructures (i.e., SHSPs); (4) selecting compatible SHSPs for each query-database structure pair; and (5) ranking the query-database structure pairs using 3 scores based on SHSP similarity, on SHSP probabilities, and on spatial compatibility of SHSPs. Structural fragment probabilities are estimated according to a mixture transition distribution model, which is an approximation of a high-order Markov chain model. With regard to sensitivity and selectivity of the structural matches, YAKUSA compares well to the best related programs, although it is by far faster: A typical database scan takes about 40 s CPU time on a desktop personal computer. It has also been implemented on a Web server for real-time searches.

Mixture Nonlinear Time-Series Analysis : Modelling and Forecasting

Gaussian mixture transition distribution (GMTD) models and mixture autoregressive (MAR) models are generally employed to describe those data sets that depict sudden bursts, outliers and flat stretches at irregular time epochs. In this paper , these two approaches are compared by considering weekly wholesale onion price data during April, 1998 to November, 2001. After eliminating trend, seasonal fluctuations are studied by fitting Box­Jenkins airline model to residual series. To this end, null hypothesis of presence of nonseasonal and seasonal stochastic trends is tested by using Osboru­Chui­Smith­Birchenhall (OCSB) auxiliary regression. Subsequently, appropriate filters in airline model for seasonal fluctuations are selected. Presence of autoregressive co nditional heteroscedasticity (ARCH) is tested by Naive Lagrange multiplier (Nave­ LM) test. Estimation of parameters is carric~d out using Expectation­Maximization (EM) algorithm and the best model is selected on the basis of Bayesian information criterion (BIC). Out­of­sample forecasting is performed for one­step and two­step ahead prediction by uaive approach, proposed by Wong and Li (2000). It is concluded that, for data under consideration, a three­component MAR model performs the best.

Regression Theory for Categorical Time Series

Categorical---or qualitative---time series data with random time-dependent covariates are frequently encountered in diverse applications as the list of examples shows. As with "ordinary'' time series, the data analyst is faced with the same problems of modeling, estimation, model checking, diagnostics and prediction. The present work shows that these questions can be attacked by means of regression theory for categorical time series whose foundation is based on generalized linear models and partial likelihood inference. A variety of models are provided to illustrate the selection of the link function and recent large sample results are reviewed. The theory is developed without resorting to the Markov assumption and to the notion of stationarity. Moreover, regression methods for categorical time series allow for parsimonious modeling and incorporation of random time-dependent covariates as opposed to other procedures. In particular, nominal and ordinal time series are analyzed and compared empirically to Markov chains and mixture transition distribution models.

Mixture transition distribution (MTD) modeling of heteroscedastic time series

Time series presenting non-Gaussian features such as heteroscedasticity or sudden bursts of activity play a central role in many fields including finance, insurance, and seismology. The heteroscedastic mixture transition distribution (HMTD) model, which generalizes several specifications previously proposed in the statistical literature, is a new model especially designed to handle series of this kind. By allowing the standard deviation of each component to be a function of the past of the observed process, a better modeling of the conditional probability distribution function of future observations is obtained. A numerical example shows that the HMTD can perform better than standard models such as ARMA and GARCH. Different issues related to the numerical estimation of mixture models are also discussed.

The Mixture Transition Distribution Model for High-Order Markov Chains and Non-Gaussian Time Series

The mixture transition distribution model (MTD) was introduced in 1985 by Raftery for the modeling of high-order Markov chains with a finite state space. Since then it has been generalized and successfully applied to a range of situations, including the analysis of wind directions, DNA sequences and social behavior. Here we review the MTD model and the developments since 1985. We first introduce the basic principle and then we present several extensions, including general state spaces and spatial statistics. Following that, we review methods for estimating the model parameters. Finally, a review of different types of applications shows the practical interest of the MTD model.

High-order extensions of the Double Chain Markov Model

The Double Chain Markov Model is a fully Markovian model for the representation of time-series in random environments. In this article, we show that it can handle transitions of high-order between both a set of observations and a set of hidden states. In order to reduce the number of parameters, each transition matrix can be replaced by a Mixture Transition Distribution model. We provide a complete derivation of the algorithms needed to compute the model. Three applications, the analysis of a sequence of DNA, the song of the wood pewee, and the behavior of young monkeys show that this model is of great interest for the representation of data that can be decomposed into a finite set of patterns.