Recurrent Markov Chains Research Articles

Potential analysis for positive recurrent Markov chains with asymptotically zero drift: Power-type asymptotics

We consider a positive recurrent Markov chain on R+ with asymptotically zero drift which behaves like −c1/x at infinity; this model was first considered by Lamperti. We are interested in tail asymptotics for the stationary measure. Our analysis is based on construction of a harmonic function which turns out to be regularly varying at infinity. This harmonic function allows us to perform non-exponential change of measure. Under this new measure Markov chain is transient with drift like c2/x at infinity and we compute the asymptotics for its Green function. Applying further the inverse transform of measure we deduce a power-like asymptotic behaviour of the stationary tail distribution. Such a heavy-tailed stationary measure happens even if the jumps of the chain are bounded. This model provides an example where possibly bounded input distributions produce non-exponential output.

Estimation in threshold autoregressive models with a stationary and a unit root regime

This paper treats estimation in a class of new nonlinear threshold autoregressive models with both a stationary and a unit root regime. Existing literature on nonstationary threshold models has basically focused on models where the nonstationarity can be removed by differencing and/or where the threshold variable is stationary. This is not the case for the process we consider, and nonstandard estimation problems are the result.This paper proposes a parameter estimation method for such nonlinear threshold autoregressive models using the theory of null recurrent Markov chains. Under certain assumptions, we show that the ordinary least squares (OLS) estimators of the parameters involved are asymptotically consistent. Furthermore, it can be shown that the OLS estimator of the coefficient parameter involved in the stationary regime can still be asymptotically normal while the OLS estimator of the coefficient parameter involved in the nonstationary regime has a nonstandard asymptotic distribution. In the limit, the rate of convergence in the stationary regime is asymptotically proportional to n−14, whereas it is n−1 in the nonstationary regime. The proposed theory and estimation method are illustrated by both simulated data and a real data example.

Asymptotic properties of covariate-adaptive randomization

Balancing treatment allocation for influential covariates is critical in clinical trials. This has become increasingly important as more and more biomarkers are found to be associated with different diseases in translational research (genomics, proteomics and metabolomics). Stratified permuted block randomization and minimization methods [Pocock and Simon Biometrics 31 (1975) 103-115, etc.] are the two most popular approaches in practice. However, stratified permuted block randomization fails to achieve good overall balance when the number of strata is large, whereas traditional minimization methods also suffer from the potential drawback of large within-stratum imbalances. Moreover, the theoretical bases of minimization methods remain largely elusive. In this paper, we propose a new covariate-adaptive design that is able to control various types of imbalances. We show that the joint process of within-stratum imbalances is a positive recurrent Markov chain under certain conditions. Therefore, this new procedure yields more balanced allocation. The advantages of the proposed procedure are also demonstrated by extensive simulation studies. Our work provides a theoretical tool for future research in this area.

Estimation in semi-parametric regression with non-stationary regressors

In this paper, we consider a partially linear model of the form $Y_t=X_t^{\tau}\theta_0+g(V_t)+\epsilon_t$, $t=1,...,n$, where $\{V_t\}$ is a $\beta$ null recurrent Markov chain, $\{X_t\}$ is a sequence of either strictly stationary or non-stationary regressors and $\{\epsilon_t\}$ is a stationary sequence. We propose to estimate both $\theta_0$ and $g(\cdot)$ by a semi-parametric least-squares (SLS) estimation method. Under certain conditions, we then show that the proposed SLS estimator of $\theta_0$ is still asymptotically normal with the same rate as for the case of stationary time series. In addition, we also establish an asymptotic distribution for the nonparametric estimator of the function $g(\cdot)$. Some numerical examples are provided to show that our theory and estimation method work well in practice.

Almost sure central limit theorem for partial sums of Markov chain

The authors prove an almost sure central limit theorem for partial sums based on an irreducible and positive recurrent Markov chain using logarithmic means, which realizes the extension of the almost sure central limit theorem for partial sums from an i.i.d. sequence of random variables to a Markov chain.

Regenerative block empirical likelihood for Markov chains

Empirical likelihood (EL) is a powerful semi-parametric method increasingly investigated in the literature. However, most authors essentially focus on an i.i.d. setting. In the case of dependent data, the classical EL method cannot be directly applied on the data but rather on blocks of consecutive data catching the dependence structure. Generalisation of EL based on the construction of blocks of increasing random length have been proposed for time series satisfying mixing conditions. Following some recent developments in the bootstrap literature, we propose a generalisation for a large class of Markov chains, based on small blocks of various lengths. Our approach makes use of the regenerative structure of Markov chains, which allows us to construct blocks which are almost independent (independent in the atomic case). We obtain the asymptotic validity of the method for positive recurrent Markov chains and present some simulation results.

NULL RECURRENT UNIT ROOT PROCESSES

The classical nonstationary autoregressive models are both linear and Markov. They include unit root and cointegration models. A possible nonlinear extension is to relax the linearity and at the same time keep general properties such as nonstationarity and the Markov property. A null recurrent Markov chain is nonstationary, andβ-null recurrence is of vital importance for statistical inference in nonstationary Markov models, such as, e.g., in nonparametric estimation in nonlinear cointegration within the Markov models. The standard random walk is an example of a null recurrent Markov chain.In this paper we suggest that the concept of null recurrence is an appropriate nonlinear generalization of the linear unit root concept and as such it may be a starting point for a nonlinear cointegration concept within the Markov framework. In fact, we establish the link between null recurrent processes and autoregressive unit root models. It turns out that null recurrence is closely related to the location of the roots of the characteristic polynomial of the state space matrix and the associated eigenvectors. Roughly speaking the process isβ-null recurrent if one root is on the unit circle, null recurrent if two distinct roots are on the unit circle, whereas the others are inside the unit circle. It is transient if there are more than two roots on the unit circle. These results are closely connected to the random walk being null recurrent in one and two dimensions but transient in three dimensions. We also give an example of a process that by appropriate adjustments can be madeβ-null recurrent for anyβ∈ (0, 1) and can also be made null recurrent without beingβ-null recurrent.

Asymptotic Optimality of Isoperimetric Constants

In this paper, we investigate the existence of L 2(π)-spectral gaps for π-irreducible, positive recurrent Markov chains with a general state space Ω. We obtain necessary and sufficient conditions for the existence of L 2(π)-spectral gaps in terms of a sequence of isoperimetric constants. For reversible Markov chains, it turns out that the spectral gap can be understood in terms of convergence of an induced probability flow to the uniform flow. These results are used to recover classical results concerning uniform ergodicity and the spectral gap property as well as other new results. As an application of our result, we present a rather short proof for the fact that geometric ergodicity implies the spectral gap property. Moreover, the main result of this paper suggests that sharp upper bounds for the spectral gap should be expected when evaluating the isoperimetric flow for certain sets. We provide several examples where the obtained upper bounds are exact.

Establishing an Adaptive Production-Procurement System with Markov Chain Approach Associated with 3C Theory

Problem statement: In this study, we classify the imperfect situations that the unreliability of production and the defective products might be incurred from production process into eight independently recurrent categories. Approach: With the property of recurrence of Markov chain, the limiting probability of each imperfect production category can be calculated through the transition probabilities matrix generated from these imperfect production categories. Results: The 3C Theory has been applied based on a certain group of products to integrate the relationship between order model and resource requirement plans. Therefore, an adaptive (s, Q) production system with Markov Chain Approach associated with (R, s, S) procurement system based on 3C Theory has been proposed to smooth supply disruptions incurred from imperfect production. Conclusion/Recommendations: We have demonstrated that the candidate solutions identified by the proposed method are not only superior to traditional EPQ model solutions, but also result in significantly smaller cost flow variability.

Semiparametric Regression Estimation in Null Recurrent Nonlinear Time Series

Estimation theory in a nonstationary environment has been very popular in recent years. Existing studies focus on nonstationarity in parametric linear, parametric nonlinear and nonparametric nonlinear models. In this paper, we consider a partially linear model of nonstationary time series, in which both parametric and nonparametric regressors are beta-null recurrent Markov chains. We propose to estimate both the unknown parameters and functions semiparametrically. We then show that the proposed estimator of the parameter vector is still asymptotically normal with the same rate as for the case of stationary time series. We also establish the asymptotic normality for the nonparametric estimator of the function and the uniform consistency of the nonparametric estimator. The simulated example is given to show that our theory and method work well in practice.

On mean recurrence times of Markov chains and spanning tree invariants

We show that the mean recurrence times of (countable state) irreducible and positively recurrent Markov chains are the spanning tree invariants of the first return loop systems. Then, by the Perron–Frobenius theorem, the spanning tree invariants of the first return loop systems of a finite state Markov chain are all equal if and only if the process is doubly stochastic, settling a conjecture on a question in [1] where it was verified for matrices of size at most three.

Nonparametric regression estimation in a null recurrent time series

We derive an asymptotic theory of nonparametric estimation for a time series regression model Z t = f( X t )+ W t , where { X t } and { Z t } are observed nonstationary processes, and { W t } is an unobserved stationary process. The class of nonstationary processes allowed for { X t } is a subclass of the class of null recurrent Markov chains. This subclass contains the random walk, unit root processes and nonlinear processes. The process { W t } is assumed to be linear and stationary.

A limit theorem for random sums modulo 1

Residues of partial sums in a class of dependent random variables, including functionals of uniformly recurrent Markov chains, are in the domain of attraction of the uniform distribution. These types of limit theorems arise for example in the multiplication of floating-point numbers.

Sharp Bounds for the Tails of Functionals of Markov Chains

This paper is devoted to establishing sharp bounds for deviation probabilities of partial sums $\sum_{i=1}^{n}f(X_{i})$, where $X=(X_{n})_{n\in{\bf N}}$ is a positive recurrent Markov chain and f is a real valued function defined on its state space. Combining the regenerative method to the Esscher transformation, these estimates are shown in particular to generalize probability inequalities proved in the independent i.i.d. case to the Markovian setting for (not necessarily uniformly) geometrically ergodic chains.

Estimation in Threshold Autoregressive Models with a Stationary and a Unit Root Regime

This paper treats estimation in a class of new nonlinear threshold autoregressive models with both a stationary and a unit root regime. Existing literature on nonstationary threshold models have basically focused on models where the nonstationarity can be removed by differencing and/or where the threshold variable is stationary. This is not the case for the process we consider, and nonstandard estimation problems are the result.This paper proposes a parameter estimation method for such nonlinear threshold autoregressive models using the theory of null recurrent Markov chains. Under certain assumptions, we show that the ordinary least squares (OLS) estimators of the parameters involved are asymptotically consistent. Furthermore, it can be shown that the OLS estimator of the coefficient parameter involved in the stationary regime can still be asymptotically normal while the OLS estimator of the coefficient parameter involved in the nonstationary regime has a nonstandard asymptotic distribution. In the limit, the rate of convergence in the stationary regime is much slower than that in the nonstationary regime. The proposed theory and estimation method are illustrated by both simulated and real data examples.

Random recurrence equations and ruin in a Markov-dependent stochastic economic environment

We develop sharp large deviation asymptotics for the probability of ruin in a Markov-dependent stochastic economic environment and study the extremes for some related Markovian processes which arise in financial and insurance mathematics, related to perpetuities and the ARCH(1) and GARCH(1, 1) time series models. Our results build upon work of Goldie [Ann. Appl. Probab. 1 (1991) 126–166], who has developed tail asymptotics applicable for independent sequences of random variables subject to a random recurrence equation. In contrast, we adopt a general approach based on the theory of Harris recurrent Markov chains and the associated theory of nonnegative operators, and meanwhile develop certain recurrence properties for these operators under a nonstandard “Gärtner–Ellis” assumption on the driving process.

Nearest neighbor conditional estimation for Harris recurrent Markov chains

This paper is concerned with consistent nearest neighbor time series estimation for data generated by a Harris recurrent Markov chain on a general state space. It is shown that nearest neighbor estimation is consistent in this general time series context, using simple and weak conditions. The results proved here, establish consistency, in a unified manner, for a large variety of problems, e.g. autoregression function estimation, and, more generally, extremum estimators as well as sequential forecasting. Finally, under additional conditions, it is also shown that the estimators are asymptotically normal.

Variance estimation in the central limit theorem for Markov chains

This article concerns the variance estimation in the central limit theorem for finite recurrent Markov chains. The associated variance is calculated in terms of the transition matrix of the Markov chain. We prove the equivalence of different matrix forms representing this variance. The maximum likelihood estimator for this variance is constructed and it is proved that it is strongly consistent and asymptotically normal. The main part of our analysis consists in presenting closed matrix forms for this new variance. Additionally, we prove the asymptotic equivalence between the empirical and the maximum likelihood estimation (MLE) for the stationary distribution.

Establishing an adaptive production system for smoothing disruptions in supply networks

For as long as there have been supply networks, there have been disruptions, and no supply chain, logistics system, or infrastructure network is immune to them. Recent events have highlighted the need for planners to consider the risk of disruptions when establishing supply networks. Supply networks disruptions have a number of causes and may take a number of forms, such as quality variance, equipment and machine unreliability, and defects and shortage incurred from imperfect production planning, implementation, and processing. In this paper, we classify the imperfect situations that the unreliability of production processes may create into eight independently recurrent categories. With the property of recurrence of Markov chain, the limiting probability of each imperfect production category can be calculated through the transition probabilities matrix generated from these imperfect production categories. Therefore, an adaptive (s, Q) production system with finite states forms a Markov chain to determine the reproduction point and the quantity of production can be established to smooth supply disruptions incurred from imperfect production.

Perfect simulation for a class of positive recurrent Markov chains

In [1] we introduced a class of positive recurrent Markov chains, named tame chains. A perfect simulation algorithm, based on the method of dominated CFTP, was then shown to exist in principle for such chains. The construction of a suitable dominating process was flawed, in that it relied on an incorrectly stated lemma ([1], Lemma 6). This claimed that a geometrically ergodic chain, subsampled at a stopping time σ , satisfies a geometric Foster–Lyapunov drift condition with coefficients not depending on σ. This is true if σ is a stopping time independent of the chain, but not if this independence does not hold. Reference [1], Lemma 6 is therefore false as stated.