Discrete Parameter Markov Processes Research Articles

$I$-Projection and Conditional Limit Theorems for Discrete Parameter Markov Processes

Let $(X, \mathscr{B})$ be a compact metric space with $\mathscr{B}$ the $\sigma$-field of Borel sets. Suppose this is the state space of a discrete parameter Markov process. Let $C$ be a closed convex set of probability measures on $X$. Known results on the asymptotic behavior of the probability that the empirical distributions $\hat{P}_n$ belong to $C$ and new results on the Markov process distribution of $\omega_0, \ldots, \omega_{n - 1}$ under the condition $\hat{P}_n \in C$ are obtained simultaneously through a large deviations estimate. In particular, the Markov process distribution under the condition $\hat{P}_n \in C$ is shown to have an asymptotic quasi-Markov property, generalizing a concept of Csiszar.

Ergodicity and central limit theorems for a class of Markov processes

We consider a class of discrete parameter Markov processes on a complete separable metric space S arising from successive compositions of i.i.d. random maps on S into itself, the compositions becoming contractions eventually. A sufficient condition for ergodicity is found, extending a result of Dubins and Freedman [8] for compact S. By identifying a broad subset of the range of the generator, a functional central limit theorem is proved for arbitrary Lipschitzian functions on S, without requiring any mixing type condition or irreducibility.

Short-run characteristics of samples drawn by random walks

An alternative to using acceptance-rejection methods to generate a sample of points distributed uniformly over a region is to employ a random walk over that region. The sequence of points generated by a random walk has been shown, under certain easily satisfied conditions, to be a realization of a vector-valued discrete parameter Markov process, and to have the uniform distribution as its limiting distribution. The purpose of this paper is to point out that even if the marginal distribution of each point is actually uniform, rather than merely being asymptotically uniform, small samples may exhibit nonuniform characteristics due to serial autocorrelation within the sample.

Asymptotic independence and individual ratio limit theorems

Let X n be a discrete parameter Markov process on general state space with transition functions P n (x, A). Individual ratio limit theorems are results about convergence as n→∞ of the ratios $$\frac{{P^{n + k} (x,A)}}{{P^n (y,B)}}$$ (1) where k is a fixed integer, A and B are subsets of state space, and x and y are starting points. This paper investigates the relationship of individual ratio limit theorems to various forms of asymptotic independence. When the state space is denumerable it is shown that, subject to standard auxiliary conditions, the much studied Strong Ratio Limit Property is equivalent to a form of asymptotic independence related to mixing conditions of Rosenblatt and Blum, Hanson, and Koopmans.

Recurrence of a Modified Random Walk and Its Application to an Economic Model

A modification of Chung and Fuchs’ (Mem. Amer. Math. Soc., 6 (1951), pp. 1–12) recurrence theorem for random walks leads to an analogous result for a different discrete parameter Markov process. This latter process is applicable to an analysis of price stabilization programs involving purchases and sales from a buffer stock.

On the Weak Convergence of Interpolated Markov Chains to a Diffusion

Let $\{\xi_k^n, k = 0, 1, \cdots\}$ denote a $R^r$ valued discrete parameter Markov process for each $n$. For each real $T > 0$, it is shown that suitable piecewise interpolations in $D^r\lbrack 0, T \rbrack$ converge weakly as $n \rightarrow \infty$, to the diffusion given by \begin{equation*}\tag{*} x(t) = x + \int^t_0 f(x(s), s) ds + \int^t_0 \sigma(x(s), s) dw(s),\end{equation*} under essentially the condition that the solution to $(^\ast)$ is unique in the sense of multivariate distributions, $f(\bullet, \bullet), \sigma(\bullet, \bullet)$ are bounded and continuous, and the scaled "infinitesimal" coefficients of the $\{\xi_k^n\}$ are close to $f(\bullet, \bullet)$ and $\sigma(\bullet, \bullet)$. It is not required that $f(\bullet, \bullet)$ or $\sigma(\bullet, \bullet)$ satisfy a uniform Lipschitz condition, nor that $\sigma(\bullet, \bullet)\sigma'(\bullet, \bullet)$ be positive definite. The result extends the result of Gikhman and Skorokhod (1969). Two examples arising in population genetics are given, where $\sigma(\bullet, \bullet)$ is not uniformly Lipschitz.

Asymptotically Optimal Tests in Markov Processes

Let $(\mathscr{X}, \mathscr{A})$ be a measurable space and let $\Theta$ be an open subset of the $k$-dimensional Euclidean space $\mathscr{E}_k$. For each $\theta\in\Theta$, let $P_\theta$ be a probability measure on $\mathscr{A}$. Let $\{ X_n, n \geqq 0\}$ be a discrete parameter Markov process defined on $(\mathscr{X}, \mathscr{A}, P_\theta), X_n$ taking values in the Borel real line $(R, \mathscr{B})$. Finally, let $\mathscr{A}_n$ be the $\sigma$-field induced by the first $n + 1$ random variables $X_0, X_1,\cdots, X_n$ from the process and let $P_{n,\theta}$ be the restriction of $P_\theta$ to the $\sigma$-field $\mathscr{A}_n$. Under suitable conditions on the process, the following results are derived. Let $\theta_0$ be an arbitrary but fixed point in $\Theta$ and let $\Delta_n(\theta_0)$ be a $k$-dimensional vector defined in terms of the random variables $X_0, X_1,\cdots, X_n; \Delta_n^\ast(\theta_0)$ stands for a certain truncated version of $\Delta_n(\theta_0)$. By means of $\Delta_n^\ast(\theta_0)$ and $h\in\mathscr{E}_k$, one defines a probability measure $R_{n,h}, n \geqq 0$. The first main result is that the sequences $\{P_{n,\theta}\}$ and $\{R_{n,h}\}$ of probability measures with $h = n^{\frac{1}{2}}(\theta - \theta_0), \theta\in\Theta$, are differentially equivalent at the point $\theta_0$. (See Definition 5.1.) This is shown in Corollary 5.1. It is also proved in Corollary 5.2 that the sequence $\{\Delta_n^\ast(\theta_0)\}$ is differentially sufficient at $\theta_0$ (see Definition 5.2) for the family $\{P_{n,\theta}; \theta\in\Theta\}$ of probability measures. Next, let $\{h_n\}$ be a bounded sequence of $h$'s in $\mathscr{E}_k$ and set $\theta_n = \theta_0 + h_nn^{-\frac{1}{2}}$. Then for hypotheses testing problems, Theorem 6.1 allows one to restrict oneself to the class of tests depending on $\Delta_n(\theta_0)$ alone, at least as far as the asymptotic power of the test under alternatives of the form $P_{n,\theta_n}$ is concerned. In Section 7, these results are applied to the case of testing hypotheses about a real-valued parameter. More specifically, asymptotically most powerful tests for testing the hypothesis $\theta = \theta_0$ against one-sided alternatives are constructed. This is covered in Theorem 7.1.1. Also an asymptotically most powerful unbiased test for testing the same hypothesis as above against two-sided alternatives is constructed in Theorem 7.1.2. The first of these problems was also dealt with in Johnson and Roussas [2] but the approach is different here. The second problem is solved in Wald [8] for the independent identically distributed case. However, both the assumptions and approach are different here in addition to the Markovian character of the random variables involved. Section 6 treats the general situation where $\Theta$ is an open subset of $\mathscr{E}_k$. Theorem 6.1 together with Theorem 6.3 provide a way for studying the corresponding hypothesis testing problem in the $k$-dimensional parameter case. Finally, at the end of the last section, an outline is presented of forthcoming results for that case. These results extend, under substantially weaker conditions, the work of Wald [8], [9] to Markov processes. The method of proof relies heavily on the development in LeCam [3]. Unless otherwise stated, limits will be taken as the sequence $\{ n\}$, or subsequences thereof, tends to $\infty$. Integrals without limits will extend over the entire appropriate space. For $h\in\mathscr{E}_k, h'$ stands for its transpose. All bounding constants will be finite numbers.

Strong ratio limits, R-recurrence and mixing properties of discrete parameter Markov processes

Strong ratio limits, R-recurrence and mixing properties of discrete parameter Markov processes

A Bound for the Law of Large Numbers for Discrete Markov Processes

An exponential bound is obtained for the law of large numbers for $S_n = \sum^n_{k=1} f(X_k)$ where $\{X_k: k = 1, 2, \cdots \}$ is a discrete parameter Markov process satisfying Doeblin's condition and $f$ is a bounded, real-valued, measurable function.

The Limiting Distribution of Brownian Motion in a Bounded Region with Instantaneous Return

A point executes Brownian motion in a bounded, connected, and open three dimensional region $D$. When it reaches the boundary $\Gamma$, at point $\alpha$, it is instantaneously returned to $D$ according to probability measure $\mu(\alpha)$ (we write $\mu(\alpha, A)$ for the measure of set $A$), and the Brownian motion is resumed. This is a Markov process and, subject to certain regularity conditions on $\Gamma$ and $\mu(\alpha)$, we derive the limiting distribution of the process. Processes of this sort have been considered by Feller [1]; he has obtained the transition probabilities of such processes. He is concerned more generally with Markov processes with continuous sample functions on a linear interval; the return may be instantaneous or after a random period of time. Let $p^0(t, \xi, A)$ be the probability that the point is in set $A$ of $D$ at time $t$ when it is initially at point $\xi$ of $D$, with the additional restriction that no boundary contacts have been made. It is known that \begin{equation*}\tag{1}p^0(t, \xi, A) = \int_A u(t, \xi, x)dx,\end{equation*} where $dx$ is the volume element about $x$ and $u$ is the solution of the equation $$\frac{1}{2}\Delta u = u_t,$$ subject to the conditions $$u(t, \xi, \alpha) = 0,\quad\alpha\varepsilon\Gamma,\quad\lim_{t \rightarrow 0} \int_C u(t, \xi, x)dx = 1,$$ where $C$ is any sphere of non-zero radius with center $\xi$ which is entirely within $D$. We may write explicitly $$u(t,\xi,x) = \sum^\infty_{k = 1} v_k(\xi)v_k(x)e^{-\lambda_kt},$$ where $\lambda_k$ is the $k$th eigenvalue and $v_k(x)$ the corresponding eigenfunction of the equation $\Delta u + 2\lambda u = 0$ subject to the boundary condition $u = 0$ on $\Gamma$. If $K(\xi, x)$ is the Green's function of $\Delta u = 0$ in $D$, then ([2], and [3], page 273) \begin{equation*}\tag{2}K(\xi, x) = \frac{1}{2} \int^\infty_0 u(t, \xi, x) dt.\end{equation*} Let $\phi(t, \xi, \alpha)dt d\alpha$ be the probability the point is absorbed at surface element $d\alpha$ of $\Gamma$ between $t$ and $t + dt$ when it is initially at point $\xi$ of $D$. Then $\phi$ is half the interior normal derivative of $u$ at point $\alpha$ of $\Gamma$ ([3], page 273). When the point is initially at $\xi$ the probability of ultimate absorption in set $S$ of $\Gamma$ is given by \begin{equation*}\tag{3}\pi(\xi,S) = \int_S \int^\infty_0 \phi(t, \xi, \alpha) dt d\alpha.\end{equation*} We may define a discrete parameter Markov process with $\Gamma$ as state space by taking as transition probability \begin{equation*}\tag{4}\pi(\alpha, S) = \int_D \pi(\xi, S)\mu(\alpha, d\xi)\end{equation*} This Markov process has a limiting distribution $\pi$ which satisfies the equation \begin{equation*}\tag{5}\pi(S) = \int_\Gamma \pi(\alpha, S)\pi(d\alpha).\end{equation*} We define a measure of sets of $D$ by $$\lambda(A) = \int_\Gamma \mu(\alpha, A)\pi(d\alpha).$$ We may now write the density function for the limiting distribution. If $M(\xi)$ is the mean time of reaching the boundary when the point is initially at $\xi$, $$M(\xi) = \int_\Gamma \int^\infty_0 t_\phi(t, \xi, \alpha) dt d\alpha.$$ then the density function of the limiting distribution is \begin{equation*}\tag{6}\frac{2 \int_D K(\xi, x)\lambda(d\xi)}{\int_D M(\xi)\lambda(d\xi)}.\end{equation*} If we are given a probability measure $\lambda$ in $D$ and the return is always according to $\lambda$, then it is clear that the limiting density of this process is also given by (6). If $\lambda$ concentrates at a single point $\xi$ we may drop the integrals in (6), and in particular we get $$M(\xi) = 2 \int_D K(\xi, x)dx.$$ We note that (6) is essentially the steady distribution of temperature in the following problem: $D$ is a homogeneous heat conducting body whose boundary is kept at temperature 0 and in which there is a constant source of heat distributed according to $\lambda$. Regarding the regularity conditions, we shall assume that $\Gamma$ is made up of finitely many surfaces, each with a continuously turning tangent plane and that $D$ has a Green's function ([4], page 262). We will assume there is a closed set $B$ in $D$ such that $$\inf_{\alpha \varepsilon \Gamma} \mu(\alpha, B) = \gamma > 0.$$