Transition Probabilities Of Markov Chain Research Articles

Countable non-homogeneous Markov chains: asymptotic behaviour

The paper deals with asymptotic properties of the transition probabilities of a countable non-homogeneous Markov chain, the main concept used in the proofs being that of the tail σ-field of the chain. A state classification similar to that existing in the homogeneous case is given and a strong ratio limit property is shown to parallel the basic limit theorem for positive homogeneous chains. Some global asymptotic properties for null chains are also derived.

Countable non-homogeneous Markov chains: asymptotic behaviour

The paper deals with asymptotic properties of the transition probabilities of a countable non-homogeneous Markov chain, the main concept used in the proofs being that of the tail σ-field of the chain. A state classification similar to that existing in the homogeneous case is given and a strong ratio limit property is shown to parallel the basic limit theorem for positive homogeneous chains. Some global asymptotic properties for null chains are also derived.

Modes of convergence of Markov chain transition probabilities

Suppose { P n ( x, A)} denotes the transition law of a general state space Markov chain { X n }. We find conditions under which weak convergence of { X n } to a random variable X with law L (essentially defined by ∝ P n ( x, dy) g( y) → ∝ L( dy) g( y) for bounded continuous g) implies that { X n } tends to X in total variation (in the sense that ∥ P n(x, .) − L ∥ → 0) , which then shows that L is an invariant measure for { X n }. The conditions we find involve some irreducibility assumptions on { X n } and some continuity conditions on the one-step transition law { P( x, A)}.

A Uniform Theory for Sums of Markov Chain Transition Probabilities

Necessary and sufficient conditions are given for boundedness of $\sup_n \|\sum^n_{k=1} (P^k(x, \bullet) - P^k(y, \bullet))\|$ and $\sup_n \|\sum^n_{k=1} (P^k(x, \bullet) - \pi\|$, where the norm is total variation and where $\pi$ is an invariant probability measure. Also conditions for convergence of $\sum^\infty_{k=1} (P^k(x, \bullet) - \pi)$ in norm are given. These results require the study of certain "small sets." Two new types are introduced: uniform sets and strongly uniform sets, and these are related to the sets introduced by Harris in his study of the existence of $\sigma$-finite invariant measure.

A Markov Model of Turnover in Aid to Families with Dependent Children

In order to gain some insight into turnover in the welfare population, a two-state (on and off welfare) Markov chain transition probability matrix is estimated with longitudinal data on welfare recipients coming on welfare in 1965. Results indicate that an enormous amount of turnover occurs in the welfare population and that the average duration of time on welfare per time on welfare is relatively modest for this group. Persons facing a wage below the minimum wage and/or high unemployment are less likely to leave welfare and more likely to return, likely to stay off for shorter periods and on for longer periods, and more likely to be on welfare than are those with low expected unemployment or wages above the minimum.

Some limit theorems for a class of network problems as related to finite Markov chains

This paper is concerned with a class of dynamic network flow problems in which the amount of flow leaving node i in one time period for node j is the fraction pij of the total amount of flow which arrived at node i during the previous time period. The fraction pij whose sum over j equals unity may be interpreted as the transition probability of a finite Markov chain in that the unit flow in state i will move to state j with probability pij during the next period of time. The conservation equations for this class of flows are derived, and the limiting behavior of the flows in the network as related to the properties of the fractions Pij are discussed.

Some limit theorems for a class of network problems as related to finite Markov chains

This paper is concerned with a class of dynamic network flow problems in which the amount of flow leaving node i in one time period for node j is the fraction pij of the total amount of flow which arrived at node i during the previous time period. The fraction pij whose sum over j equals unity may be interpreted as the transition probability of a finite Markov chain in that the unit flow in state i will move to state j with probability pij during the next period of time. The conservation equations for this class of flows are derived, and the limiting behavior of the flows in the network as related to the properties of the fractions Pij are discussed.

Uniform rates of convergence for Markov chain transition probabilities

Uniform rates of convergence for Markov chain transition probabilities

Choice of Initial States in Estimating Transition Probabilities of a Finite Ergodic Markov Chain

Choice of Initial States in Estimating Transition Probabilities of a Finite Ergodic Markov Chain

Choice of initial states in estimating transition probabilities of a finite ergodic Markov chain

The properties of the maximum likelihood estimators for the transition probabilities in a two-state ergodic Markov chain depend upon the choice of the initial states. For example, the initial states may be suitably fixed in advance or may be observed from the equilibrium distribution of the chain. A large number of relatively short realizations are assumed available and the asymptotic variances are examined in some detail. The relevant asymptotic covariance matrices are derived also for the s-state chain.

Approximate Stationary Probability Vectors of a Finite Markov Chain

The set of stationary probability vectors arising when the transition probabilities of an n-state Markov chain are perturbed, is defined. They are shown to lie in a convex cone bounded by at most $3n$ hyperplanes. There is an infinite number of transition matrices which will yield a given stationary probability vector. The effect of the same perturbation of this set of matrices is examined and several relations are established. It is shown that the largest perturbations in the stationary probability vector tend to occur for diagonally dominant matrices and some particular results are given for this case.

HF Markov Chain Models and Measured Error Averages

A set of calculated channel parameters are postulated for a 2800-km HF ionospheric F2 -mode propagation path to organize error sequences into stationary samples. Measurements of the single sideband-frequency-shift keying (SSB-FSK) binary error-free run distributions are used with the parameters to obtain stationary Markov chain transition probabilities. A relation between the transition probabilities and the average error probability is verified for the HF channel model. A statistical analysis of error probability and error-free run distribution variation with calculated parameter values suggests that the Markov chain transition probabilities are a function of these parameters.

State spaces for Markov chains

If p ( t , i , j ) p(t,i,j) is the transition probability (from i to j in time t) of a continuous parameter Markov chain, with p ( 0 + , i , i ) = 1 p(0 + ,i,i) = 1 , entrance and exit spaces for p are defined. If L [ L ∗ ] L[{L^ \ast }] is an entrance [exit] space, the function p ( ⋅ , ⋅ , j ) [ p ( ⋅ , i , ⋅ ) / h ( ⋅ ) ] p( \cdot , \cdot ,j)[p( \cdot ,i, \cdot )/h( \cdot )] has a continuous extension to ( 0 , ∞ ) × L [ ( 0 , ∞ ) × L ∗ (0,\infty ) \times L[(0,\infty ) \times {L^ \ast } , for a certain norming function h on L ∗ {L^ \ast } ]. It is shown that there is always a space which is both an entrance and exit space. On this space one can define right continuous strong Markov processes, for the parameter interval [0, b], with the given transition function as conditioned by specification of the sample function limits at 0 and b.

Integral representations of transition probabilities and serial covariances of certain markov chains

This note is concerned with integral representations of some transition probabilities of countable state space Markov chains embedded in birth and death processes, of the serial covariances of functions defined on such chains when stationary, and finally the properties of the spectral measure of stationary processes with monotonically decreasing or completely monotonic serial covariances.

Integral representations of transition probabilities and serial covariances of certain markov chains

This note is concerned with integral representations of some transition probabilities of countable state space Markov chains embedded in birth and death processes, of the serial covariances of functions defined on such chains when stationary, and finally the properties of the spectral measure of stationary processes with monotonically decreasing or completely monotonic serial covariances.

240. Note: On Estimating the Equilibrium and Transition Probabilities of a Finite-State Markov Chain from the Same Data

240. Note: On Estimating the Equilibrium and Transition Probabilities of a Finite-State Markov Chain from the Same Data

General transition probabilities for finite Markov chains

We consider a stationary discrete-time Markov chain with a finite number m of possible states which we designate by 1,…,m. We assume that at time t = 0 the process is in an initial state i with probability (i = 1,…, m) and such that and .

Integral representations for transition probabilities of Markov chains with a general state space

Integral representations for transition probabilities of Markov chains with a general state space

XXIV.—Conditions for the Ergodicity of Non-homogeneous Finite Markov Chains

SynopsisIn this note we study the asymptotic behaviour of a product of matrices where Pj is a matrix of transition probabilities in a non-homogeneous finite Markov chain. We give conditions that (i) the rows of P(n) tend to identity and that (ii) P(n) tends to a limit matrix with identical rows.