Skip-free Processes Research Articles

Kemeny's constant for countable Markov chains

In this paper, we aim to investigate Kemeny's constant for discrete-time or continuous-time, irreducible and positive recurrent Markov chains on a countable state space. We show that Kemeny's constant is infinite for a discrete-time Markov chain on an infinitely countable state space, which confirms a conjecture in page 1031 of [3]. Moreover, it is shown that the study on Kemeny's constant for a continuous-time Markov chain with a bounded generator can be converted to that for a discrete-time Markov chain through the uniformization technique. It is different and challenging to analyze Kemeny's constant for a chain with unbounded generator. Finally, a upward skip-free process is considered and some discussions for Kemeny's constant are given.

A new solution algorithm for skip-free processes to the left

This paper proposes a new solution algorithm for steady state models describing skip-free processes to the left where each level has one phase. The computational complexity of the algorithm is independent of the number of levels of the system. If the skip parameter of the skip-free process is significantly smaller than the number of levels of the system, our algorithm numerically outperforms existing algorithms for skip-free processes. The proposed algorithm is based on a novel method for applying generalized Fibonacci series to the solution of steady state models.

Approximating Ergodic Average Reward Continuous-Time Controlled Markov Chains

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> We study the approximation of an ergodic average reward continuous-time denumerable state Markov decision process (MDP) by means of a sequence of MDPs. Our results include the convergence of the corresponding optimal policies and the optimal gains. For a controlled upwardly skip-free process, we show some computational results to illustrate the convergence theorems. </para>

Average optimality for continuous-time Markov decision processes in Polish spaces

This paper is devoted to studying the average optimality in continuous-time Markov decision processes with fairly general state and action spaces. The criterion to be maximized is expected average rewards. The transition rates of underlying continuous-time jump Markov processes are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. We first provide two optimality inequalities with opposed directions, and also give suitable conditions under which the existence of solutions to the two optimality inequalities is ensured. Then, from the two optimality inequalities we prove the existence of optimal (deterministic) stationary policies by using the Dynkin formula. Moreover, we present a ``semimartingale characterization'' of an optimal stationary policy. Finally, we use a generalized Potlach process with control to illustrate the difference between our conditions and those in the previous literature, and then further apply our results to average optimal control problems of generalized birth--death systems, upwardly skip-free processes and two queueing systems. The approach developed in this paper is slightly different from the ``optimality inequality approach'' widely used in the previous literature.

Uniqueness criteria for continuous-time Markov chains with general transition structures

We derive necessary and sufficient conditions for the existence of bounded or summable solutions to systems of linear equations associated with Markov chains. This substantially extends a famous result of G. E. H. Reuter, which provides a convenient means of checking various uniqueness criteria for birth-death processes. Our result allows chains with much more general transition structures to be accommodated. One application is to give a new proof of an important result of M. F. Chen concerning upwardly skip-free processes. We then use our generalization of Reuter's lemma to prove new results for downwardly skip-free chains, such as the Markov branching process and several of its many generalizations. This permits us to establish uniqueness criteria for several models, including the general birth, death, and catastrophe process, extended branching processes, and asymptotic birth-death processes, the latter being neither upwardly skip-free nor downwardly skip-free.

Uniqueness criteria for continuous-time Markov chains with general transition structures

We derive necessary and sufficient conditions for the existence of bounded or summable solutions to systems of linear equations associated with Markov chains. This substantially extends a famous result of G. E. H. Reuter, which provides a convenient means of checking various uniqueness criteria for birth-death processes. Our result allows chains with much more general transition structures to be accommodated. One application is to give a new proof of an important result of M. F. Chen concerning upwardly skip-free processes. We then use our generalization of Reuter's lemma to prove new results for downwardly skip-free chains, such as the Markov branching process and several of its many generalizations. This permits us to establish uniqueness criteria for several models, including the general birth, death, and catastrophe process, extended branching processes, and asymptotic birth-death processes, the latter being neither upwardly skip-free nor downwardly skip-free.

Kendall's identity for the first crossing time revisited

We give a new relatively compact proof of the famous identity for the distribution of the first hitting time of a linear boundary by a skip-free process with stationary independent increments. The proof uses martingale identities and change of measure.

Large deviations results for subexponential tails, with applications to insurance risk

Consider a random walk or Lévy process { S t } and let τ( u) = inf { t⩾0 : S t > u}, P ( u) (·) = P(· | τ( u) < ∞). Assuming that the upwards jumps are heavy-tailed, say subexponential (e.g. Pareto, Weibull or lognormal), the asymptotic form of the P ( u) -distribution of the process { S t } up to time τ( u) is described as u → ∞. Essentially, the results confirm the folklore that level crossing occurs as result of one big jump. Particular sharp conclusions are obtained for downwards skip-free processes like the classical compound Poisson insurance risk process where the formulation is in terms of total variation convergence. The ideas of the proof involve excursions and path decompositions for Markov processes. As a corollary, it follows that for some deterministic function a( u), the limiting P ( u) -distribution of τ(u) a(u) is either Pareto or exponential, and corresponding approximations for the finite time ruin probabilities are given.

Comparing first-passage times for semi-Markov skip-free processes

For two semi-Markov positive skip-free processes on N conditions are given such that the first-passage times from the reflecting state 0 to a preassigned threshold are ordered according to Laplace and to increasing linear orderings. A comparison of first-passage times is performed via the new notion of ‘dual weak likelihood ratio ordering’. An application to life times of devices subject to shocks and repairs is finally provided.

Relationships in stationary jump processes with countable state space and their applications to queues

We consider a stationary continuous time process with a finite or countable state space, which is usually called a stationary pure jump process because it changes its state by jumps. Relationships between the stationary distributions at an arbitrary time, just before and after jump instants are obtained by using conditional sojourn times in states and in sets of states. For a skip free process, these distributions have product form expressions in terms of the conditional mean sojourn times. The results are applied to queueing models. We extend some known relationships between queueing characteristics to batch arrival and batch serve queues with stationary inputs. They also give a unified approach for truncation expressions for finite queues.

Some remarks on a One-dimensional skip-free process with repulsion

AbstractWe extend the results obtained by Hines and Thompson for a Markov chain which has a single reflecting barrier at the origin, nearest neighbour transitions and which moves from {j} to {j + l} with probability j/(j + 1). Martingale limit theorems are used to work out an asymptotic theory for a general class of such chains for which the probability above has the form l – λ(j) = O&gt;λ(j)&gt;1 (j ∈N),λ(j)→ O (j →∞)and Σλ(j)=∞ We discuss the case where the last sum is finite and some alternative versions of the general case.

First-passage times in skip-free processes

We derive simple criteria to ensure the finiteness of the mean first-passage times into semi-infinite sets for real-valued skip-free processes, which move by random jumps in one direction and deterministically, for a random time, in the other. These criteria are applied to state-dependent queueing systems, to give finiteness of the mean of the busy period, and to a self-correcting birth process.

A generalised ratio identity

An identity is derived relating the distribution of a continuous time, skip-free process with two impenetrable barriers to the distribution of a process with one barrier. It is indicated how the methods used here can also be used to give results for related processes with different types of barriers. The identity is then used to derive explicit results giving the distribution of the process with two impenetrable barriers.

On exponential ergodicity and spectral structure for birth-death processes, II

In Part I, Feller's boundary theory was described with simple conditions for process classification. The implications of this boundary classification scheme for spectral structure and exponential ergodicity are examined in Part II. Conditions under which the spectral span is finite or infinite are established. A time-dependent norm is exhibited describing the exponentiality of the convergence and its uniformity. Specific systems are discussed in detail: Contents: 1. 7. Spectral structure for the M/M/I process 2. 8. Exponential ergodicity for processes with entrance, exit, and regular boundaries 3. 9. Exponential ergodicity for processes with natural boundaries 4. 10. Uniformity of exponential convergence 5. 11. Finite and Infinite spectral span 6. 12. Skip-free processes on the full lattice