Perturbed Markov Processes Research Articles

Cauchy noise and affiliated stochastic processes

By departing from the previous attempt [Phys. Rev. E 51, 4114 (1995)] we give a detailed construction of conditional and perturbed Markov processes, under the assumption that the Cauchy law of probability replaces the Gaussian law (appropriate for the Wiener process) as the model of primordial noise. All considered processes are regarded as probabilistic solutions of the so-called Schrödinger interpolation problem, whose validity is thus extended to the jump-type processes and their step process approximants.

Ergodic Control of a Singularly Perturbed Markov Process in Discrete Time with General State and Compact Action Spaces

Ergodic control of singularly perturbed Markov chains with general state and compact action spaces is considered. A new method is given for characterization of the limit of invariant measures, for perturbed chains, when the perturbation parameter goes to zero. It is also demonstrated that the limit control principle is satisfied under natural ergodicity assumptions about controlled Markov chains. These assumptions allow for the presence of transient states, a situation that has not been considered in the literature before in the context of control of singularly perturbed Markov processes with long-run-average cost functionals.

Asymptotic Series for Singularly Perturbed Kolmogorov–Fokker–Planck Equations

We derive limit theorems for the transition densities of diffusion processes and develop asymptotic expansions for solutions of a class of singularly perturbed Kolmogorov–Fokker–Planck equations. The model under consideration can be viewed as a Markov process having two time scales. One of them is a rapidly changing scale, and the other is a slowly varying one. The study is motivated by a wide range of applications involving singularly perturbed Markov processes in manufacturing systems, reliability analysis, queueing networks, statistical physics, population biology, financial economics, and many other related fields. In this work, the asymptotic expansion is constructed explicitly. It is shown that the initial layer terms in the expansion decay at an exponential rate. Error bounds on the remainder terms also are obtained. The validity of the expansion is rigorously justified.

On Transition Densities of Singularly Perturbed Diffusions with Fast and Slow Components

We derive asymptotic properties of transition densities for singularly perturbed diffusion processes with fast and slow components. Our study focuses on the Kolmogorov–Fokker–Planck equations. The model can be viewed as a diffusion process having two time scales and is motivated by a wide variety of applications involving singularly perturbed Markov processes in manufacturing systems, homogenization, reliability analysis, queueing networks, statistical physics, population biology, financial economics, and many other related fields. By virtue of the methods of matched singular perturbation, asymptotic expansion is constructed for the transition density. The expansion includes both regular part and boundary layer corrections. Detailed justification of the asymptotic expansion is given, and error bounds are also provided.

The Evolution of Conventions

The author shows how a group of individuals can learn to play a coordination game without any common knowledge and with only a small amount of rationality. The game is repeated many times by different players. Each player chooses an optimal reply based on incomplete information about what other players have done in the past. Occasionally they make mistakes. When the likelihood of mistakes is very small, typically one coordination equilibrium will be played almost all of the time over the long run. This stochastically stable equilibrium can be computed analytically using a general theorem the author proves on perturbed Markov processes. Copyright 1993 by The Econometric Society.

On ergodic control problems for singularly perturbed Markov processes

In this paper ergodic control problems (optimal stopping, impulsive control, and stochastic control for singularly perturbed Feller Markov processes) are studied. As the main result the so-called limit control principle is shown to hold in each case. The results obtained depend on the averaging properties of the perturbed system which follow from the fact that the perturbing process does not depend on either the perturbed process or on the control.

On some problems arising in asymptotic analysis of Markov processes with singularly perturbed generators

In, the paper we study singularly perturbed Markov processes. Some new results in convergence of resolvents, weak convergence and convergence of semigroups are obtained. The results are applied to optima1 stopping and impulse control of singular1y perturbed Markov processes. Examples justifying the assumptions made in text are also presented

Equivalent descriptions of perturbed Markov processes

An abstract is not available for this content so a preview has been provided. Please use the Get access link above for information on how to access this content.

Equivalent descriptions of perturbed Markov processes

Perturbation models as described in detail by Syski [9] are investigated from the point of view of transformations of time or of state. We consider whether the perturbed Markov process is equivalent to a state or time transformation of a process similar to the unperturbed one. Ways of ensuring such equivalences are discussed and a brief look at perturbations for continuous parameter, continuous state space processes is also taken.