Homogeneous Markov Process Research Articles

Limit theorem for Markov processes with finite number of states

The asymptotic behavior is found for a solution of a multidimensional regeneration equation composed for a class of homogeneous Markov processes. An application of the result found is given to the study of consolidated processes.

Closed-form solution for the distribution of the total time spent in a subset of states of a homogeneous Markov process during a finite observation period

Markov process are widely used to model computer systems. De Souza e Silva and Gail [3] calculated numerically the distribution of the cumulative operational time of repairable computer systems modelled by Markovian processes, that is, the distribution of the total time during which the system was in operation over a finite observation period. An extension of their approach is presented here. A closed-form solution is obtained for the distribution of the total time spent in a subset of states of a homogeneous Markov process during a finite observation period, which is theoretically and numerically interesting. We also give an application of this result to a fault-tolerant system.

Closed-form solution for the distribution of the total time spent in a subset of states of a homogeneous Markov process during a finite observation period

Markov process are widely used to model computer systems. De Souza e Silva and Gail [3] calculated numerically the distribution of the cumulative operational time of repairable computer systems modelled by Markovian processes, that is, the distribution of the total time during which the system was in operation over a finite observation period. An extension of their approach is presented here. A closed-form solution is obtained for the distribution of the total time spent in a subset of states of a homogeneous Markov process during a finite observation period, which is theoretically and numerically interesting. We also give an application of this result to a fault-tolerant system.

Invariant Markov processes on compact groups and correlation functions

Random processes governing the time evolution of probability distributions of many physical systems can be described by continuous homogeneous Markov processes taking values from compact groups. Assuming the transition probability function of the process to be invariant in the sense that P t ( x, B) = P t ( e, x -1 B) with e being the neutral element of the group, the harmonic analysis (Weyl theory) is applied to study the properties of the Markov semi-group. The infinitesimal operator and generating functional are decomposed using the Levy-Khinchin formula. Under some auxiliary assumptions the components of this decomposition are interpreted as generators of a one parameter subgroup, Brownian motion and a jump process. The formalism is illustrated for several models of processes taking values from compact Lie groups. The properties of the correlation functions of time dependent random variables are investigated.

Transition events fluctuations in jump markovian dynamics. A general approach to Šolc's problem

We introduce a new formalism for computing the moments of transition events for nonhomogeneous Markov jump processes. Our method is applied directly to the master equation and does not involve the use of diffusion approximation. The general theory is applied to produce exact expressions for means and dispersions. For time homogeneous Markov processes with a finite number of connected states we are able to prove that both means and dispersions asymptotically increase linearly in time.

Benchmark results for particle transport in a binary Markov statistical medium

We give numerical benchmark results for particle transport in a randomly mixed binary medium, with the mixing statistics described as a homogeneous Markov process. A Monte Carlo procedure is used to generate a physical realization of the statistics, and a discrete ordinate numerical transport solution is generated for this realization. The ensemble averaged solution, as well as the variance, is obtained by averaging a large number of such calculations. Reflection and transmission results are given for several problems in both rod and planar geometry. In a separate development, two coupled transport equations are derived which formally described transport in a random binary mixture for arbitrary mixing statistics. Closing these equations by approximating their coupling terms in a low order and intuitive way leads to a model for stochastic transport previously obtained via the master equation. The present derivation, based upon approximating exact equations, allows in principle the opportunity to develop more accurate models by making higher order approximations in the coupling terms.

THE COMPUTER SIMULATION OF PARTICLES SEPARATION PROCESS ON SHAKING TABLE

ABSTRACT An attempt has been made to develop a computer model of shaking table to describing particles separation. The model is based on the way that the particle separation process of shaking table can be divided Into two sub-processes, namely, the vertical stratification and the separation of stratified particles, the former can be regarded as a homogeneous Markov process, the latter can, however, be considered as the cross transfer process with three equal-probabilities in the longitudinal direction of the table. The transfer probabilities of Markov process can be determined via the mathematical analysis based on the fluid mechanics and vibration principles. The cross transfer probabilities depend mainly on the wash water and inclination of shaking table. All of the transfer probabilities of the two sub-processes can develop a computer model which can simulate the particles separation process. Once input the feed and operating parameters to the computer model, prediction can be made on the separation r...

Calculating availability and performability measures of repairable computer systems using randomization

Repairable computer systems are considered, the availability behavior of which can be modeled as a homogeneous Markov process. The randomization method is used to calculate various measures over a finite observation period related to availability modeling of these systems. These measures include the distribution of the number of events of a certain type, the distribution of the length of time in a set of states, and the probability of a near-coincident fault. The method is then extended to calculate performability distributions. The method relies on coloring subintervals of the finite observation period based on the particular application, and then calculating the measure of interest using these colored intervals.

Multistate Markov models for systems with dependent units

A system that can be described by a homogeneous continuous-time discrete-state Markov process is treated. The case in which transition rates for each unit depend on the current state of the system is considered. The condition in which the transition-rate matrix of the system has the form of a modified Kronecker sum of transition-rate matrices of its units is investigated. An algorithm based on the Kronecker algebra is introduced for determining the transition-rate matrix of the system. its use is particularly efficient during construction of machines when reliability is evaluated for several systems with the same structure but different transition rates. >

Cussic transport problems in binary homogeneous markov statistical mixtures

Abstract Exact analytic solutions are given for certain classic linear transport problems describing particle flow in a statistical medium. The medium is assumed to consist of two randomly mixed immiscible fluids, with the mixing statistics described as a two state homogeneous Markov process. The problems treated, in both rod and planar geometry, are: (1) the halfspace albedo problem; (2) the halfspace emissivity problem; (3) the infinite medium Green's function problem; and (4) the halfspace Hilne problem.

Statistical analysis of channel current from a membrane patch I. Some stochastic properties of ion channels or molecular systems in equilibrium

The stochastic behavior of single-channel current in a steady-state has been interpreted as the channel's state transitions between several open and shut states, and these transitions have been regarded as a homogeneous Markov process. When a channel is in equilibrium, the principle of detailed balance holds for every step in the state transition scheme. Here we show two stochastic properties of a channel, or any molecule obeying a reversible state transition scheme, under the constraint of detailed balance. First, the distribution functions and the probability density functions of shut or open dwell-time are expressed by the sum of exponential terms with positive coefficients. The same holds for the time-dependent open (or shut) frequency after the shut (or open) transition. Second, the time course of state transition from the state SI to SJ (PI,J(t] is proportional to its reverse transition time course (PJ,I(t], even if SI and SJ are widely separated. The same relation holds also for a transition scheme having transition pathways to the absorbing states. If analysis of a channel current record shows it to be incompatible with either of these two properties, the channel is not in equilibrium but in a steady-state with an energy-consuming cyclic flow. These two properties are also useful for the analysis of any molecular process obeying a homogeneous Markov process or a network of first-order chemical reactions.

A Converse to a Theorem of P. Levy

By a well-known theorem of P. Levy, if $(X_t)$ is a standard Brownian motion on $\mathbb{R}$ with $X_0 = 0$ and if $H_t = \min_{u \leq t}X_u$, then $(Y_t) = (X_t - H_t)$ is Brownian motion with 0 as a reflecting lower boundary. More generally, if $X$ is allowed to have nonzero drift or a reflecting lower boundary at $A < 0$, then the process $Y = X - H$ is still a diffusion process. We prove the converse result: If $X$ is a diffusion on an interval $I \subset \mathbb{R}$ which contains 0 as an interior point, and if $(Y_t) = (X_t - H_t)$ is a time homogeneous strong Markov process (when $X_0 = 0$), then $X$ must be a Brownian motion on $I$ (with drift $\mu$, variance parameter $\sigma^2 > 0$, killing rate $c \geq 0$ and reflection at $\inf I$ in case $\inf I > -\infty$).

On optimal image digitization

Nielsen et al. recently addressed the problem of determining the optimal discretization grid and quantization depth when a given bivariate function f(x, y) has to be described with a predetermined number of bits. This was done under the assumption that the function value range and mean "fluctuation rates" in the x and y direction are given, and that ideal point sampling with zero-order-hold interpolation is used in reconstructing the image. This correspondence outlines an alternative approach, based on the assumption that f (x, y) is the sample function of a 2-D stationary stochastic process with a known covariance function. We use standard integral sampling and obtain closed form solutions under the assumption that f(x, y) is (the sample of) a homogeneous and separable Markov process.

Rapprochement of consolidated stochastic systems and nonhomogeneous semi-Markov systems

The problem of asymptotic consolidation has been considered in a large number of investigations; they can be divided into three directions. In [I-3] one develops an analytic approach, connected with the investigation of the behavior of the solutions of the Markov renewal equations, which operates effectively at the analysis of homogeneous Markov processes (MP) and semi-Markov processes (SMP). In [4, 5], by direct probabilistic methods one has obtained a rapproachement of the characteristics of a consoliated process and of a process of the SMP type for processes with a discrete set of states. In [6-9] one has developed a new approach to the problems of asymptotic consolidation, making use of a special class of processes, viz., switching processes, which is effectively applied in [~0-12] to the analysis of both homogeneous and nonhomogeneous MP and SMP.

Instability of the Harmonic Oscillator with Small Noise

We construct asymptotic expansions for the exponential growth rate (Lyapunov exponent) and rotation number of the random oscillator when the noise is small and is defined by a temporally homogeneous Markov process with a finite number of states. In the case of two states (the telegraph process) we obtain additional terms in the expansions, affording comparison between the exact values and the asymptotic formulas.

One-place unbounded stochastic petri nets: Ergodic criteria and steady-state solutions

This paper presents the analysis of Markov stochastic Petri nets having one unbounded place. These nets represent queues the arrival and departure of customers of which can be grouped and depend on a finite homogeneous Markov process. In the first section we give a formal definition of these nets, called C-Q nets (complex-queue nets), and we state some qualitative properties of such nets (particularly we state that the reachability graph is composed by an infinite sequence of isomorphic subgraphs). In the second section we give a complete classification of ergodic properties of C-Q nets. The Markov process associated with the marking are transient, null recurrent, and positive recurrent, according to the sign of the scalar quantity C 1, N max (where C 1 is the incidence vector of the unbounded place p 1, N max is the maximal expected firing rate of transitions of the net). The third section is devoted to the computation of the steady-state probabilities of C-Q nets. Because the generator of the marking process is a quasi-birth-and-death matrix, we recall the results for such processes (matrix geometric solution). We present the solution based on the generalized eigenvalues. The particular structure of the generator associated to a C-Q net induces a third method to compute the steady-state probabilities that are more efficient for grouped arrival and departure.

Calculating Cumulative Operational Time Distributions of Repairable Computer Systems

We consider repairable computer systems, those for which repair can be performed to put the system back in operation. The behavior of the system is assumed to be modeled as a homogeneous Markov process. We calculate numerically the distribution of cumulative operational time, which is the distribution of the total time during which the system was in operation over a finite observation period. The method is based on the randomization technique. The main advantages include the ability to specify error tolerances in advance, numerical stability, and simplicity of implementation. We also show that other quantities of interest can be calculated as a byproduct of the method without any significant extra computational effort.

Possible forms for dwell-time histograms from single-channel current records

Certain macromolecules embedded in the cell membranes of a variety of cells behave as gated ion-selective pores or channels. The length of time that a channel remains open or closed is not deterministic in nature and must be described in terms of relative probabilities. If channels act independently of each other and appropriate experimental conditions can be maintained, the behavior of a channel can be described by a homogeneous Markov process. Using this representation, the relative probability of observing openings (or closings) of various durations can be described by a sum of discrete components which are related to the underlying model of the kinetic behavior of the channel. Generally, these discrete components are taken to be simple decaying exponentials; however, exponentially decaying oscillatory components (as well as certain others which are discussed) are consistent with the Markov process representation. The presence of components other than simple decaying exponentials is shown to imply the violation of detailed balance in the steady-state (which requires energy), and thus, the presence of cyclic pathways in models which accurately represent the kinetic behavior of the channel. Oscillatory components, if present, will in general decay at a faster rate than the slowest decaying component, which, except under a very restricted set of conditions, will be a simple exponential.

Excursion Laws of Markov Processes in Classical Duality

We study excursion laws of two Markov processes $X, \hat{X}$ in duality, out of closed, homogeneous optional sets $M, \hat{M}$ generated by a pair of dual terminal times $R$ and $\hat{R}$. The duality assumptions enable one to compute the laws of $(R, X_R)$ using the pair of exit systems of the two processes. With this, one is able to compute the conditional laws of the excursions, given the boundary process. It turns out that these depend only on the values of the boundary process at the beginning and end of the excursions. We obtain for all $x, y(x \neq y)$ the laws $p^{x,y}$ of the excursions conditioned to start at $x$ and end at $y$. Under these laws, the excursion process is a homogeneous Markov process. It's transition laws are computed. We use the results above to treat excursions that straddle perfect terminal times.

Maximum likelihood estimation of subsequence conservation

A statistical method is presented for comparing protein sequences by partitioning the polymers and estimating each subsegment's degree of conservation. Conservation is measured as a function of the number of transitions occurring in the underlying time homogeneous Markov process assumed to govern amino acid mutations. The Markovian assumption also permits estimation of the ancestral sequence. Partitioning and estimation are carried out via maximum likelihood. The method is contrasted with the commonly utilized percent homology measure. A moving likelihood ratio plot to aid in identifying regions of high conservation is suggested as an analogue to moving hydrophobicity plots. An application is presented which identifies highly conserved regions in thymidylate synthase from L. casei and E. coli.