Discrete-state Markov Chain Research Articles

Analyzing the effect of permanent, intermittent and transient faults on a gracefully degrading microcomputer

In this paper investigations into the effect of permanent, intermittent and transient faults on a gracefully degrading microcomputer are reported. Two discrete-state continuous time Markov chains are used in order to model the fault/error handling and the fault occurence repair processes. The models are separately solved with the aid of decomposition/aggregation technique. A numerical algorithm is also used in order to evaluate performability of the microcomputer. Finally, the influence of permanent, intermittent and transient faults on this performance related reliability measure are analyzed.

Reliability of fully and partially replicated systems

The reliability of two replication methods, partial and full, is analyzed for a two-process/four-processor case. Three types of systems are analyzed and compared: no repair, finite repair-rate and instantaneous repair. These systems are modeled using discrete-state continuous-time Markov chains. The condition under which a partially replicated system might yield a better reliability than that of a fully replicated system is quantified and expressed in terms of system design parameters. Partial replication is most favorable in systems without repair capability and this advantage is manifested most when the underlying hardware is unreliable. >

On a simple proof of uniformization for continuous and discrete-state continuous-time Markov chains

Uniformization has proven to be a convenient tool for modeling and computational purposes to analyze continuous-time Markov chain applications. For the case of continuous-state space, uniformization is also intuitively obvious and most likely employed in practice. This note merely aims to show that uniformization for both the discrete- and continuous state cases is directly formalized by a simple uniqueness result from the literature

On a simple proof of uniformization for continuous and discrete-state continuous-time Markov chains

An abstract is not available for this content so a preview has been provided. As you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Numerical transient analysis of markov models

We consider the numerical evaluation of Markov model transient behavior. Our research is motivated primarily by computer system dependability modeling. Other application areas include finitecapacity queueing models, closed queueing networks and inventory models. We focus our attention on the general problem of finding the state probability vector of a large, continuous-time, discrete-state Markov chain. Two computational approaches are examined in detail: uniformization and numerical linear multistep methods for ordinary differential equation solution. In general, uniformization provides greater accuracy but deals poorly with stiffness. A special stable ordinary differential equation solver deals well with stiffness, but it provides increased accuracy only at much greater cost. Examples are presented to illustrate the behavior of the techniques discussed as a function of model size, model stiffness, increased accuracy requirements and mission time.

Two remarks on insensitive stochastic models

This paper contains two contributions to the insensitivity theory for generalized semi-Markov schemes, namely (1) a detailed account of a close relationship between insensitive schemes and partially balanced discrete-state Markov chains, and (2) an extension of the notion of an insensitive scheme in order to incorporate insensitivity phenomena not previously covered.

Two remarks on insensitive stochastic models

This paper contains two contributions to the insensitivity theory for generalized semi-Markov schemes, namely (1)a detailed account of a close relationship between insensitive schemes and partially balanced discrete-state Markov chains, and(2)an extension of the notion of an insensitive scheme in order to incorporate insensitivity phenomena not previously covered.

The Uniqueness of Stationary Distributions for the GI/G/S Queue

Kiefer and Wolfowitz (Kiefer, J., J. Wolfowitz. 1955. On the theory of queues with many servers. Amer. Math. Soc. 78 1–18.) showed that there exists a unique proper stationary limiting distribution for the stochastic process representing the workload at arrival epochs in a multiple-server queue when the utilization per server is less than one. An alternate proof of the uniqueness is presented here which avoids using approximating discrete-state Markov chains.

A Stochastic Model for Change in Group Structure

Abstract This paper develops and tests a continuous time discrete state Markov Chain as a model for change in group structure. Movement of triads among different states is focused upon. It is shown formally that a triad cannot move to a state where all the pair relationships are mutual without incurring intransitivity. An empirical test on the change in sociometric structures provides support for the model.

A stochastic model for change in group structure

This paper develops and tests a continuous time discrete state Markov Chain as a model for change in group structure. Movement of triads among different states is focused upon. It is shown formally that a triad cannot move to a state where all the pair relationships are mutual without incurring intransitivity. An empirical test on the change in sociometric structures provides support for the model.

Recurrence times of draft-patterns from reservoirs

Expressions for the mean and variance of the recurrence time of non-overlapping draft-patterns of draft from a Moran Reservoir Model (discrete-state and discrete-time Markov chain) are derived using Feller's Renewal argument. In addition an expression for the mean recurrence time for self-overlapping patterns of draft is derived using run-theory.

Recurrence times of draft-patterns from reservoirs

Expressions for the mean and variance of the recurrence time of non-overlapping draft-patterns of draft from a Moran Reservoir Model (discrete-state and discrete-time Markov chain) are derived using Feller's Renewal argument. In addition an expression for the mean recurrence time for self-overlapping patterns of draft is derived using run-theory.

Markov chains and embedded Markov chains in geology

Geological data are structured as first-order, discrete-state discrete-time Markov chains in two main ways. In one, observations are spaced equally in time or space to yield transition probability matrices with nonzero elements in the main diagonal; in the other, only state transitions are recorded, to yield matrices with diagonal elements exactly equal to zero. The mathematical differences in these two approaches are reviewed here, using stratigraphic data as an example. Simulations from chains with diagonal elements greater than zero always yield geometric distributions of lithologic unit thickness, and their use is recommended only if the input data have the same distribution. For thickness distributions lognormally or otherwise distributed, the embedded chain is preferable. The mathematical portions of this paper are well known, but are not readily available in publications normally used by geologists. One purpose of this paper is to provide an explicit treatment of the mathematical foundations on which applications of Markov processes in geology depend.