Markovian Methods Research Articles

Association of multivariate phase-type distributions, with applications to shock models

A random vector is said to be of (multivariate) phase-type if it can be represented as the vector of random times until absorptions into various stochastically closed subsets of the finite state space in an absorbing Markov chain. The phase-type distributions are useful since Markovian methods may be applicable in the situations where one adopts a (univariate or multivariate) phase-type distribution for time intervals that are needed in setting up a stochastic model. This paper studies the dependence nature of multivariate phase-type distributions and some related shock models, and it shows that under some mild conditions, the multivariate phase-type distributions are positively associated. The association properties for the lifetimes of components operating in some common shock environments are also obtained.

Markovian models for the developmental study of social behavior.

Behavioral development involves changes in the probabilities of both social and nonsocial activities and the sequential pattern of activities over time. A number of methods have been offered for the analysis of these patterns of behavioral sequences. However, there continue to be problematic issues, including the analysis of nonstationary data; accommodation of changes in patterns within an observation period, or over repeated observations or age; and identification of differences in pattern changes between individuals or groups, and the factors responsible for these differences. In this work, we analyze data from 15 young monkeys (Macaca nemestrina) using classification and Markovian methods, including a new approach to nonstationary data called the double-chain Markov model (DCCM). These methods allowed us to identify differences in behavior patterns that differentiate between normal subjects and those presenting developmental anomalies.

The stationary G/G/s queue with non‐identical servers

We extend a recently developed factorization method to the case of the G/G/s queue with non‐identical servers, by presenting three simple properties which lead to a simple numerical calculation method. We compare our results with those determined by classical Markovian (phase) methods in the case of the symmetrical M/G/s queue, and for the mean queueing delay we compare with results given by traffic simulation.

A new class of performance results for a fractional Brownian traffic model

Recent Bellcore studies have shown that high-speed data traffic exhibits “long-range dependence”, characterized byH>0.5, whereH is the Hurst parameter of the traffic. In the wake of those studies, there has been much interest in developing tractable analytical models for traffic with long-range dependence, for use in performance evaluation and traffic engineering. Norros has used a traffic model known as Fractional Brownian Motion (FBM) to derive several analytical results on the behavior of a queue subject to such an arrival process. In this paper, we derive a new class of results, also based on the FBM model, which reveal rather curious and unexpected “crossover” properties of the Hurst parameter of the traffic, as regards its effect on the behavior of queues. These results, together with those of Norros, serve to enhance our understanding of the significance of the Hurst parameterH for traffic engineering. In particular, Krishnan and Meempat have used the crossover property derived here to explain, in part, a gap that existed between the results of two sets of Bellcore studies, one casting doubt on the usefulness of Markovian traffic models and methods whenH>0.5, and the other furnishing an example of successful traffic engineering with Markovian methods for traffic known to haveH>0.5. The results derived here can be used to obtainconservative estimates of the multiplexing gains achieved when independent traffic sources with the same Hurst parameterH are multiplexed for combined transmission. In turn, such estimates yield guidelines for the engineering of ATM links that are subject to traffic with long-range dependence.

Optimal portfolio for a small investor in a market model with discontinuous prices

A consumption-investment problem is considered for a small investor in the case of a market model in which prices evolve according to a stochastic equation with a jump-process component. The techniques we use include the martingale representation theorem, Lagrange multiplier methods, and Markovian methods for the resolution of stochastic differential equations. We establish a Black-Scholes formula.

Modeling Time-Varying Dynamical Systems

Abstract A global methodology of identification, estimation and forecasting of transfer function (Box-Jenkins) models with deterministically varying parameters is provided. First, properties of stability and forecasting algorithms are investigated by means of Markovian representations and methods of solution of nonstationary difference equations. Next, the degree of the polynomials of the system is specified with typical off-line methods, and the shape of the coefficients (parameter functions) is identified by means of recursive (on-line) algorithms. Finally, the identified parameter functions are inserted in the model and their coefficients are estimated (off-line) on the original data by means of pseudolinear regression techniques.

Modeling Time-Varying Dynamical Systems

Abstract A global methodology of identification, estimation and forecasting of transfer function (Box-Jenkins) models with deterministically varying parameters is provided. First, properties of stability and forecasting algorithms are investigated by means of Markovian representations and methods of solution of nonstationary difference equations. Next, the degree of the polynomials of the system is specified with typical off-line methods, and the shape of the coefficients (parameter functions) is identified by means of recursive (on-line) algorithms. Finally, the identified parameter functions are inserted in the model and their coefficients are estimated (off-line) on the original data by means of pseudolinear regression techniques.

Efficient Monte Carlo Procedures for Generating Points Uniformly Distributed over Bounded Regions

We consider the Monte Carlo problem of generating points uniformly distributed within an arbitrary bounded (measurable) region. The class of Markovian methods considered generate points asymptotically uniformly distributed within the region. Computational experience suggests the methods are potentially superior to conventional rejection techniques for large dimensional regions.

An analysis of Markovian model microfield methods for stark broadening

Stark broadening theories which concentrate on the statistics of the plasma electric microfield rather than the dynamics of collisions have come to be known as Model Microfield Methods. In the present paper we present an analysis of Stark broadening problems based on Markovian model microfield statistics. Our derivation permits an easy comparison with traditional Stark broadening theories such as the impact and unified theories; this comparison is used to clarify the physical nature of the approximations employed by Markovian models. The strengths and weaknesses of various models are discussed, emphasizing the kangaroo process of Brissaud and Frisch, and methods are suggested for improving the current model microfield approach.

Distributional errors in reliability

The complexity of distributional error analyses has handicapped the revival of analytical methods of reliability being based on the exponentiality of time to unit failure and, most often, repair. The assertation, that these hypothesis hardly affect the calculation of reliability of strongly redundant systems with reliable units, is substantiated by studying parallel systems with identical units. As a by-product a surprising bridge between reliability and steady-state availability is established. This result is obtained both by Markovian theory and methods free of any distributional hypothesis.

Behavioral contracting within the families of delinquents

The technique of behavioral contracting is used to strengthen the control of family and school over the behavior of delinquents. A behavioral contract is a means of scheduling the exchange of positive reinforcements among two or more persons. The use of these contracts is predicated upon four assumptions: (1) receipt of positive reinforcements in interpersonal exchanges is a privilege rather than a right; (2) effective interpersonal agreements are governed by the norm of reciprocity; (3) the value of an interpersonal exchange is a direct function of the range, rate and magnitude of the positive reinforcements mediated by that exchange; and (4) rules create freedom in interpersonal exchanges. The use of a behavioral contract with one delinquent girl is described and analyzed using Markovian methods.

Stochastic Processes Occurring in the Theory of Queues and their Analysis by the Method of the Imbedded Markov Chain

The stochastic processes which occur in the theory of queues are in general not Markovian and special methods are required for their analysis. In many cases the problem can be greatly simplified by restricting attention to an imbedded Markov chain. In this paper some recent work on single-server queues is reviewed from this standpoint, and the method is then applied to the analysis of the following many-server queuing-system: Input: the inter-arrival times are independently and identically distributed in an arbitrary manner. Queue-discipline: first come, served. Service-mechanism: a general number, $s$, of servers; negative-exponential service-times. If $Q$ is the number of people waiting at an instant just preceding the arrival of a new customer, and if $w$ is the waiting time of an arbitrary customer, then it will be shown that the equilibrium distribution of $Q$ is a geometric series mixed with a concentration at $Q = 0$ and that the equilibrium distribution of $w$ is a negative-exponential distribution mixed with a concentration at $w = 0$. (In the particular case of a single server this property of the waiting-time distribution was discovered by W. L. Smith.) The paper concludes with detailed formulae and numerical results for the following particular cases: Numbers of servers: s = 1, 2 and 3. Types of input: (i) Poissonian and (ii) regular.