Homogeneous Continuous-time Markov Chain Research Articles

Entanglement invariants and phylogenetic branching

It is possible to consider stochastic models of sequence evolution in phylogenetics in the context of a dynamical tensor description inspired from physics. Approaching the problem in this framework allows for the well developed methods of mathematical physics to be exploited in the biological arena. We present the tensor description of the homogeneous continuous time Markov chain model of phylogenetics with branching events generated by dynamical operations. Standard results from phylogenetics are shown to be derivable from the tensor framework. We summarize a powerful approach to entanglement measures in quantum physics and present its relevance to phylogenetic analysis. Entanglement measures are found to give distance measures that are equivalent to, and expand upon, those already known in phylogenetics. In particular we make the connection between the group invariant functions of phylogenetic data and phylogenetic distance functions. We introduce a new distance measure valid for three taxa based on the group invariant function known in physics as the tangle. All work is presented for the homogeneous continuous time Markov chain model with arbitrary rate matrices.

Stability Estimates for Finite Homogeneous Continuous-time Markov Chains

This paper obtains new stability estimates on infinite time interval and limit stability estimates for a finite homogeneous continuous-time Markov chain with a unique stationary distribution. The connection between the stability of the Markov chain under perturbation of the generator and the rate of convergence to stationarity is considered. Markov chains with a strongly accessible state are given special attention.

Entanglement invariants and phylogenetic branching

It is possible to consider stochastic models of sequence evolution in phylogenetics in the context of a dynamical tensor description inspired from physics. Approaching the problem in this framework allows for the well developed methods of mathematical physics to be exploited in the biological arena. We present the tensor description of the homogeneous continuous time Markov chain model of phylogenetics with branching events generated by dynamical operations. Standard results from phylogenetics are shown to be derivable from the tensor framework. We summarize a powerful approach to entanglement measures in quantum physics and present its relevance to phylogenetic analysis. Entanglement measures are found to give distance measures that are equivalent to, and expand upon, those already known in phylogenetics. In particular we make the connection between the group invariant functions of phylogenetic data and phylogenetic distance functions. We introduce a new distance measure valid for three taxa based on the group invariant function known in physics as the "tangle". All work is presented for the homogeneous continuous time Markov chain model with arbitrary rate matrices.

Transient analysis of some rewarded Markov models using randomization with quasistationarity detection

Rewarded homogeneous continuous-time Markov chain (CTMC) models can be used to analyze performance, dependability and performability attributes of computer and telecommunication systems. In this paper, we consider rewarded CTMC models with a reward structure including reward rates associated with states and two measures summarizing the behavior in time of the resulting reward rate random variable: the expected transient reward rate at time t and the expected averaged reward rate in the time interval [0, t]. Computation of those measures can be performed using the randomization method, which is numerically stable and has good error control. However, for large stiff models, the method is very expensive. Exploiting the existence of a quasistationary distribution in the subset of transient states of discrete-time Markov chains with a certain structure, we develop a new variant of the (standard) randomization method, randomization with quasistationarity detection, covering finite CTMC models with state space S/spl cup/{f/sub 1/,f/sub 2/,..., f/sub A/}, A/spl ges/1, where all states in S are transient and reachable among them and the states f/sub i/ are absorbing. The method has the same good properties as the standard randomization method and can be much more efficient. We also compare the performance of the method with that of regenerative randomization.

The Markov Chain Market

We consider a financial market driven by a continuous time homogeneous Markov chain. Conditions for absence of arbitrage and for completeness are spelled out, non-arbitrage pricing of derivatives is discussed, and details are worked out for some cases. Closed form expressions are obtained for interest rate derivatives. Computations typically amount to solving a set of first order partial differential equations. An excursion into risk minimization in the incomplete case illustrates the matrix techniques that are instrumental in the model.

The Markov Chain Market

We consider a financial market driven by a continuous time homogeneous Markov chain. Conditions for absence of arbitrage and for completeness are spelled out, non-arbitrage pricing of derivatives is discussed, and details are worked out for some cases. Closed form expressions are obtained for interest rate derivatives. Computations typically amount to solving a set of first order partial differential equations. An excursion into risk minimization in the incomplete case illustrates the matrix techniques that are instrumental in the model.

Weak Convergence of a Planar Random Evolution to the Wiener Process

The weak convergence of the distributions of a symmetrical random evolution in a plane controlled by a continuous-time homogeneous Markov chain with n, n≥3, states to the distribution of a two-dimensional Brownian motion, as the intensity of transitions tends to infinity, is proved.

Modelling hierarchical systems by a continuous-time homogeneous Markov chain using two-wave panel data

An open hierarchical (manpower) system divided into a totally ordered set of k grades is discussed. The transitions occur only from one grade to the next or to an additional (k+1)th grade representing the external environment of the system. The model used to describe the dynamics of the system is a continuous-time homogeneous Markov chain with k+1 states and infinitesimal generator R = (r ij ) satisfying r ij = 0 if i > j or i + 1 < j ≤ k (i, j = 1,…,k+1), the transition matrix P between times 0 and 1 being P = exp R . In this paper, two-wave panel data about the hierarchical system are considered and the resulting fact that, in general, the maximum-likelihood estimated transition matrix cannot be written as an exponential of an infinitesimal generator R having the form described above. The purpose of this paper is to investigate when this can be ascribed to the effect of sampling variability.

Modelling hierarchical systems by a continuous-time homogeneous Markov chain using two-wave panel data

An open hierarchical (manpower) system divided into a totally ordered set of k grades is discussed. The transitions occur only from one grade to the next or to an additional (k+1)th grade representing the external environment of the system. The model used to describe the dynamics of the system is a continuous-time homogeneous Markov chain with k+1 states and infinitesimal generator R = (rij) satisfying rij = 0 if i > j or i + 1 < j ≤ k (i, j = 1,…,k+1), the transition matrix P between times 0 and 1 being P = expR. In this paper, two-wave panel data about the hierarchical system are considered and the resulting fact that, in general, the maximum-likelihood estimated transition matrix cannot be written as an exponential of an infinitesimal generator R having the form described above. The purpose of this paper is to investigate when this can be ascribed to the effect of sampling variability.

Modelling hierarchical systems by a continuous-time homogeneous Markov chain using two-wave panel data

Modelling hierarchical systems by a continuous-time homogeneous Markov chain using two-wave panel data

Compatibility of multi‐wave panel data and the continuous‐time homogeneous Markov chain. An analysis of a continuous‐time process by means of discrete‐time longitudinal observations

Multi-wave panel data are observations at two or more points in time on a continuously changing attribute of interest (e.g. behaviour). In this paper, the adequacy of the continuous-time homogeneous Markov chain (CTHMC) model is assessed for describing the process of change underlying such data. In the case of equidistant observational times, it may happen that the maximum-likelihood estimate of the transition probability matrix between successive observational times from these data cannot arise from a CTHMC. It is investigated whether this event can be ascribed to chance through the introduction of an hypothesis test. © 1998 John Wiley & Sons, Ltd.

Compatibility of multi-wave panel data and the continuous-time homogeneous Markov chain. An analysis of a continuous-time process by means of discrete-time longitudinal observations

Multi-wave panel data are observations at two or more points in time on a continuously changing attribute of interest (e.g. behaviour). In this paper, the adequacy of the continuous-time homogeneous Markov chain (CTHMC) model is assessed for describing the process of change underlying such data. In the case of equidistant observational times, it may happen that the maximum-likelihood estimate of the transition probability matrix between successive observational times from these data cannot arise from a CTHMC. It is investigated whether this event can be ascribed to chance through the introduction of an hypothesis test.

A finite capacity multi-server multi-queueing priority model with non-renewal input

We consider a non-preemptive head-of-the-line multi-server multi-queueing priority model with finite buffer capacity for each priority class. As an arrival process, a generalized Markovian arrival process with marked transitions is used. The service-time distribution is of phase-type and identical for the different priority classes. The model is described by a homogeneous continuous-time Markov chain (CTMC). From the steady-state distribution of the CTMC, which is calculated by matrix-geometric methods, we derive the steady-state distributions immediately after arrival instants of the different priority classes. Applying matrix-analytic methods, we calculate the Laplace-Stieltjes Transform (LST) of the actual waiting times for the different priority classes.

Waiting times of a finite-capacity multi-server model with non-preemptive priorities

We consider a non-preemptive head of the line multi-server priority model with finite capacity. The arrival processes of the different priority classes are independent Poisson processes. The service times are exponentially distributed and identical for the different priority classes. The model is described by a homogeneous continuous-time Markov chain. For the two-class model we derive an explicit representation of its steady-state distribution. Applying matrix-analytic methods we calculate the Laplace-Stieltjes Transform of the actual waiting time of each priority class of a p-class system.

Continuous-time markov chains and compound poisson process with circulant intensity matrices

A classical result states that for a finite-state homogeneous continuous-time Markov chain with finite state space and intensity matrix Q=(qk) the matrix of transition probabilities is given by . In contrast to this infinite series expression, we derive an explicit finite sum formula for this transition matrix for situations where Q is a circulant matrix. Our result is motivated by the compound Poisson process (with discrete random i.i.d. variable Yj and a Poisson counting process It is shown that the stochastic process Xt = Dt mod n is a Markov process on E with a circulant intensity matrix Q and we apply the previous results to calculate, e.g., the distribution and the expectation of Xt The process provides a stochastic model for,e.g., channel assignment in telecommunication, bus occupancies, box packing etc.

Uniformization for nonhomogeneous Markov chains

The discrete Poissonian representation for transition probabilities of homogeneous continuous-time Markov chains, known as uniformization or randomization, is extended to time-inhomogeneous chains. For computational purposes a discrete-time approximation is also provided.

Likelihood ratio gradient estimation for stochastic systems

Consider a computer system having a CPU that feeds jobs to two input/output (I/O) devices having different speeds. Let θ be the fraction of jobs routed to the first I/O device, so that 1 - θ is the fraction routed to the second. Suppose that α = α(θ) is the steady-sate amount of time that a job spends in the system. Given that θ is a decision variable, a designer might wish to minimize α(θ) over θ. Since α(·) is typically difficult to evaluate analytically, Monte Carlo optimization is an attractive methodology. By analogy with deterministic mathematical programming, efficient Monte Carlo gradient estimation is an important ingredient of simulation-based optimization algorithms. As a consequence, gradient estimation has recently attracted considerable attention in the simulation community. It is our goal, in this article, to describe one efficient method for estimating gradients in the Monte Carlo setting, namely the likelihood ratio method (also known as the efficient score method). This technique has been previously described (in less general settings than those developed in this article) in [6, 16, 18, 21]. An alternative gradient estimation procedure is infinitesimal perturbation analysis; see [11, 12] for an introduction. While it is typically more difficult to apply to a given application than the likelihood ratio technique of interest here, it often turns out to be statistically more accurate. In this article, we first describe two important problems which motivate our study of efficient gradient estimation algorithms. Next, we will present the likelihood ratio gradient estimator in a general setting in which the essential idea is most transparent. The section that follows then specializes the estimator to discrete-time stochastic processes. We derive likelihood-ratio-gradient estimators for both time-homogeneous and non-time homogeneous discrete-time Markov chains. Later, we discuss likelihood ratio gradient estimation in continuous time. As examples of our analysis, we present the gradient estimators for time-homogeneous continuous-time Markov chains; non-time homogeneous continuous-time Markov chains; semi-Markov processes; and generalized semi-Markov processes. (The analysis throughout these sections assumes the performance measure that defines α(θ) corresponds to a terminating simulation.) Finally, we conclude the article with a brief discussion of the basic issues that arise in extending the likelihood ratio gradient estimator to steady-state performance measures.

The effect of execution policies on the semantics and analysis of stochastic Petri nets

Petri nets in which random delays are associated with atomic transitions are defined in a comprehensive framework that contains most of the models already proposed in the literature. To include generally distributed firing times into the model one must specify the way in which the next transition to fire is chosen, and how the model keeps track of its past history; this set of specifications is called an execution policy. A discussion is presented of the impact that different execution policies have on semantics of the mode, as well as the characteristics of the stochastic process associated with each of these policies. When the execution policy is completely specified by the transition with the minimum delay (race policy) and the firing distributions are of the phase type, an algorithm is provided that automatically converts the stochastic process into a continuous time homogeneous Markov chain. An execution policy based on the choice of the next transition to fire independently of the associated delay (preselection policy) is introduced, and its semantics is discussed together with possible implementation strategies. >

Modeling of Concurrent Task Execution in a Distributed System for Real-Time Control

In a distributed system that implements real-time control, computational tasks are distributed over different nodes for execution to improve response time and system reliability. To model system behavior, tasks in each node are first decomposed into activities. The activities and precedence constraints among them are then modeled by a generalized stochastic Petri net (GSPN). Finally, a sequence of homogeneous continuous-time Markov chains (CTMC's) is built from the GSPN to model the concurrent task execution in the system.

Exact Inference for Continuous Time Markov Chain Models

Methods for exact Bayesian inference under a uniform diffuse prior are set forth for the continuous time homogeneous Markov chain model. It is shown how the exact posterior distribution of any function of interest may be computed using Monte Carlo integration. The solution handles the problems of embeddability in a very natural way, and provides (to our knowledge) the only solution that systematically takes this problem into account. The methods are illustrated using several sets of data.