Continuous-time Discrete-state Markov Process Research Articles

Exact analysis of summary statistics for continuous-time discrete-state Markov processes on networks using graph-automorphism lumping

We propose a unified framework to represent a wide range of continuous-time discrete-state Markov processes on networks, and show how many network dynamics models in the literature can be represented in this unified framework. We show how a particular sub-set of these models, referred to here as single-vertex-transition (SVT) processes, lead to the analysis of quasi-birth-and-death (QBD) processes in the theory of continuous-time Markov chains. We illustrate how to analyse a number of summary statistics for these processes, such as absorption probabilities and first-passage times. We extend the graph-automorphism lumping approach [Kiss, Miller, Simon, Mathematics of Epidemics on Networks, 2017; Simon, Taylor, Kiss, J. Math. Bio. 62(4), 2011], by providing a matrix-oriented representation of this technique, and show how it can be applied to a very wide range of dynamical processes on networks. This approach can be used not only to solve the master equation of the system, but also to analyse the summary statistics of interest. We also show the interplay between the graph-automorphism lumping approach and the QBD structures when dealing with SVT processes. Finally, we illustrate our theoretical results with examples from the areas of opinion dynamics and mathematical epidemiology.

A Unified Approach to Reliability, Availability, Performability Analysis Based on Markov Processes with Rewards

This paper discusses a unified approach to reliability, availability and performability analysis of complex engineering systems. Theoretical basis of this approach is continuous-time discrete state Markov processes with rewards. From reliability modeling point of view complex systems are the systems with static and dynamic redundancy, imperfect fault coverage, various recovery strategies, multilevel operation and varying severity of failure states. We propose a unified method of calculating the reliability, availability and performability indices based on the definition of special forms of reward matrix. This method proved to be effective in calculating both cumulative and instantaneous measures in steady-state and transient cases. We describe special analytical software which implements suggested method. We demonstrate the flexibility of the proposed method and software by analyzing multilevel process unit with protection and demand-based warm standby system.

A master equation and moment approach for biochemical systems with creation-time-dependent bimolecular rate functions.

Noise and stochasticity are fundamental to biology and derive from the very nature of biochemical reactions where thermal motion of molecules translates into randomness in the sequence and timing of reactions. This randomness leads to cell-to-cell variability even in clonal populations. Stochastic biochemical networks have been traditionally modeled as continuous-time discrete-state Markov processes whose probability density functions evolve according to a chemical master equation (CME). In diffusion reaction systems on membranes, the Markov formalism, which assumes constant reaction propensities is not directly appropriate. This is because the instantaneous propensity for a diffusion reaction to occur depends on the creation times of the molecules involved. In this work, we develop a chemical master equation for systems of this type. While this new CME is computationally intractable, we make rational dimensional reductions to form an approximate equation, whose moments are also derived and are shown to yield efficient, accurate results. This new framework forms a more general approach than the Markov CME and expands upon the realm of possible stochastic biochemical systems that can be efficiently modeled.

A score-test on measurement errors in rating transition times

We model credit rating histories as continuous-time discrete-state Markov processes. Infrequent monitoring of the debtors’ solvency will result in erroneous observations of the rating transition times, and consequently in biased parameter estimates. We develop a score test against such measurement errors in the transition data that is independent of the error distribution. We derive the asymptotic χ2-distribution for the test statistic under the null by stochastic limit theory. The test is applied to an international corporate portfolio, while accounting for economic and debtor-specific covariates. The test indicates that measurement errors in the transition times are a real problem in practice.

CO oxidation on Ir(111) surfaces under large non-Gaussian noise

In this article we consider the CO oxidation on Ir(111) surfaces under large external noise with large autocorrelation imposed on the composition of the feed gas, both in experiments and in theory. We report new experimental results that show how the fluctuations force the reaction rate to jump between two well defined states. The statistics of the reaction rate depend on those of the external noise, and neither of them have a gaussian distribution, and thus they cannot be modeled by white or colored noise. A continuous-time discrete-state Markov process is proposed as a suitable model for the observed phenomena. The model captures the main features of the observed fluctuations and can be modified to accommodate other surface reactions and other systems under non-gaussian external noise.

H∞ control for sampled-data linear systems with two Markov processes

This paper is concerned with the design problem of H∞ digital switching control for linear continuous systems with Markovian jumping parameters. The controller is digital and monitored by the jumping parameters of the plant. The closed-loop system is a hybrid one defined on a hybrid time space (composed of a continuous-time and a discrete-time) and a sample space. The sample space is specified by two separable continuous-time discrete-state Markov processes, one appearing in the open-loop system, and the other appearing in control action, which is different with the traditional Markovian jumping process. Our attention is focused on designing digital output feedback controllers for the system with two Markovian jumping processes such that both stochastic stability and a prescribed H∞ performance are achieved. The problem of robust H∞ control for systems with parameters uncertainties is also studied. It is shown that the sampled-data control problems for linear Markovian jumping systems with and without parameter uncertainties can be solved in terms of the solutions to a set of intercoupled matrix inequalities. Two numerical examples are given to show the design procedures. Copyright © 2005 John Wiley & Sons, Ltd.

Control of singularly perturbed systems with Markovian jump parameters: an H∞ approach

In this paper, we study the problem of H ∞ control for singularly perturbed linear continuous-time systems with Markovian jump parameters. The system under consideration is described by a state-space model, consisting of slow and fast dynamics with a continuous-time discrete-state Markov process taking values on a finite set. On the basis of reduced-order technique, our attention is focused on the designing of state feedback control such that both stochastic stability and a prescribed H ∞ performance for the full-order system are achieved. It has been shown that the above problem can be solved if a set of coupled Riccati equations has solution. Furthermore, the asymptotic structure of composite mode-dependent controller is characterized, which shows that the controller is independent of the singular perturbation ε, when ε is sufficiently small.

Some Models for Discretized Series of Events

A discretized series of events is a binary time series that indicates whether or not events of a point process in the line occur in successive intervals. Such data are common in environmental applications. We describe a class of models for them, based on an unobserved continuous-time discrete-state Markov process, which determines the rate of a doubly stochastic Poisson process, from which the binary time series is constructed by discretization. We discuss likelihood inference for these processes and their second-order properties and extend them to multiple series. An application involves modeling the times of exposures to air pollution at a number of receptors in Western Europe.

Some Models for Discretized Series of Events

Abstract A discretized series of events is a binary time series that indicates whether or not events of a point process in the line occur in successive intervals. Such data are common in environmental applications. We describe a class of models for them, based on an unobserved continuous-time discrete-state Markov process, which determines the rate of a doubly stochastic Poisson process, from which the binary time series is constructed by discretization. We discuss likelihood inference for these processes and their second-order properties and extend them to multiple series. An application involves modeling the times of exposures to air pollution at a number of receptors in Western Europe.

Markovian-model for systems with independent units

This paper deals with a system of n-independent repairable units that can be described by a homogeneous continuous-time discrete-state Markov process. Transition probabilities matrix of the system has the form of a modified Kronecker sum of transition-rate matrices of its units is investigated. The set of 2 n differential equations are fully described. The case of steady state of availability are treated as special cases. Sample of results are presented for testing the consistency of the model.

Multistate Markov models for systems with dependent units

A system that can be described by a homogeneous continuous-time discrete-state Markov process is treated. The case in which transition rates for each unit depend on the current state of the system is considered. The condition in which the transition-rate matrix of the system has the form of a modified Kronecker sum of transition-rate matrices of its units is investigated. An algorithm based on the Kronecker algebra is introduced for determining the transition-rate matrix of the system. its use is particularly efficient during construction of machines when reliability is evaluated for several systems with the same structure but different transition rates. >