Continuous-time Markov Chain Research Articles

Mutual Linearity of Nonequilibrium Network Currents.

For continuous-time Markov chains and open unimolecular chemical reaction networks, we prove that any two stationary currents are linearly related upon perturbations of a single edge's transition rates, arbitrarily far from equilibrium. We extend the result to nonstationary currents in the frequency domain, provide and discuss an explicit expression for the current-current susceptibility in terms of the network topology, and discuss possible generalizations. In practical scenarios, the mutual linearity relation has predictive power and can be used as a tool for inference or model proof testing.

A genetic algorithm and particle swarm optimization for redundancy allocation problem in systems with limited number of non-cooperating repairmen

This article considered the redundancy allocation problem (RAP) in repairable heterogeneous systems composed of k-out-of-n: G subsystems with a limited number of repairmen. The unit costs of the binary-state components within the subsystems remain constant over time. Exponential characteristics of independent failure and repair processes were assumed. To evaluate subsystem and system availability, a Continuous Time Markov Chain (CTMC) adapted to three standby modes (cold, warm, and hot) was developed. The characteristics of the redundant system and the assumption of a limited number of repairmen contribute to the existing scientific literature on RAP. Two nature-inspired metaheuristic approaches were developed to maximize the availability of the coherent system as an objective function to optimize redundancy with a cost constraint on component purchase. The proposed approaches were tested on a 15-unit large-scale system. The effectiveness and performance of the algorithms was evaluated as a function of the number of generations/iterations and population/swarm size. The genetic algorithm (GA) proved to be highly effective in finding the global optimum and outperformed particle swarm optimization (PSO) in terms of computational performance. Analysis of metaheuristic algorithms can offer valuable insights into future research on redundancy optimization. The sensitivity analysis of the large-scale system clearly demonstrates that the number of repairmen and cost constraints (up to three times the sum of the value of active components) have a significant impact on the system’s availability. Above this level, increasing spending on system building, regardless of the number of repairmen, is not justified by considerations of system availability.

Stability of Queueing Systems with Impatience, Balking and Non-Persistence of Customers

The operation of many queueing systems is adequately described by the structured multidimensional continuous-time Markov chains. The most well-studied classes of such chains are level-independent Quasi-Birth-and-Death processes, GI/M/1 type and M/G/1 type Markov chains, generators of which have the block tri-diagonal, lower- and upper-Hessenberg structure, respectively. All these classes assume that the matrices of transition rates are quasi-Toeplitz. This property greatly simplifies their analysis but makes them inappropriate for the study of many important systems, e.g., retrial queues with a retrial rate depending on the number of customers in orbit, queues with impatient customers, etc. The importance of such systems attracts significant interest to their analysis. However, in the literature, there is a methodological gap relating to the ergodicity condition of the corresponding Markov chains. To fulfill this gap and facilitate the analysis of a wide range of such systems, we show that under non-restrictive assumptions, the following hold true: (i) if the customers can balk or are impatient or non-persistent, then the Markov chain describing the behavior of the system belongs to the class of asymptotically quasi-Toeplitz Markov chains; (ii) this chain is ergodic; (iii) known algorithms can be applied for the calculation of the stationary distribution of the corresponding queueing system.

Stochastic filtering of reaction networks partially observed in time snapshots

Stochastic reaction network models arise in intracellular chemical reactions, epidemiological models and other population process models, and are a class of continuous time Markov chains which have the nonnegative integer lattice as state space. We consider the problem of estimating the conditional probability distribution of a stochastic reaction network given exact partial state observations in time snapshots. We propose a particle filtering method called the targeting method. Our approach takes into account that the reaction counts in between two observation snapshots satisfy linear constraints and also uses inhomogeneous Poisson processes as proposals for the reaction counts to facilitate exact interpolation. We provide rigorous analysis as well as numerical examples to illustrate our method and compare it with other alternatives.

Quantum-embeddable stochastic matrices

The classical embeddability problem asks whether a given stochastic matrix T, describing transition probabilities of a d-level system, can arise from the underlying homogeneous continuous-time Markov process. Here, we investigate the quantum version of this problem, asking of the existence of a Markovian quantum channel generating state transitions described by a given T. More precisely, we aim at characterising the set of quantum-embeddable stochastic matrices that arise from memoryless continuous-time quantum evolution. To this end, we derive both upper and lower bounds on that set, providing new families of stochastic matrices that are quantum-embeddable but not classically-embeddable, as well as families of stochastic matrices that are not quantum-embeddable. As a result, we demonstrate that a larger set of transition matrices can be explained by memoryless models if the dynamics is allowed to be quantum, but we also identify a non-zero measure set of random processes that cannot be explained by either classical or quantum memoryless dynamics. Finally, we fully characterise extreme stochastic matrices (with entries given only by zeros and ones) that are quantum-embeddable.

VIX Options Valuation via Continuous-Time Markov Chain Approximation and Ito-Taylor Expansion

VIX Options Valuation via Continuous-Time Markov Chain Approximation and Ito-Taylor Expansion

Seasonal variability and stochastic branching process in malaria outbreak probability

Background: Malaria is the world’s most fatal and challenging parasitic disease, caused by the Plasmodium parasite, which is transmitted to humans by the bites of infected female mosquitoes. Bangladesh is the most vulnerable region to spread malaria because of its geographic position. In this paper, we have considered the dynamics of vector-host models and observed the stochastic behavior. This study elaborates on the seasonal variability and calculates the probability of disease outbreaks. Methods: We present a model for malaria disease transmission and develop its corresponding continuous-time Markov chain (CTMC) representation. The proposed vector-host models illustrate the malaria transmission model along with sensitivity analysis. The deterministic model with CTMC curves is depicted to show the randomness in real scenarios. Sequentially, we expand these studies to a time-varying stochastic vector-host model that incorporates seasonal variability. Phase plane analysis is conducted to explore the characteristics of the disease, examine interactions among various compartments, and evaluate the impact of key parameters. The branching process approximation is developed for the corresponding vector-host model to calculate the probability outbreak. Numerous numerical results are accomplished to observe the analytical investigation. Results: Seasonality and contact patterns affect the dynamics of disease outbreaks. The numerical illustration provides that the probability of a disease outbreak depends on the infected host or vector. Additionally, periodic transmission rates have a great influence on the probability outbreak. The basic reproduction number (R0) is derived, which is the main justification for studying the dynamical behavior of epidemic models. Conclusions: Seasonal variability significantly impacts malaria transmission, and the probability of disease outbreaks is influenced by time and the initial number of infected individuals. Moreover, the branching process approximation is applicable when the population size is large enough and the basic reproduction number is less than 1. In the future, such analysis can help decision-makers understand the impact of various parameters and their stochastic behavior in the vector-host model to prevent such types of disease outbreaks.

One-dimensional continuous-time quantum Markov chains: qubit probabilities and measures

Quantum Markov chains (QMCs) are positive maps on a trace-class space describing open quantum dynamics on graphs. Such objects have a statistical resemblance with classical random walks, while at the same time they allow for internal (quantum) degrees of freedom. In this work we study continuous-time QMCs on the integer line, half-line and finite segments, so that we are able to obtain exact probability calculations in terms of the associated matrix-valued orthogonal polynomials and measures. The methods employed here are applicable to a wide range of settings, but we will restrict ourselves to classes of examples for which the Lindblad generators are induced by a single positive map, and such that the Stieltjes transforms of the measures and their inverses can be calculated explicitly.

Latent classification model for censored longitudinal binary outcome.

Latent classification model is a class of statistical methods for identifying unobserved class membership among the study samples using some observed data. In this study, we proposed a latent classification model that takes a censored longitudinal binary outcome variable and uses its changing pattern over time to predict individuals' latent class membership. Assuming the time-dependent outcome variables follow a continuous-time Markov chain, the proposed method has two primary goals: (1) estimate the distribution of the latent classes and predict individuals' class membership, and (2) estimate the class-specific transition rates and rate ratios. To assess the model's performance, we conducted a simulation study and verified that our algorithm produces accurate model estimates (ie, small bias) with reasonable confidence intervals (ie, achieving approximately 95% coverage probability). Furthermore, we compared our model to four other existing latent class models and demonstrated that our approach yields higher prediction accuracies for latent classes. We applied our proposed method to analyze the COVID-19 data in Houston, Texas, US collected between January first 2021 and December 31st 2021. Early reports on the COVID-19 pandemic showed that the severity of a SARS-CoV-2 infection tends to vary greatly by cases. We found that while demographic characteristics explain some of the differences in individuals' experience with COVID-19, some unaccounted-for latent variables were associated with the disease.

Integrated analysis of reliability, power, and performance for IoT devices and servers

The Internet of Things (IoT) is currently widely used in various sectors and spaces. IoT devices are becoming small yet powerful servers and perform server-like functions. Reliability is a critical aspect in both IoT devices and servers, as they work together to create a robust and dependable IoT ecosystem. Power and performance are two other major considerations of an IoT system. Modeling, analysis, evaluation, and optimization of reliability, power, and performance for IoT devices and servers are major components in IoT systems development and deployment. In this paper, we conduct an integrated study of reliability, power, and performance for IoT devices and servers by mathematically rigorous modeling and analysis. The contributions of the paper can be summarized as follows. We establish a continuous-time Markov chain (CTMC) model that incorporates server failure rate, server repair rate, task arrival rate, and task processing rate. Using such an analytical model, we can calculate the server availability, the average task response time, and the average power consumption. We point out that there is an optimal server speed that minimizes the power-time product and a combined cost-performance metric of power, performance, and reliability. We show the impact of server reliability on response time, power consumption, server utilization, and the power-performance tradeoff. To the best of the author’s knowledge, this is the first paper that takes a combined approach to modeling and analysis of reliability, power, and performance for IoT devices and servers. It has been noticed that there has been little such theoretically solid investigation in the existing literature. Therefore, this paper has made tangible contributions and significant advances in the joint understanding of reliability, power, performance, and their interplay in IoT devices and servers quantitatively and mathematically.

Stochastic modeling of stationary scalar Gaussian processes in continuous time from autocorrelation data

We consider the problem of constructing a vector-valued linear Markov process in continuous time, such that its first coordinate is in good agreement with given samples of the scalar autocorrelation function of an otherwise unknown stationary Gaussian process. This problem has intimate connections to the computation of a passive reduced model of a deterministic time-invariant linear system from given output data in the time domain. We construct the stochastic model in two steps. First, we employ the AAA algorithm to determine a rational function which interpolates the z-transform of the discrete data on the unit circle and use this function to assign the poles of the transfer function of the reduced model. Second, we choose the associated residues as the minimizers of a linear inequality constrained least squares problem which ensures the positivity of the transfer function’s real part for large frequencies. We apply this method to compute extended Markov models for stochastic processes obtained from generalized Langevin dynamics in statistical physics. Numerical examples demonstrate that the algorithm succeeds in determining passive reduced models and that the associated Markov processes provide an excellent match of the given data.

Kinetic Network in Milestoning: Clustering, Reduction, and Transition Path Analysis.

We present a reduction of the Milestoning (ReM) algorithm to analyze the high-dimensional Milestoning kinetic network. The algorithm reduces the Milestoning network to low dimensions but preserves essential kinetic information, such as local residence time, exit time, and mean first passage time between any two states. This is achieved in three steps. First, nodes (milestones) in the high-dimensional Milestoning network are grouped into clusters based on the metastability identified by an auxiliary continuous-time Markov chain. Our clustering method is applicable not only to time-reversible networks but also to nonreversible networks generated from practical simulations with statistical fluctuations. Second, a reduced network is established via network transformation, containing only the core sets of clusters as nodes. Finally, transition pathways are analyzed in the reduced network based on the transition path theory. The algorithm is illustrated using a toy model and a solvated alanine dipeptide in two and four dihedral angles.

Abstract PR013: Detecting pairwise and higher-order antagonistic epistatic effects among somatic cancer genotypes to discover synthetic lethality

Abstract The mutual exclusivity of somatic variants in cancer suggests that current somatic variants can exert antagonistic epistatic effects on the selection of new mutations, contributing to a complex landscape of evolutionary trajectories. However, quantifying pairwise and higher-order epistatic effects—which are essential to estimation of the trajectory of likely cancer genotypes—has been a challenge. We have developed a continuous-time Markov chain model that enables the estimation of mutation origination and fixation, dependent on somatic cancer genotype. We coupled the continuous-time Markov chain model with an approach that deconvolutes the underlying mutation rates and selection intensities across trajectories of oncogenesis. By elucidating the routes of variant evolution and the strengths of selective forces at play, our approach lays the groundwork for quantifying antagonistic epistatic effects of specific somatic genotypes. We applied our approach to report the variant mutation rates, selection intensities, and consequent substitution rates for lung adenocarcinoma, comparing smoker and nonsmoker tumor cohorts. We describe the distinct influences of smoking on underlying mutation rates and on physiology affecting the selective regime of lung tissue. Inference of the most likely routes of site-specific variant evolution, as well as estimation of the selection strength operating on each step along the route represents a key component for prioritization, development, and implementation of personalized cancer therapies that target synthetic lethality. Citation Format: Jorge A. Alfaro-Murillo, Krishna Dasari, Jeffrey P. Townsend. Detecting pairwise and higher-order antagonistic epistatic effects among somatic cancer genotypes to discover synthetic lethality [abstract]. In: Proceedings of the AACR Special Conference in Cancer Research: Expanding and Translating Cancer Synthetic Vulnerabilities; 2024 Jun 10-13; Montreal, Quebec, Canada. Philadelphia (PA): AACR; Mol Cancer Ther 2024;23(6 Suppl):Abstract nr PR013.

Deterministic and stochastic analysis of a coral reef ecosystem with grazed macroalgae

Based on the work of Mumby et al. [Thresholds and the resilience of Caribbean coral reefs, Nature 450 (2007) 98–101], this study is devoted to investigate the deterministic and stochastic features of a coral reef ecosystem in which macroalgae and coral compete to occupy algae turfs, and the macroalgae are grazed by parrotfish. By taking the grazing rate as the focused parameter, we completely analyze the global dynamics of the deterministic coral reef ecosystem, including bistable phenomenon, linear equilibria and degenerate attractor. It is found that for different grazing rates, the coral reef system may exhibit rich dynamics which are closely dependent on the initial values. Furthermore, we derive a stochastic coral reef model in the form of continuous-time Markov chain (CTMC), and estimate the extinction probabilities of macroalgae and coral by using the branching process theory. Analytical results reveal that the macroalgae or coral species will go to extinction in a positive probability, even if it can survive in the deterministic environment.

Efficient valuation of guaranteed minimum accumulation benefits in regime switching jump diffusion models with lapse risk

We present a streamlined valuation method for the guaranteed minimum accumulation benefits incorporated within variable annuity contracts. At each contract renewal date, the insurer updates the policyholder's account value to the higher of the guaranteed value and the equity-linked investment value. In addition, we introduce lapse risks into the variable annuity contract, modeling the lapse decision under the assumption of stochastic intensity. Utilizing a combination of continuous-time Markov chain approximation and the Fourier cosine series expansion method, we derive closed-form valuation formulas under regime switching jump diffusion models. Numerical simulations showcase the precision and effectiveness of the proposed approach.

The Arsenal of Perturbation Bounds for Finite Continuous-Time Markov Chains: A Perspective

Perturbation bounds are powerful tools for investigating the phenomenon of insensitivity to perturbations, also referred to as stability, for stochastic and deterministic systems. This perspective article presents a focused account of some of the main concepts and results in inequality-based perturbation theory for finite state-space, time-homogeneous, continuous-time Markov chains. The diversity of perturbation bounds and the logical relationships between them highlight the essential stability properties and factors for this class of stochastic processes. We discuss the linear time dependence of general perturbation bounds for Markov chains, as well as time-independent (i.e., time-uniform) perturbation bounds for chains whose stationary distribution is unique. Moreover, we prove some new results characterizing the absolute and relative tightness of time-uniform perturbation bounds. Specifically, we show that, in some of them, an equality is achieved. Furthermore, we analytically compare two types of time-uniform bounds known from the literature. Possibilities for generalizing Markov-chain stability results, as well as connections with stability analysis for other systems and processes, are also discussed.

Perturbation analysis for continuous-time Markov chains in a weak sense

AbstractBy the technique of augmented truncations, we obtain the perturbation bounds on the distance of the finite-time state distributions of two continuous-time Markov chains (CTMCs) in a type of weaker norm than the V-norm. We derive the estimates for strongly and exponentially ergodic CTMCs. In particular, we apply these results to get the bounds for CTMCs satisfying Doeblin or stochastically monotone conditions. Some examples are presented to illustrate the limitation of the V-norm in perturbation analysis and to show the quality of the weak norm.

The Determining Role of Covariances in Large Networks of Stochastic Neurons.

Biological neural networks are notoriously hard to model due to their stochastic behavior and high dimensionality. We tackle this problem by constructing a dynamical model of both the expectations and covariances of the fractions of active and refractory neurons in the network's populations. We do so by describing the evolution of the states of individual neurons with a continuous-time Markov chain, from which we formally derive a low-dimensional dynamical system. This is done by solving a moment closure problem in a way that is compatible with the nonlinearity and boundedness of the activation function. Our dynamical system captures the behavior of the high-dimensional stochastic model even in cases where the mean-field approximation fails to do so. Taking into account the second-order moments modifies the solutions that would be obtained with the mean-field approximation and can lead to the appearance or disappearance of fixed points and limit cycles. We moreover perform numerical experiments where the mean-field approximation leads to periodically oscillating solutions, while the solutions of the second-order model can be interpreted as an average taken over many realizations of the stochastic model. Altogether, our results highlight the importance of including higher moments when studying stochastic networks and deepen our understanding of correlated neuronal activity.

Dynamic analysis of HIV infection model with CTL immune response and cell-to-cell transmission

HIV infection is still a serious worldwide public health problem. During the early stage of HIV infection, the number of infected cells and viruses are extremely low and the process of infection is stochastic. In this paper, we use a stochastic model consisting of continuous-time Markov chain to investigate CTL immune response and two modes of infection (viral transmission and cellular transmission). A multitype branching process is used to approximate the probability of clearance and outbreak of HIV infection near the infection-free equilibrium. In addition, we compare the dynamics of the deterministic model with the results of the stochastic model, and the differences are as follows: (1) when R0>1, the deterministic model shows HIV infection outbreak, while the stochastic model indicates a positive probability of HIV infection elimination; (2) the initial number of infected cells and viruses have no effect on the dynamics of the deterministic model, whereas the dynamics of the stochastic model are highly dependent on the initial sizes and cannot be ignored.

Random-Effects Substitution Models for Phylogenetics via Scalable Gradient Approximations.

Phylogenetic and discrete-trait evolutionary inference depend heavily on an appropriate characterization of the underlying character substitution process. In this paper, we present random-effects substitution models that extend common continuous-time Markov chain models into a richer class of processes capable of capturing a wider variety of substitution dynamics. As these random-effects substitution models often require many more parameters than their usual counterparts, inference can be both statistically and computationally challenging. Thus, we also propose an efficient approach to compute an approximation to the gradient of the data likelihood with respect to all unknown substitution model parameters. We demonstrate that this approximate gradient enables scaling of sampling-based inference, namely Bayesian inference via Hamiltonian Monte Carlo, under random-effects substitution models across large trees and state-spaces. Applied to a dataset of 583 SARS-CoV-2 sequences, an HKY model with random-effects shows strong signals of nonreversibility in the substitution process, and posterior predictive model checks clearly show that it is a more adequate model than a reversible model. When analyzing the pattern of phylogeographic spread of 1441 influenza A virus (H3N2) sequences between 14 regions, a random-effects phylogeographic substitution model infers that air travel volume adequately predicts almost all dispersal rates. A random-effects state-dependent substitution model reveals no evidence for an effect of arboreality on the swimming mode in the tree frog subfamily Hylinae. Simulations reveal that random-effects substitution models can accommodate both negligible and radical departures from the underlying base substitution model. We show that our gradient-based inference approach is over an order of magnitude more time efficient than conventional approaches.