Continuous-time Markov Chain Research Articles

Abstract 1251: Tobacco smoke alters the adaptive landscape of lung adenocarcinoma and influences the strength of epistatic interactions

Abstract Tobacco smoke produces both mutagenic and physiological effects, altering the mutation rate of cells and the selection acting on them in the damaged lung environment, and facilitating its role in driving cancer progression. In addition to these gene-by-environment interactions, gene-by-gene epistatic interactions also shape the adaptive landscape, influencing the evolutionary trajectories of cancer. Understanding the influence of tobacco smoke on the epistatic trajectories of lung cancer progression would help guide the development of targeted therapies that are more effective for smoker or never-smoker populations. To address this need, we constructed a continuous-time Markov chain model for the evolutionary trajectories of lung adenocarcinoma (LUAD), the most common subtype of lung cancer and the most frequent subtype among never-smokers. Using thousands of tumor sample sequences aggregated across studies, we profiled the trajectories of LUAD in smokers and never-smokers and estimated the rates at which mutations are gained from each possible set of pre-existing mutations, or genetic states. We then accounted for differences in baseline mutation rates to quantify the selective benefit of mutations. We find that epistasis is prevalent in LUAD, with several strong synergistic interactions—such as between RB1 and EGFR mutations- and antagonistic interactions-such as between EGFR and KEAP1 or KRAS mutations. Additionally, we identify non-additive epistatic effects: STK11 mutations are synergistic with KEAP1 and KRAS mutations but do not experience additional selection when both are mutated. In contrast, GNAS mutations are not synergistic with KEAP1 or RBM10 mutations alone but are synergistic with their co-mutation. Smoking has a large influence on the adaptive landscape, with several common mutations being preferentially selected in never-smoker cancers, including EGFR, SMAD4, GNAS, and PIK3CA mutations. KEAP1, STK11, and KRAS mutations are preferentially selected in smoker cancers. Smoking’s physiological influence also affects the strength of certain epistatic interactions, such as between STK11 and KEAP1 mutations. Overall, we find that smoking not only increases the mutation rate of lung cells but also substantially alters the adaptive landscape of lung adenocarcinoma, leading to clinically relevant differences in selection on mutations between smoker and never-smoker cancers. We additionally detect frequent pairwise and higher-order epistatic effects in LUAD that may inform the personalized application of targeted therapies. Citation Format: Krishna Dasari, Jorge Alfaro-Murillo, Jeffrey Townsend. Tobacco smoke alters the adaptive landscape of lung adenocarcinoma and influences the strength of epistatic interactions [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 1251.

Methods and conversations in (post)modern thermodynamics

Lecture notes after the doctoral school (Post)Modern Thermodynamics held at the University of Luxembourg, December 2022, 5-7, covering and advancing continuous-time Markov chains, network theory, stochastic thermodynamics, large deviations, deterministic and stochastic chemical reaction networks, metastability, martingales, quantum thermodynamics, and foundational issues.

Queuing-Inventory System with Catastrophes in the Warehouse: Case of Rare Catastrophes

A model of a single-server queuing-inventory system (QIS) with a limited waiting buffer for consumer customers (c-customers) and catastrophes has been developed. When a catastrophe occurs, all items in the system’s warehouse are destroyed, but c-customers in the system are still waiting for replenishment. In addition to c-customers, negative customers (n-customers) are also taken into account, each of which displaces one c-customer (if any). The policy (s, S) is used to replenish stocks. If, when a customer enters, the system warehouse is empty, then, according to Bernoulli’s trials, this customer either leaves the system without goods or joins the buffer. The mathematical model of the investigated QIS is constructed in the form of a continuous-time Markov chain (CTMC). Both exact and approximate methods for calculating the steady-state probabilities of constructed CTMCs are proposed and closed-form expressions are obtained for calculating the performance measures. Numerical evaluations are presented, demonstrating the high accuracy of the developed approximate formulas, as well as the behavior of performance measures depending on the input parameters. In addition, an optimization problem is solved to obtain the optimal value of the reorder point to minimize the expected total cost.

Forward starting options pricing under a regime-switching jump-diffusion model with Wishart stochastic volatility and stochastic interest rate

Vanilla options become effective immediately after they are entered, while some exotic options will only come to effective some time after they are bought or sold. Forward starting options are one kind of such exotic options started actually at some pre-specified future date. This paper presents an extension of regime-switching jump diffusion model, in which the parameters are driven by a continuous time and stationary Markov chain on a finite state space, by introducing the Wishart process into the instantaneous variance-covariance matrix of the risky asset price and stochastic interest rate. We derive the discounted conditional joint characteristic function and the forward characteristic function of the log-asset price and its the instantaneous variance-covariance process, and thereby the price of forward starting options are well evaluated by the probabilistic approach combined with the Fourier-cosine (COS) method. We also provide efficient Monte Carlo simulation of this proposed model, and simulated solutions to forward starting options pricing within a two-state regime switching framework. Numerical results show that the COS method is accurate and efficient for pricing forward starting options. Finally, we analyse impacts of some main parameters (especially, parameters in the Wishart stochastic volatility) in this proposed model on option prices and Δ values. Also, we consider the forward implied volatility. Furthermore, the forward starting options under the regime-switching jump diffusion model with Wishart stochastic volatility and stochastic interest rate which we derived are more generalized than those recently appeared in the derivatives pricing literature, and thus have wider application.

Dynamical Analysis of an Improved Bidirectional Immunization SIR Model in Complex Network.

In order to investigate the impact of two immunization strategies-vaccination targeting susceptible individuals to reduce their infection rate and clinical medical interventions targeting infected individuals to enhance their recovery rate-on the spread of infectious diseases in complex networks, this study proposes a bilinear SIR infectious disease model that considers bidirectional immunization. By analyzing the conditions for the existence of endemic equilibrium points, we derive the basic reproduction numbers and outbreak thresholds for both homogeneous and heterogeneous networks. The epidemic model is then reconstructed and extensively analyzed using continuous-time Markov chain (CTMC) methods. This analysis includes the investigation of transition probabilities, transition rate matrices, steady-state distributions, and the transition probability matrix based on the embedded chain. In numerical simulations, a notable concordance exists between the outcomes of CTMC and mean-field (MF) simulations, thereby substantiating the efficacy of the CTMC model. Moreover, the CTMC-based model adeptly captures the inherent stochastic fluctuation in the disease transmission, which is consistent with the mathematical properties of Markov chains. We further analyze the relationship between the system's steady-state infection density and the immunization rate through MCS. The results suggest that the infection density decreases with an increase in the immunization rate among susceptible individuals. The current research results will enhance our understanding of infectious disease transmission patterns in real-world scenarios, providing valuable theoretical insights for the development of epidemic prevention and control strategies.

Optimal production management when there is regime switching and production constraints

AbstractWe study a production problem in which the cumulative consumer demand for an item follows a Brownian motion with drift, with both the drift and the variance parameters modulated by a continuous-time Markov chain that represents the regime. The company wants to maintain the inventory level as close as possible to a target inventory level, but there is a linear cost of production. We assume that the production rate is nonnegative. The company is penalized for deviations from the inventory target level and the cost of production, and the objective is to minimize the total discounted penalty costs. We consider two models. In the first model, there is no upper bound for the production rate, and in the second model there is an upper bound for the production rate. We solve both problems analytically and obtain the optimal production policy and the minimal total expected discounted cost. Our solutions allow us to obtain interesting managerial insights.

SIS Epidemic Spreading Under Multilayer Population Dispersal in Patchy Environments

We study SIS epidemic spreading models under population dispersal on multi-layer networks. We consider a patchy environment in which each patch comprises individuals belonging to different classes. Individuals disperse to other patches on a multi-layer network in which each layer corresponds to a class. The dispersal on each layer is modeled by a Continuous Time Markov Chain (CTMC). At each time, individuals disperse according to their CTMC and subsequently interact with the local individuals in the patch according to an SIS model. We establish the existence of various equilibria under different parameter regimes and establish their (almost) global asymptotic stability using Lyapunov techniques. We also derive simple conditions that highlight the influence of the multi-layer network on the stability of these equilibria. For this model, we study optimal intervention strategies using a convex optimization framework. Finally, we numerically illustrate the influence of the multi-layer network structure and the effectiveness of the optimal intervention strategies.

A Compositional Semantics of Boolean-Logic Driven Markov Processes

Boolean-logic driven Markov processes (BDMPs) is a prominent dynamic extension of static fault trees to model repairable and complex dynamic systems. While BDMPs are intensively used in an industrial context for dependability analysis of energy systems, its formal semantics has not been systematically treated. To date, BDMPs are defined as a library of the domain-specific dependability-modelling language Figaro, a library that is neither open source nor publicly available. A rigorous semantic underpinning of BDMPs is indispensable for (1) developing BDMP analysis tools and (2) comparing its expressive power to other related reliability modelling languages. This paper presents a formal semantics to BDMPs using Markov automata (MA), an extension of continuous-time Markov chains (CTMCs) with action transitions that can be used to compose complex MA from smaller MA. This enables us to provide a compositional semantics. That is, we express the semantics of each individual BDMP element as an MA and obtain the MA for the entire BDMP by combining the MA of its elements. This makes the semantics comprehensible, for those who are familiar with automata theory, and easily extensible with new BDMP elements, e.g., to model security aspects. After the entire BDMP is considered, the actions in its MA that were used to “glue” the MA of BDMP elements, are ignored. This results in a CTMC that is amenable to exact numerical analysis by, e.g., efficient probabilistic model-checking techniques. We report on a prototypical implementation of our semantics and empirically show that our semantics yields dependability metrics that correspond to the interpretation by the Figaro knowledge base of BDMPs.

The effect of demographic stochasticity on Zika virus transmission dynamics: Probability of disease extinction, sensitivity analysis, and mean first passage time.

Viral infections spread by mosquitoes are a growing threat to human health and welfare. Zika virus (ZIKV) is one of them and has become a global worry, particularly for women who are pregnant. To study ZIKV dynamics in the presence of demographic stochasticity, we consider an established ZIKV transmission model that takes into consideration the disease transmission from human to mosquito, mosquito to human, and human to human. In this study, we look at the local stability of the disease-free and endemic equilibriums. By conducting the sensitivity analysis both locally and globally, we assess the effect of the model parameters on the model outcomes. In this work, we use the continuous-time Markov chain (CTMC) process to develop and analyze a stochastic model. The main distinction between deterministic and stochastic models is that, in the absence of any preventive measures such as avoiding travel to infected areas, being careful from mosquito bites, taking precautions to reduce the risk of sexual transmission, and seeking medical care for any acute illness with a rash or fever, the stochastic model shows the possibility of disease extinction in a finite amount of time, unlike the deterministic model shows disease persistence. We found that the numerically estimated disease extinction probability agrees well with the analytical probability obtained from the Galton-Watson branching process approximation. We have discovered that the disease extinction probability is high if the disease emerges from infected mosquitoes rather than infected humans. In the context of the stochastic model, we derive the implicit equation of the mean first passage time, which computes the average amount of time needed for a system to undergo its first state transition.

Probability Density Estimation for Composite Systems Reliability Indices Using Cross-Entropy Based Continuous Time Markov Chain Simulation

Probability Density Estimation for Composite Systems Reliability Indices Using Cross-Entropy Based Continuous Time Markov Chain Simulation

Estimation–Calibration of Continuous-Time Non-Homogeneous Markov Chains with Finite State Space

We propose a method for fitting transition intensities to a sufficiently large set of trajectories of a continuous-time non-homogeneous Markov chain with a finite state space. Starting with simulated data computed with Gompertz–Makeham transition intensities, we apply the proposed method to fit piecewise linear intensities and then compare the transition probabilities corresponding to both the Gompertz–Makeham transition intensities and the fitted piecewise linear intensities; the main comparison result is that the order of magnitude of the average fitting error per unit time—chosen as a year—is always less than 1%, thus validating the methodology proposed.

EAPRAD: A MAC protocol for Enhancing Access Probability and Reducing Access Delay in VANETs

Vehicular Ad Hoc Networks (VANETs) present challenges in terms of access collision and merge collision, leading to increased access delay in end-to-end vehicle communication. How to address these issues remains an unsolved problem. In this paper, we propose EAPRAD (Enhancing Access Probability and Reducing Access Delay), a novel MAC (Media Access Control) protocol based on a Markov model. Our protocol defines the number of vehicle accessing time slot failures, with different failure states forming a partial Markov chain. The highest failure status vehicle is prioritized to access the time slot, effectively resolving the problem of indefinite time slot reservations during the back-off phase. Furthermore, after acquiring the time slot, vehicles may still experience package collision, particularly when vehicles traveling in opposite directions share the same frequency band. To address this, our protocol introduces a new MAC frame structure that partitions available frequency band resources based on distinct driving directions, eliminating merge collision in opposite directions. For vehicles traveling in the same direction, intermediate nodes are utilized to proactively anticipate and prevent package collision. To model the state transition of access collision probability and merge, we develop a Continuous Time Markov Chain (CTMC) analytical model. Simulation results demonstrate that EAPRAD reduces end-to-end delay and beacon lost rate (BLR), thereby improving overall network performance in VANETs.

Fluctuation Theory of Continuous-Time, Skip-Free Downward Markov Chains with Applications to Branching Processes with Immigration

We develop a comprehensive methodology for the fluctuation theory of continuous-time, skip-free Markov chains, extending and improving the recent work of Choi and Patie for discrete-time, skip-free Markov chains. As a significant application, we use it to derive a full set of fluctuation identities regarding exiting a finite or infinite interval for Markov branching processes with immigration, thereby uncovering many new results for this classic family of continuous-time Markov chains. The theory also allows us to recover in a simple manner fluctuation identities for skip-free downward compound Poisson processes.

Optimal Control of Probability on a Target Set for Continuous-Time Markov Chains

In this paper, a stochastic optimal control problem is considered for a continuous-time Markov chain taking values in a denumerable state space over a fixed finite horizon. The optimality criterion is the probability that the process remains in a target set before and at a certain time. The optimal value is a superadditive capacity of target sets. Under some minor assumptions for the controlled Markov process, we establish the dynamic programming principle, based on which we prove that the value function is a classical solution of the Hamilton-Jacobi-Bellman equation on a discrete lattice space. We then prove that there exists an optimal deterministic Markov control under the compactness assumption of control domain. We further prove that the value function is the unique solution of the HJB equation. We also consider the case starting from the outside of target set and give the corresponding results. Finally, we apply our results to two examples.

Abstract IA014: Detecting epistatic effects among somatic cancer mutations across oncogenesis

Abstract The somatic evolution of cancer depends on the rate at which new mutations are acquired, as well as the selective pressure acting on them. Further, mutual exclusivity or co-occurrence of somatic variants suggests that current somatic variants can exert antagonistic or synergistic epistatic effects on the selection of new mutations, contributing to a complex landscape of evolutionary trajectories. However, quantifying these epistatic effects—which are essential to estimation of the trajectory of likely cancer genotoypes—has been a challenge, especially for higher-order epistatic effects. To estimate these epistatic effects, we have developed a continuous-time Markov chain model that enables the estimation of mutation origination and fixation, dependent on somatic cancer genotype. Coupling the continuous-time Markov chain model with a mutation-rate deconvolution approach, we have estimated the underlying mutation rates and selection intensities across trajectories of oncogenesis. Specifically, we report the mutation rates, selection intensities, and consequent variant fluxes for lung adenocarcinoma, informed by a compilation of samples from multiple data sets including The Cancer Genome Atlas and the AACR Project GENIE. Furthermore, we compare smoker and nonsmoker tumor cohorts describing the distinct evolutionary trajectories jointly, influenced by differential mutation and also by the differential physiological effects that smoking can have on the selective regime of lung tissue. Our approach enables inference of the most likely routes of site-specific variant evolution, as well as estimation of the selection strength operating on each step along the route—a key component of what we need to know to prioritize, develop, and implement personalized cancer therapies. Citation Format: Jorge A. Alfaro-Murillo, Krishna Dasari, Jeffrey P. Townsend. Detecting epistatic effects among somatic cancer mutations across oncogenesis [abstract]. In: Proceedings of the AACR Special Conference in Cancer Research: Translating Cancer Evolution and Data Science: The Next Frontier; 2023 Dec 3-6; Boston, Massachusetts. Philadelphia (PA): AACR; Cancer Res 2024;84(3 Suppl_2):Abstract nr IA014.

Lyme Disease Models of Tick-Mouse Dynamics with Seasonal Variation in Births, Deaths, and Tick Feeding.

Lyme disease is the most common vector-borne disease in the United States impacting the Northeast and Midwest at the highest rates. Recently, it has become established in southeastern and south-central regions of Canada. In these regions, Lyme disease is caused by Borrelia burgdorferi, which is transmitted to humans by an infected Ixodes scapularis tick. Understanding the parasite-host interaction is critical as the white-footed mouse is one of the most competent reservoir for B. burgdorferi. The cycle of infection is driven by tick larvae feeding on infected mice that molt into infected nymphs and then transmit the disease to another susceptible host such as mice or humans. Lyme disease in humans is generally caused by the bite of an infected nymph. The main aim of this investigation is to study how diapause delays and demographic and seasonal variability in tick births, deaths, and feedings impact the infection dynamics of the tick-mouse cycle. We model tick-mouse dynamics with fixed diapause delays and more realistic Erlang distributed delays through delay and ordinary differential equations (ODEs). To account for demographic and seasonal variability, the ODEs are generalized to a continuous-time Markov chain (CTMC). The basic reproduction number and parameter sensitivity analysis are computed for the ODEs. The CTMC is used to investigate the probability of Lyme disease emergence when ticks and mice are introduced, a few of which are infected. The probability of disease emergence is highly dependent on the time and the infected species introduced. Infected mice introduced during the summer season result in the highest probability of disease emergence.

Differentiated uniformization: a new method for inferring Markov chains on combinatorial state spaces including stochastic epidemic models

AbstractWe consider continuous-time Markov chains that describe the stochastic evolution of a dynamical system by a transition-rate matrix Q which depends on a parameter $$\theta $$ θ . Computing the probability distribution over states at time t requires the matrix exponential $$\exp \,\left( tQ\right) \,$$ exp t Q , and inferring $$\theta $$ θ from data requires its derivative $$\partial \exp \,\left( tQ\right) \,/\partial \theta $$ ∂ exp t Q / ∂ θ . Both are challenging to compute when the state space and hence the size of Q is huge. This can happen when the state space consists of all combinations of the values of several interacting discrete variables. Often it is even impossible to store Q. However, when Q can be written as a sum of tensor products, computing $$\exp \,\left( tQ\right) \,$$ exp t Q becomes feasible by the uniformization method, which does not require explicit storage of Q. Here we provide an analogous algorithm for computing $$\partial \exp \,\left( tQ\right) \,/\partial \theta $$ ∂ exp t Q / ∂ θ , the differentiated uniformization method. We demonstrate our algorithm for the stochastic SIR model of epidemic spread, for which we show that Q can be written as a sum of tensor products. We estimate monthly infection and recovery rates during the first wave of the COVID-19 pandemic in Austria and quantify their uncertainty in a full Bayesian analysis. Implementation and data are available at https://github.com/spang-lab/TenSIR.

Model Simulation of Continuous Time Markov Chain Susceptible Infected Recovered-Bacterial Population for Cholera Disease

<p>Epidemic is an outbreak of an infectious disease rapidly in a population at a certain place and time. Epidemic models are used to explains the spread pattern of disease. The continuous time Markov chain susceptible infected recovered-bacterial population in the aquatic reservoir (CTMC SIR-B) model is a stochastic model, which considers the effect of bacterial population. The human population are classified into 3 groups. There are susceptible, infected, and recovered groups. Then, there are bacterial population which can infectious the cholera disease to human. CTMC SIR-B model considers treatment and water sanitation parameters. The spread of cholera disease can be modeled as CTMC SIR-B. Cholera is an acute intestinal infectious disease caused by the bacterium Vibrio cholerae. Cholera can be transmitted through the human digestive system. The symptoms of cholera disease are diarrhea, vomiting, and dehydration. The dehydration if not handled properly, may cause death. The aims of this research are to build and simulate the CTMC SIR-B model for cholera disease. The result of the model simulation shows that there is no significant difference between various values of treatment and water sanitation parameters. The pattern of the cholera disease spread describes that the transmission of cholera can occur from human to human even though there is no population of bacteria in the aquatic reservoir.</p><p><em></em><strong><em>Keywords</em></strong><strong><em>: </em></strong><em>cholera</em><em>; ctmc sir-b;</em><em> epidemic model</em><em>;</em><em> stochastic.</em></p><p> </p>

Multi-state SVIRD Model with Continuous-time Markov Chain Assumption on the Spread of Infectious Diseases

The spread of infectious diseases is generally described using mathematical models. This paper discusses the spread of infectious diseases using a multi-state SVIRD model, assuming that a continuous-time Markov chain (CTMC) occurs in a closed population and is examined regularly. This article aims to generate transition probabilities and parameter estimates using the maximum likelihood method. The multi-state SVIRD model assuming CTMC uses a transition intensity and transition probability approach consisting of five primary states: susceptible, vaccinated, infected, recovered, and deceased. The infected state is divided into two: infected before and after being vaccinated. The result is an estimator of transition intensity with sojourn time which is exponentially distributed to produce a transition probability matrix. Then the algorithm for the CTMC SVIRD model is given. The multi-state SVIRD model algorithm can be used directly if the epidemic case is still in single-wave to determine the transition probability. In contrast, for multi-wave cases, it is necessary to detect changepoints to determine wave boundaries to make predictions more accurate. The main contributions of this study are using the CTMC assumption, a stochastic model for determining the parameters of the differential equation formed by the compartment model and adding vaccinated status to the model. In addition, it also provides ways to overcome multi-wave epidemic cases so the prediction results are more accurate.

On the surprising effectiveness of a simple matrix exponential derivative approximation, with application to global SARS-CoV-2

The continuous-time Markov chain (CTMC) is the mathematical workhorse of evolutionary biology. Learning CTMC model parameters using modern, gradient-based methods requires the derivative of the matrix exponential evaluated at the CTMC's infinitesimal generator (rate) matrix. Motivated by the derivative's extreme computational complexity as a function of state space cardinality, recent work demonstrates the surprising effectiveness of a naive, first-order approximation for a host of problems in computational biology. In response to this empirical success, we obtain rigorous deterministic and probabilistic bounds for the error accrued by the naive approximation and establish a "blessing of dimensionality" result that is universal for a large class of rate matrices with random entries. Finally, we apply the first-order approximation within surrogate-trajectory Hamiltonian Monte Carlo for the analysis of the early spread of Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) across 44 geographic regions that comprise a state space of unprecedented dimensionality for unstructured (flexible) CTMC models within evolutionary biology.