Continuous-time Markov Chain Research Articles

Inference on autoregulation in gene expression with variance-to-mean ratio

Some genes can promote or repress their own expressions, which is called autoregulation. Although gene regulation is a central topic in biology, autoregulation is much less studied. In general, it is extremely difficult to determine the existence of autoregulation with direct biochemical approaches. Nevertheless, some papers have observed that certain types of autoregulations are linked to noise levels in gene expression. We generalize these results by two propositions on discrete-state continuous-time Markov chains. These two propositions form a simple but robust method to infer the existence of autoregulation from gene expression data. This method only needs to compare the mean and variance of the gene expression level. Compared to other methods for inferring autoregulation, our method only requires non-interventional one-time data, and does not need to estimate parameters. Besides, our method has few restrictions on the model. We apply this method to four groups of experimental data and find some genes that might have autoregulation. Some inferred autoregulations have been verified by experiments or other theoretical works.

A Markovian model of asynchronous multi-stage manufacturing lines fabricating discrete parts

Asynchronous serial manufacturing lines that fabricate discrete parts are traditionally used in mass production, which represents a key sector in the global economy. Recent technological solutions for the modularization and standardization of manufacturing stations have led to this type of manufacturing system being reconfigured more often than in the past. Therefore, synthetic but accurate performance-evaluation models have become relevant as kernels in decision supporting tools for the continuous improvement of manufacturing systems. This paper presents a novel analytical model for the performance evaluation of asynchronous unreliable manufacturing lines fabricating discrete parts with finite buffers and deterministic processing times. This approach is based on continuous-time continuous-flow Markov chains. The general concept of operational cycles in discrete production is integrated into the modeling. The proposed model was validated using a discrete event simulation. The results demonstrate the accuracy and robustness of this model in evaluating a wide set of performance measures. The advantages of using this approach with respect to a purely continuous model were demonstrated. The applicability of the model to actual industrial scenarios was also demonstrated in a use case involving a high-volume assembly line.

Bridging Causal Reversibility and Time Reversibility: A Stochastic Process Algebraic Approach

Causal reversibility blends reversibility and causality for concurrent systems. It indicates that an action can be undone provided that all of its consequences have been undone already, thus making it possible to bring the system back to a past consistent state. Time reversibility is instead considered in the field of stochastic processes, mostly for efficient analysis purposes. A performance model based on a continuous-time Markov chain is time reversible if its stochastic behavior remains the same when the direction of time is reversed. We bridge these two theories of reversibility by showing the conditions under which causal reversibility and time reversibility are both ensured by construction. This is done in the setting of a stochastic process calculus, which is then equipped with a variant of stochastic bisimilarity accounting for both forward and backward directions.

On regime-switching European option pricing

The concern of this article is to derive a regime switching model that can be utilized to price European call options for a financial market that exhibits structural changes with time. The model is formulated based on the fact that the underlying asset process is described by a geometric Brownian motion that is modulated by a continuous-time Markov chain with two regimes. Moreover, by an application of the change of measure technique, an option price is derived under the risk neutral valuation and the model parameter estimates is performed by use of the maximum likelihood estimation. The model implementation is carried out by utilizing the Russell 2000 and Facebook in dices data sets. The model results are compared with that of the Black-Scholes model in order to establish the model with better results in terms of predicting the European call option prices. In general, the data sets have common characteristics of financial time series across the regimes and the volatility process spends longer time in regime 2 than it stays in regime 1. The predicted call option prices from both models are more or less similar across the market indices; however, the results of the Black-Scholes model are a bit closer to the market prices than that of the regime-switching model across the two markets. Therefore, the Black-Scholes model slightly gives better results for the Russell 2000 and Facebook indices data sets as compared with the RS model.

Accessibility Study of mHealth Systems Based on The Internet of Things (IoT)

The Internet of Things (IoT) has become part of people's daily life, allowing physical and digital contact. The rise of mobile devices and scientific and technological advances in health have led to breakthroughs in meeting consumer needs. mHealth refers to using mobile devices to improve healthcare services, increase medical care, and reduce costs. Mobile cloud computing (MCC) lets users bypass mobile device limits on processing, storage, and battery life. A body area network uses implanted wireless sensors to remotely monitor patients (WBAN). These networks collect and distribute data for disease diagnosis and prevention. This mix of technologies allows hospitals and clinics new ways to treat and monitor patients. This research examines IoT availability in mHealth. The analysed architecture includes wireless sensors to monitor patients, an intra-BAN, a mobile device with a battery and communication interfaces, an extra-BAN with Wi-Fi and 4G connectivity, and a cloud environment to store data. Hierarchical models were developed using RBD and continuous-time Markov chains (CTMC). Each component's MTTF (mean time to failure) and MTTR (mean time to repair) values are used to quantify system or section availability. Experiments were conducted to test mHealth availability parameters. Intra-BAN, Zigbee, or Bluetooth protocols have no meaningful impact on system availability. For households, two small routers in the extra-BAN are more effective than one large router. Finally, backup batteries and power banks boost availability. The offered models can help developers and maintainers scale mHealth systems based on service needs.

Simulation modeling of glutamate cysteine ligase activity

L-Y-glutamyl-L-cysteinyl glycine, or glutathione, as one of the basic intracellular antioxidants, plays a vital role in cellular metabolism. In mammalian cells, glutathione is synthesized via two steps. The first step that is considered rate limiting is catalyzed by glutamate cysteine ligase. In this work, a stochastic algorithm based on continuous-time Markov chains was used to simulate the activity of glutamate-cysteine ligase. Several different mechanisms of enzymatic activity including reversible inhibition of glutathione, and an ATP binding motif have been considered. Based on physiological metabolite measurements made for human erythrocytes, the activity of glutamate cysteine ligase was determined. There are many possible ways for substrates to bind to an active site of the studied enzyme, but, only the mechanism by which primary binding to ATP can occur makes it possible to obtain the catalytic rate value similar to that of the experimentally measured glutamatecysteine ligase activity relative to physiological concentrations of substrates. In other cases, the values differ by more than one order of magnitude. The performed analysis allows the conclusion that when models for glutathione biosynthesis are constructed in vivo conditions, the ATP concentration and reversible inhibition of glutathione should be taken into account.

Taenia solium taeniasis and cysticercosis: extinction or outbreak

Taenia solium taeniasis and cysticercosis are neglected zoonotic diseases that affect human health and economies of developing countries. In this work, we formulate and analyze deterministic and continuous time Markov chain (CTMC) stochastic models to determine parameters that drive Taenia solium taeniasis and cysticercosis and the likelihood of their extinction. The basic reproduction number R_0 is computed by the next generation matrix approach, sensitivity index of each parameter in R_0 is derived by the normalized forward sensitivity index and the likelihood of diseases’ extinction is computed by the multitype branching process. The analysis shows that humans with Taenia solium taeniasis, infectious pork and Taenia solium eggs in the environment play an important role in the transmission of Taenia solium taeniasis and cysticercosis, and the model exhibits forward bifurcation at R_0=1. This implies that R_0<1 is a sufficient condition to eliminate Taenia solium taeniasis and cysticercosis. For CTMC model, analysis shows that the probability of Taenia solium taeniasis and cysticercosis extinction is high if the diseases emerge from humans with Taenia solium cysticercosis and there is an outbreak if the diseases emerge from either humans with Taenia solium taeniasis or infectious pork or Taenia solium eggs in the environment. To control Taenia solium taeniasis and cysticercosis, the intervention strategies should focus on improving hygiene and sanitation for reducing shedding rate of Taenia solium eggs in the environment, inspection of pork for reducing the rate of acquiring Taenia solium taeniasis and spraying of insecticides for killing Taenia solium eggs in the environment.

Abstract 862: Pairwise and higher-order epistatic effects among somatic cancer mutations across oncogenesis of lung adenocarcinoma

Abstract The somatic evolution of cancer depends on the rate at which new mutations are acquired, as well as the selective pressure acting on them. Further, mutual exclusivity or co-occurrence of mutations suggests that variants can exert antagonistic or synergistic epistatic effects on the selection of new mutations, contributing to a complex landscape of evolutionary trajectories. Estimation of pairwise and higher-order epistatic effects—essential to estimation of the trajectory of likely cancer genotoypes—has remained a challenge. We have developed a continuous-time Markov chain model that enables the estimation of mutation origination and fixation (flux), dependent on somatic cancer genotype. Coupling the continuous-time Markov chain model with a deconvolution approach provides estimates of underlying mutation rates and selection across the trajectory of oncogenesis. We demonstrate computation of fluxes and selection coefficients in a somatic evolutionary model of lung adenocarcinoma informed by a compilation of samples from multiple data sets including The Cancer Genome Atlas and the AACR Project GENIE. The approach enables inference of the most likely routes of site-specific variant evolution and estimation of the selection strength operating on each step along the route, a key component of what we need to know to develop and implement personalized cancer therapies. Finally, by comparing smoker and nonsmoker tumor cohorts, we reveal and describe distinct evolutionary trajectories, influenced not only by differential mutation, but also by the differential physiological effects that smoking can have on the selective regime of lung tissue. These distinctions disambiguate the physiological and mutational influences of smoking and can inform personalized treatment of the two groups. Citation Format: Jorge A. Alfaro-Murillo, Krishna Dasari, Jeffrey P. Townsend. Pairwise and higher-order epistatic effects among somatic cancer mutations across oncogenesis of lung adenocarcinoma [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 862.

Application of normal cones to the computation of solutions of the nonlinear Kolmogorov backward equation

A numerical approach to computing solutions of a generalized Kolmogorov backward equation is proposed, in which the stochastic matrix is replaced by a nonlinear operator obtained as the lower bound of a set of stochastic matrices. The equation is central to the theory of imprecise Markov chains in continuous time, which has made rapid progress in recent years. One of the obstacles to its implementation remains the high computational complexity, with the prevailing existing approaches relying on a discretization of the time interval. In order to achieve sufficient accuracy of the approximations, the grid must typically contain a large number of points on which optimization steps are performed, usually using linear programming. The main goal of this work is to develop a new, more efficient approach by significantly reducing the number of optimization steps required to achieve the prescribed accuracy of the solutions. Our approach is based on the Lipschitz continuity of the solutions of the equation with respect to time, which results in the optimization problems occurring at nearby points of the time interval having similar optimal solutions. This property is exploited using the theory of normal cones of convex polytopes. If the solution vectors remain within the same normal cone of a polytope corresponding to the nonlinear operator in a given interval, the optimization problem to be solved becomes linear, which allows much faster computations. This paper is primarily concerned with providing the theoretical basis for the new technique. However, initial tests show that it significantly outperforms existing methods in most cases.

Comparison Theorems for Stochastic Chemical Reaction Networks

Continuous-time Markov chains are frequently used as stochastic models for chemical reaction networks, especially in the growing field of systems biology. A fundamental problem for these Stochastic Chemical Reaction Networks (SCRNs) is to understand the dependence of the stochastic behavior of these systems on the chemical reaction rate parameters. Towards solving this problem, in this paper we develop theoretical tools called comparison theorems that provide stochastic ordering results for SCRNs. These theorems give sufficient conditions for monotonic dependence on parameters in these network models, which allow us to obtain, under suitable conditions, information about transient and steady-state behavior. These theorems exploit structural properties of SCRNs, beyond those of general continuous-time Markov chains. Furthermore, we derive two theorems to compare stationary distributions and mean first passage times for SCRNs with different parameter values, or with the same parameters and different initial conditions. These tools are developed for SCRNs taking values in a generic (finite or countably infinite) state space and can also be applied for non-mass-action kinetics models. When propensity functions are bounded, our method of proof gives an explicit method for coupling two comparable SCRNs, which can be used to simultaneously simulate their sample paths in a comparable manner. We illustrate our results with applications to models of enzymatic kinetics and epigenetic regulation by chromatin modifications.

A combined strategy for dynamic probabilistic risk assessment of fission battery designs using EMRALD and DEPM

The notion of nuclear reactors with battery-like capabilities, called fission batteries, puts forth system requirements and design constraints that have so far been unseen in the nuclear power production industry. Such restrictions require fission batteries to be modular, integrated, autonomous, tamper-proof (i.e., resilient, fault-tolerant, all-weather, and safe), and affordable. With design requirements specifying no human intervention for operation, and minimal connectivity to remote monitoring networks, fission batteries are unique among existing nuclear power plants and emerging advanced reactor designs. Given these attributes, traditional probabilistic risk assessment (PRA) of fission batteries is expected to require dynamic methods to model advanced aspects, such as self-diagnosis, self-adjustment, and duration-prediction capabilities, as they are key ingredients for unattended operations. In addition, availability models need to integrate autonomous control, associated error-detection algorithms, and adversarial human actions. Currently, no existing framework demonstrably assesses these advanced attributes. This paper introduces and demonstrates an integrated framework for the dynamic modeling of fission battery designs. The proposed framework comprises a combined modeling strategy that uses the dual-graph error propagation methodology (DEPM) based on the continuous-time Markov chain (CTMC) models implemented in OpenPRA Error Propagation (OpenErrorPro) and the dynamic PRA tool, Event Modeling Risk Assessment using Linked Diagrams (EMRALD), based on discrete dynamic event trees (DDET). This combination overcomes some of the limitations of the tools when used independently. It enables detailed dynamic analysis to produce time explicit results to support the development of fission battery traditional PRA models. To evaluate the utility of this novel approach, a demonstration case is shown that models the hypothesized response of a fission battery design to an external fire event. DEPM CTMCs and alternative failure approaches are coupled with EMRALD to characterize and quantify the likelihood of the event sequences. The results show that the combined framework effectively captures the dynamic aspects of fission battery design in terms of the timing and realism of modeled events. Given the complexity of the failure scenarios, we believe that EMRALD and DEPM are necessary and complementary when the need for high-resolution analysis offsets the challenges of detailed modeling.

Behavioural equivalences for continuous-time Markov processes

AbstractBisimulation is a concept that captures behavioural equivalence of states in a variety of types of transition systems. It has been widely studied in a discrete-time setting. The core of this work is to generalise the discrete-time picture to continuous time by providing a notion of behavioural equivalence for continuous-time Markov processes. In Chen et al. [(2019). Electronic Notes in Theoretical Computer Science347 45–63.], we proposed two equivalent definitions of bisimulation for continuous-time stochastic processes where the evolution is a flow through time: the first one as an equivalence relation and the second one as a cospan of morphisms. In Chen et al. [(2020). Electronic Notes in Theoretical Computer Science.], we developed the theory further: we introduced different concepts that correspond to different behavioural equivalences and compared them to bisimulation. In particular, we studied the relation between bisimulation and symmetry groups of the dynamics. We also provided a game interpretation for two of the behavioural equivalences. The present work unifies the cited conference presentations and gives detailed proofs.

A Comprehensive Model of Android Software Aging and Rejuvenation Considering Battery Saving

The more complex the software system, the greater the number of possible combinations of defects that can cause errors, resulting in software defects that are difficult to isolate and expensive to correct in the development stage. An essential feature of such defects is a gradual deterioration in software performance finishing with software failure—software aging. Mobile devices are particularly vulnerable to software aging. Thus, there is a constant need for methods and tools to eliminate the effects of aging in mobile systems based on modeling the aging process, including the study of metrics and aging factors and the development of more reliable and adequate aging and rejuvenation models. This paper summarizes the previously developed Android software aging and rejuvenation models and presents a comprehensive model of aging and rejuvenation for the Android operating system. The comprehensive model is based on continuous-time Markov Chains and considers different aging levels, mobile device activity, and battery status. The aging and rejuvenation model can be used to assess the software quality, allows obtaining expressions for indicators of software rejuvenation efficiency, and can be used to design and select parameters of the software rejuvenation method considering battery saving.

Concentration Inequalities for Output Statistics of Quantum Markov Processes

We derive new concentration bounds for time averages of measurement outcomes in quantum Markov processes. This generalizes well-known bounds for classical Markov chains, which provide constraints on finite-time fluctuations of time-additive quantities around their averages. We employ spectral, perturbation and martingale techniques, together with non-commutative L_2 theory, to derive: (i) a Bernstein-type concentration bound for time averages of the measurement outcomes of a quantum Markov chain, (ii) a Hoeffding-type concentration bound for the same process, (iii) a generalization of the Bernstein-type concentration bound for counting processes of continuous-time quantum Markov processes, (iv) new concentration bounds for empirical fluxes of classical Markov chains which broaden the range of applicability of the corresponding classical bounds beyond empirical averages. We also suggest potential application of our results to parameter estimation and consider extensions to reducible quantum channels, multi-time statistics and time-dependent measurements, and comment on the connection to so-called thermodynamic uncertainty relations.

Reliability Enhancement of Fast Charging Station under Electric Vehicle Supply Equipment Failures and Repairs

This work focuses on the enhancement of the charging reliability of both scheduled (SEVs) and opportunistic (UEVs) electric vehicle (EV) users in an EV fast charging station (FCS). The proposed charging coordination strategies allow UEVs to exploit unused charging resources to optimally utilize the limited charging resources of FCS. However, the optimum utilization of limited charging resources of an FCS while assuring a reliable charging process for plugged-in EVs under random failures of electric vehicle supply equipment (EVSE) is a real challenge for the FCS controller. When the FCS admits UEVs in addition to SEVs, assuring a satisfactory quality of service to both EV user categories is also dispensable. Therefore, we analyze the performance of reservation of off-board mobile chargers (MOBCs) to enhance the charging reliability of EV users while achieving high charging resource utilization. This work proposes resource allocation and charging coordination strategies for an FCS where MOBCs are used to enhance the charging reliability of both SEVs and UEVs. Moreover, the proposed dynamic charging resource coordination strategies are analyzed with a continuous time Markov-chain (CTMC). The presented results from the CTMC model demonstrate that the proposed strategies outperform the EV charging process of the FCS in terms of high resource utilization and reliability while guaranteeing a satisfactory quality of service to EV users.

Synchronous speculative simulation of tightly coupled agents in continuous time on CPUs and GPUs

Traditionally, parallel discrete-event simulations of agent-based models in continuous time are organized around logical processes exchanging time-stamped events, which clashes with the properties of models in which tightly coupled agents frequently and instantaneously access each other’s states. To illustrate the challenges of such models and to derive a solution, we consider the domain-specific modeling language ML3, which allows modelers to succinctly express transitions and interactions of linked agents based on a continuous-time Markov chain (CTMC) semantics. We propose synchronous optimistic synchronization algorithms tailored toward simulations of fine-grained interactions among tightly coupled agents in highly dynamic topologies and present implementations targeting multicore central processing units (CPUs) as well as many-core graphics processing units (GPUs). By dynamically restricting the temporal progress per round to ensure that at most one transition or state access per agent, the synchronization algorithms enable efficient direct agent interaction and limit the required agent state history to only a single current and projected state. To maintain concurrency given actions that depend on dynamically updated macro-level properties, we introduce a simple relaxation scheme with guaranteed error bounds. Using an extended variant of the classical susceptible-infected-recovered network model, we benchmark and profile the performance of the different algorithms running on CPUs and on a data center GPU.

A general approximation method for optimal stopping and random delay

AbstractThis study examines the continuous‐time optimal stopping problem with an infinite horizon under Markov processes. Existing research focuses on finding explicit solutions under certain assumptions of the reward function or underlying process; however, these assumptions may either not be fulfilled or be difficult to validate in practice. We developed a continuous‐time Markov chain (CTMC) approximation method to find the optimal solution, which applies to general reward functions and underlying Markov processes. We demonstrated that our method can be used to solve the optimal stopping problem with a random delay, in which the delay could be either an independent random variable or a function of the underlying process. We established a theoretical upper bound for the approximation error to facilitate error control. Furthermore, we designed a two‐stage scheme to implement our method efficiently. The numerical results show that the proposed method is accurate and rapid under various model specifications.

Timed hazard networks: Incorporating temporal difference for oncogenetic analysis.

Oncogenetic graphical models are crucial for understanding cancer progression by analyzing the accumulation of genetic events. These models are used to identify statistical dependencies and temporal order of genetic events, which helps design targeted therapies. However, existing algorithms do not account for temporal differences between samples in oncogenetic analysis. This paper introduces Timed Hazard Networks (TimedHN), a new statistical model that uses temporal differences to improve accuracy and reliability. TimedHN models the accumulation process as a continuous-time Markov chain and includes an efficient gradient computation algorithm for optimization. Our simulation experiments demonstrate that TimedHN outperforms current state-of-the-art graph reconstruction methods. We also compare TimedHN with existing methods on a luminal breast cancer dataset, highlighting its potential utility. The Matlab implementation and data are available at https://github.com/puar-playground/TimedHN.

Utilizing Continuous Time Markov Chain for analyzing video-on-demand streaming in multimedia systems

In multimedia systems, Tree and Mesh-based topologies are the popular ways to organize the peers in an overlay and provide video streaming services. Unlike Content Delivery Networks (CDN), providing streaming services through a Peer-to-Peer (P2P) overlay poses the challenge of unpredictable churn for both provider and consumer peers. The dynamically arriving and departing peers, with Video Cassette Recorder (VCR) actions like jump, pause, forward, rewind, etc., controlled by end users, makes it challenging to reason about such systems. Though techniques like prototype test beds, simulation, etc., are successful to some extent in analyzing the behavior of such complex systems before the actual deployment, they either require lots of effort to set up or fail to capture the finer anomalies. Therefore, there is a need to have a formal framework that can specify the system at the desired abstraction level and capture the stochastic behavior exhibited by the actors of such systems. This paper addresses this issue and provides a platform for analyzing the behavior of real-time Video-on-Demand (VoD) streaming behavior with VCR actions over P2P overlays. To capture the stochastic behavior of actors, the Continuous Time Markov Chain (CTMC) is used to design a model of both the tree and mesh-based systems. This CTMC model is written in the Probabilistic Symbolic Model Checker (PRISM) language, which assists in containing the state space explosion and specifies the QoUX properties using Continuous Stochastic Logic (CSL). The proposed methodology uses the scenarios described in various simulation-based studies and industry standards of VoD systems available in the proposed formal model and finds a striking resemblance in the results. Eventually, given a joining-delay threshold of 170 ms, a P2P-Tree-based architecture resulted in an optimal tree height of 4-5 levels; after that, the startup delay of playback gets affected. Similarly, for a given startup delay threshold of 700 ms after departure and seek-jump actions, the maximum P2P-Tree height should be 5. We have also compared the performance properties of P2P tree and mesh-based VoD streaming systems with industry standards of centralized VoD systems. It is observed that the performance of P2P-tree and P2P-mesh-based VoD streaming systems also meet the industry standards. The results we got from the proposed formal framework are conclusive and establish the applicability of the approach.

Non-homogeneous continuous-time Markov chain withcovariates: Applications to ambulatory hypertension monitoring.

Hypertension significantly increases the risk for many health conditions including heart disease and stroke. Hypertensive patients often have continuous measurements of their blood pressure to better understand how it fluctuates over the day. The continuous-time Markov chain (CTMC) is commonly used to study repeated measurements with categorical outcomes. However, the standard CTMC may be restrictive, because the rates of transitions between states are assumed to be constant through time, while the transition rates for describing the dynamics of hypertension are likely to be changing over time. In addition, the applications of CTMC rarely account for the effects of other covariates on state transitions. In this article, we considered a non-homogeneous continuous-time Markov chain with two states to analyze changes in hypertension while accounting for multiple covariates. The explicit formulas for the transition probability matrix as well as the corresponding likelihood function were derived. In addition, we proposed a maximum likelihood estimation algorithm for estimating the parameters in the time-dependent rate function. Lastly, the model performance was demonstrated through both a simulation study and application to ambulatory blood pressure data.