Continuous-time Markov Chain Model Research Articles

Modeling Emergency Traffic Using a Continuous-Time Markov Chain

This paper aims to propose a novel call for help traffic (SOS) and study its impact over Machine-to-Machine (M2M) and Human-to-Human (H2H) traffic in Internet of Things environments, specifically during disaster events. During such events (e.g., the spread COVID-19), SOS traffic, with its predicted exponential increase, will significantly influence all mobile networks. SOS traffic tends to cause many congestion overload problems that significantly affect the performance of M2M and H2H traffic. In our project, we developed a new Continuous-Time Markov Chain (CTMC) model to analyze and measure radio access performance in terms of massive SOS traffic that influences M2M and H2H traffic. Afterwards, we validate the proposed CTMC model through extensive Monte Carlo simulations. By analyzing the traffic during an emergency case, we can spot a huge impact over the three traffic types of M2M, H2H and SOS traffic. To solve the congestion problems while keeping the SOS traffic without any influence, we propose to grant the SOS traffic the highest priority over the M2M and H2H traffic. However, by implementing this solution in different proposed scenarios, the system becomes able to serve all SOS requests, while only 20% of M2M and H2H traffic could be served in the worst-case scenario. Consequently, we can alleviate the expected shortage of SOS requests during critical events, which might save many humans and rescue them from being isolated.

Predicting the Impact of Distributed Denial of Service (DDoS) Attacks in Long-Term Evolution for Machine (LTE-M) Networks Using a Continuous-Time Markov Chain (CTMC) Model

The way we connect with the physical world has completely changed because of the advancement of the Internet of Things (IoT). However, there are several difficulties associated with this change. A significant advancement has been the emergence of intelligent machines that are able to gather data for analysis and decision-making. In terms of IoT security, we are seeing a sharp increase in hacker activities worldwide. Botnets are more common now in many countries, and such attacks are very difficult to counter. In this context, Distributed Denial of Service (DDoS) attacks pose a significant threat to the availability and integrity of online services. In this paper, we developed a predictive model called Markov Detection and Prediction (MDP) using a Continuous-Time Markov Chain (CTMC) to identify and preemptively mitigate DDoS attacks. The MDP model helps in studying, analyzing, and predicting DDoS attacks in Long-Term Evolution for Machine (LTE-M) networks and IoT environments. The results show that using our MDP model, the system is able to differentiate between Authentic, Suspicious, and Malicious traffic. Additionally, we are able to predict the system behavior when facing different DDoS attacks.

Performing global sensitivity analysis on simulations of a continuous-time Markov chain model motivated by epidemiology

Performing global sensitivity analysis on simulations of a continuous-time Markov chain model motivated by epidemiology

Reliability analysis for 5G industrial network with application and dynamic coupling

The analysis of reliability is vital in ensuring the operation of 5G industrial networks (5GIN). With 5GIN are no longer merely chains of physical components, they consist of different applications that are dynamically deployed on base platform. Such a sharing of infrastructure between applications complicates the fault logic and greatly impacts their operation. Current reliability analysis tends to treat system as static, fail to take applications and dynamic coupling into accounts. This paper proposes a model consisting of network evolution model of 5GIN (NEM5G) and reliability-based sensitivity analysis (SA). NEM5G depicts the interaction of physical components and applications, which provides a description of dynamic coupling within system and its effect on reliability. Furthermore, SA helps identify influential parameters within 5GIN to support reliability optimization. The simulation results show that, our model achieves steady-state reliability aligned with that of the Continuous Time Markov Chain (CTMC) model in a simplified intelligent optical network case. When facing complex steel processing scenario, simulation results illustrate the variation trend of 5GIN reliability, further presents an importance ranking of parameters and analyzes available design strategies.

Integrated analysis of reliability, power, and performance for IoT devices and servers

The Internet of Things (IoT) is currently widely used in various sectors and spaces. IoT devices are becoming small yet powerful servers and perform server-like functions. Reliability is a critical aspect in both IoT devices and servers, as they work together to create a robust and dependable IoT ecosystem. Power and performance are two other major considerations of an IoT system. Modeling, analysis, evaluation, and optimization of reliability, power, and performance for IoT devices and servers are major components in IoT systems development and deployment. In this paper, we conduct an integrated study of reliability, power, and performance for IoT devices and servers by mathematically rigorous modeling and analysis. The contributions of the paper can be summarized as follows. We establish a continuous-time Markov chain (CTMC) model that incorporates server failure rate, server repair rate, task arrival rate, and task processing rate. Using such an analytical model, we can calculate the server availability, the average task response time, and the average power consumption. We point out that there is an optimal server speed that minimizes the power-time product and a combined cost-performance metric of power, performance, and reliability. We show the impact of server reliability on response time, power consumption, server utilization, and the power-performance tradeoff. To the best of the author’s knowledge, this is the first paper that takes a combined approach to modeling and analysis of reliability, power, and performance for IoT devices and servers. It has been noticed that there has been little such theoretically solid investigation in the existing literature. Therefore, this paper has made tangible contributions and significant advances in the joint understanding of reliability, power, performance, and their interplay in IoT devices and servers quantitatively and mathematically.

Abstract PR013: Detecting pairwise and higher-order antagonistic epistatic effects among somatic cancer genotypes to discover synthetic lethality

Abstract The mutual exclusivity of somatic variants in cancer suggests that current somatic variants can exert antagonistic epistatic effects on the selection of new mutations, contributing to a complex landscape of evolutionary trajectories. However, quantifying pairwise and higher-order epistatic effects—which are essential to estimation of the trajectory of likely cancer genotypes—has been a challenge. We have developed a continuous-time Markov chain model that enables the estimation of mutation origination and fixation, dependent on somatic cancer genotype. We coupled the continuous-time Markov chain model with an approach that deconvolutes the underlying mutation rates and selection intensities across trajectories of oncogenesis. By elucidating the routes of variant evolution and the strengths of selective forces at play, our approach lays the groundwork for quantifying antagonistic epistatic effects of specific somatic genotypes. We applied our approach to report the variant mutation rates, selection intensities, and consequent substitution rates for lung adenocarcinoma, comparing smoker and nonsmoker tumor cohorts. We describe the distinct influences of smoking on underlying mutation rates and on physiology affecting the selective regime of lung tissue. Inference of the most likely routes of site-specific variant evolution, as well as estimation of the selection strength operating on each step along the route represents a key component for prioritization, development, and implementation of personalized cancer therapies that target synthetic lethality. Citation Format: Jorge A. Alfaro-Murillo, Krishna Dasari, Jeffrey P. Townsend. Detecting pairwise and higher-order antagonistic epistatic effects among somatic cancer genotypes to discover synthetic lethality [abstract]. In: Proceedings of the AACR Special Conference in Cancer Research: Expanding and Translating Cancer Synthetic Vulnerabilities; 2024 Jun 10-13; Montreal, Quebec, Canada. Philadelphia (PA): AACR; Mol Cancer Ther 2024;23(6 Suppl):Abstract nr PR013.

Random-Effects Substitution Models for Phylogenetics via Scalable Gradient Approximations.

Phylogenetic and discrete-trait evolutionary inference depend heavily on an appropriate characterization of the underlying character substitution process. In this paper, we present random-effects substitution models that extend common continuous-time Markov chain models into a richer class of processes capable of capturing a wider variety of substitution dynamics. As these random-effects substitution models often require many more parameters than their usual counterparts, inference can be both statistically and computationally challenging. Thus, we also propose an efficient approach to compute an approximation to the gradient of the data likelihood with respect to all unknown substitution model parameters. We demonstrate that this approximate gradient enables scaling of sampling-based inference, namely Bayesian inference via Hamiltonian Monte Carlo, under random-effects substitution models across large trees and state-spaces. Applied to a dataset of 583 SARS-CoV-2 sequences, an HKY model with random-effects shows strong signals of nonreversibility in the substitution process, and posterior predictive model checks clearly show that it is a more adequate model than a reversible model. When analyzing the pattern of phylogeographic spread of 1441 influenza A virus (H3N2) sequences between 14 regions, a random-effects phylogeographic substitution model infers that air travel volume adequately predicts almost all dispersal rates. A random-effects state-dependent substitution model reveals no evidence for an effect of arboreality on the swimming mode in the tree frog subfamily Hylinae. Simulations reveal that random-effects substitution models can accommodate both negligible and radical departures from the underlying base substitution model. We show that our gradient-based inference approach is over an order of magnitude more time efficient than conventional approaches.

Abstract PO5-24-09: Optimizing the Therapeutic Treatment of Breast Cancer Through a Dual Data-driven Theoretical Model

Abstract Despite rapid advancements in treatments and technology, breast cancer remains the second leading cause of cancer related morbidity in women in the US. As a heterogeneous disease, the unique genetic and phenotypic profile of each tumor complicates the prediction of treatment responses and optimized treatment strategies. One primary manifestation of phenotypic plasticity and concomitant intra-tumoral heterogeneity is the Epithelial-Mesenchymal Transition (EMT). The acquisition of migratory and invasive properties through EMT facilitates tumor metastasis and results in worse patient outcomes. As a result, the EMT and its reverse process (MET) often influence treatment response. However, the different mechanisms through which these therapeutic drugs affect the EMT-related phenotypic switching rates as well as each phenotype’s replication rates are unclear, and further investigation is required for better understanding their precise effect for directing optimal treatment strategies and improved patient outcomes. Here, we compile an atlas of publicly available metastatic and non-metastatic RNA sequencing data of breast cancer cell lines treated with therapeutic and chemotherapeutic drugs and propose a dual data-driven theoretical framework for understanding the effects of each drug on breast cancer treatment response. We infer the cell fractions, EMT phenotypic transition rates and phenotypic replication rates of each case through an extension of our previously developed hybrid data-driven and theoretical model, COMET. We infer single-cell RNA sequencing-derived EMT-related trajectories and apply a continuous time Markov chain model to infer inter-state transition rates. By analyzing the atlas of RNA sequencing data and integration with our framework, we gain novel insights into the context-specific mechanisms by which therapeutic drugs affect the phenotypic transition rates. This comprehensive understanding will contribute to optimizing breast cancer therapy and ultimately improving patient outcomes. Citation Format: A. Najafi. Optimizing the Therapeutic Treatment of Breast Cancer Through a Dual Data-driven Theoretical Model [abstract]. In: Proceedings of the 2023 San Antonio Breast Cancer Symposium; 2023 Dec 5-9; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2024;84(9 Suppl):Abstract nr PO5-24-09.

A Binary-State Continuous-Time Markov Chain Model for Offshoring and Reshoring

We present a two-country model (North and South) that describes the phenomenon of offshoring and reshoring. The model is a continuous time-controlled Markov chain with binary states. The main trade-off involves production costs and transaction costs between one country and another. In the first part of this paper, we identify the key parameters of the model: the difference in unit production costs between the two countries considered, the marginal cost of transitioning between countries, and the incentive paid by the North country to all companies that have not relocated at the end of the planning interval. The final goal of our paper is to understand how national tax incentives can influence this process.

Abstract 1251: Tobacco smoke alters the adaptive landscape of lung adenocarcinoma and influences the strength of epistatic interactions

Abstract Tobacco smoke produces both mutagenic and physiological effects, altering the mutation rate of cells and the selection acting on them in the damaged lung environment, and facilitating its role in driving cancer progression. In addition to these gene-by-environment interactions, gene-by-gene epistatic interactions also shape the adaptive landscape, influencing the evolutionary trajectories of cancer. Understanding the influence of tobacco smoke on the epistatic trajectories of lung cancer progression would help guide the development of targeted therapies that are more effective for smoker or never-smoker populations. To address this need, we constructed a continuous-time Markov chain model for the evolutionary trajectories of lung adenocarcinoma (LUAD), the most common subtype of lung cancer and the most frequent subtype among never-smokers. Using thousands of tumor sample sequences aggregated across studies, we profiled the trajectories of LUAD in smokers and never-smokers and estimated the rates at which mutations are gained from each possible set of pre-existing mutations, or genetic states. We then accounted for differences in baseline mutation rates to quantify the selective benefit of mutations. We find that epistasis is prevalent in LUAD, with several strong synergistic interactions—such as between RB1 and EGFR mutations- and antagonistic interactions-such as between EGFR and KEAP1 or KRAS mutations. Additionally, we identify non-additive epistatic effects: STK11 mutations are synergistic with KEAP1 and KRAS mutations but do not experience additional selection when both are mutated. In contrast, GNAS mutations are not synergistic with KEAP1 or RBM10 mutations alone but are synergistic with their co-mutation. Smoking has a large influence on the adaptive landscape, with several common mutations being preferentially selected in never-smoker cancers, including EGFR, SMAD4, GNAS, and PIK3CA mutations. KEAP1, STK11, and KRAS mutations are preferentially selected in smoker cancers. Smoking’s physiological influence also affects the strength of certain epistatic interactions, such as between STK11 and KEAP1 mutations. Overall, we find that smoking not only increases the mutation rate of lung cells but also substantially alters the adaptive landscape of lung adenocarcinoma, leading to clinically relevant differences in selection on mutations between smoker and never-smoker cancers. We additionally detect frequent pairwise and higher-order epistatic effects in LUAD that may inform the personalized application of targeted therapies. Citation Format: Krishna Dasari, Jorge Alfaro-Murillo, Jeffrey Townsend. Tobacco smoke alters the adaptive landscape of lung adenocarcinoma and influences the strength of epistatic interactions [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 1251.

Dynamical Analysis of an Improved Bidirectional Immunization SIR Model in Complex Network.

In order to investigate the impact of two immunization strategies-vaccination targeting susceptible individuals to reduce their infection rate and clinical medical interventions targeting infected individuals to enhance their recovery rate-on the spread of infectious diseases in complex networks, this study proposes a bilinear SIR infectious disease model that considers bidirectional immunization. By analyzing the conditions for the existence of endemic equilibrium points, we derive the basic reproduction numbers and outbreak thresholds for both homogeneous and heterogeneous networks. The epidemic model is then reconstructed and extensively analyzed using continuous-time Markov chain (CTMC) methods. This analysis includes the investigation of transition probabilities, transition rate matrices, steady-state distributions, and the transition probability matrix based on the embedded chain. In numerical simulations, a notable concordance exists between the outcomes of CTMC and mean-field (MF) simulations, thereby substantiating the efficacy of the CTMC model. Moreover, the CTMC-based model adeptly captures the inherent stochastic fluctuation in the disease transmission, which is consistent with the mathematical properties of Markov chains. We further analyze the relationship between the system's steady-state infection density and the immunization rate through MCS. The results suggest that the infection density decreases with an increase in the immunization rate among susceptible individuals. The current research results will enhance our understanding of infectious disease transmission patterns in real-world scenarios, providing valuable theoretical insights for the development of epidemic prevention and control strategies.

Abstract IA014: Detecting epistatic effects among somatic cancer mutations across oncogenesis

Abstract The somatic evolution of cancer depends on the rate at which new mutations are acquired, as well as the selective pressure acting on them. Further, mutual exclusivity or co-occurrence of somatic variants suggests that current somatic variants can exert antagonistic or synergistic epistatic effects on the selection of new mutations, contributing to a complex landscape of evolutionary trajectories. However, quantifying these epistatic effects—which are essential to estimation of the trajectory of likely cancer genotoypes—has been a challenge, especially for higher-order epistatic effects. To estimate these epistatic effects, we have developed a continuous-time Markov chain model that enables the estimation of mutation origination and fixation, dependent on somatic cancer genotype. Coupling the continuous-time Markov chain model with a mutation-rate deconvolution approach, we have estimated the underlying mutation rates and selection intensities across trajectories of oncogenesis. Specifically, we report the mutation rates, selection intensities, and consequent variant fluxes for lung adenocarcinoma, informed by a compilation of samples from multiple data sets including The Cancer Genome Atlas and the AACR Project GENIE. Furthermore, we compare smoker and nonsmoker tumor cohorts describing the distinct evolutionary trajectories jointly, influenced by differential mutation and also by the differential physiological effects that smoking can have on the selective regime of lung tissue. Our approach enables inference of the most likely routes of site-specific variant evolution, as well as estimation of the selection strength operating on each step along the route—a key component of what we need to know to prioritize, develop, and implement personalized cancer therapies. Citation Format: Jorge A. Alfaro-Murillo, Krishna Dasari, Jeffrey P. Townsend. Detecting epistatic effects among somatic cancer mutations across oncogenesis [abstract]. In: Proceedings of the AACR Special Conference in Cancer Research: Translating Cancer Evolution and Data Science: The Next Frontier; 2023 Dec 3-6; Boston, Massachusetts. Philadelphia (PA): AACR; Cancer Res 2024;84(3 Suppl_2):Abstract nr IA014.

On the surprising effectiveness of a simple matrix exponential derivative approximation, with application to global SARS-CoV-2

The continuous-time Markov chain (CTMC) is the mathematical workhorse of evolutionary biology. Learning CTMC model parameters using modern, gradient-based methods requires the derivative of the matrix exponential evaluated at the CTMC's infinitesimal generator (rate) matrix. Motivated by the derivative's extreme computational complexity as a function of state space cardinality, recent work demonstrates the surprising effectiveness of a naive, first-order approximation for a host of problems in computational biology. In response to this empirical success, we obtain rigorous deterministic and probabilistic bounds for the error accrued by the naive approximation and establish a "blessing of dimensionality" result that is universal for a large class of rate matrices with random entries. Finally, we apply the first-order approximation within surrogate-trajectory Hamiltonian Monte Carlo for the analysis of the early spread of Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) across 44 geographic regions that comprise a state space of unprecedented dimensionality for unstructured (flexible) CTMC models within evolutionary biology.

The Significance of Stochastic CTMC Over Deterministic Model in Understanding the Dynamics of Lymphatic Filariasis With Asymptomatic Carriers

Lymphatic filariasis is a leading cause of chronic and irreversible damage to human immunity. This paper presents deterministic and continuous‐time Markov chain (CTMC) stochastic models regarding lymphatic filariasis dynamics. To account for randomness and uncertainties in dynamics, the CTMC model was formulated based on deterministic model possible events. A deterministic model’s outputs suggest that disease extinction is feasible when the secondary threshold infection number is below one, while persistence becomes likely when the opposite holds true. Furthermore, the significant contribution of asymptomatic carriers was identified. Results indicate that persistence is more likely to occur when the infection results from asymptomatic, acutely infected, or infectious mosquitoes. Consequently, the CTMC stochastic model is essential in capturing variabilities, randomness, associated probabilities, and validity across different scales, whereas oversimplification and unpredictability of inherent may not be featured in a deterministic model.

A stochastic multi-host model for West Nile virus transmission

When initially introduced into a susceptible population, a disease may die out or result in a major outbreak. We present a Continuous-Time Markov Chain model for enzootic WNV transmission between two avian host species and a single vector, and use multitype branching process theory to determine the probability of disease extinction based upon the type of infected individual initially introducing the disease into the population – an exposed vector, infectious vector, or infectious host of either species. We explore how the likelihood of disease extinction depends on the ability of each host species to transmit WNV, vector biting rates on host species, and the relative abundance of host species, as well as vector abundance. Theoretical predictions are compared to the outcome of stochastic simulations. We find the community composition of hosts and vectors, as well as the means of disease introduction, can greatly affect the probability of disease extinction.

A Continuous-Time Markov Chain Model for the Spread of COVID-19

Summary Since late 2019 the novel coronavirus, also known as COVID-19, has caused a pandemic that persists. This paper shows how a continuous-time Markov chain model for the spread of COVID-19 can be used to explain, and justify to undergraduate students, strategies now being used in attempts to control the virus. The material in the paper is written at the level of students who are taking an introductory course on the theory and applications of stochastic processes.

Pairwise and higher-order epistatic effects among somatic cancer mutations across oncogenesis

Cancer occurs as a consequence of multiple somatic mutations that lead to uncontrolled cell growth. Mutual exclusivity and co-occurrence of mutations imply—but do not prove—that mutations exert synergistic or antagonistic epistatic effects on oncogenesis. Knowledge of these interactions, and the consequent trajectories of mutation and selection that lead to cancer has been a longstanding goal within the cancer research community. Recent research has revealed mutation rates and scaled selection coefficients for specific recurrent variants across many cancer types. However, there are no current methods to quantify the strength of selection incorporating pairwise and higher-order epistatic effects on selection within the trajectory of likely cancer genotoypes. Therefore, we have developed a continuous-time Markov chain model that enables the estimation of mutation origination and fixation (flux), dependent on somatic cancer genotype. Coupling this continuous-time Markov chain model with a deconvolution approach provides estimates of underlying mutation rates and selection across the trajectory of oncogenesis. We demonstrate computation of fluxes and selection coefficients in a somatic evolutionary model for the four most frequently variant driver genes (TP53, LRP1B, KRAS and STK11) from 565 cases of lung adenocarcinoma. Our analysis reveals multiple antagonistic epistatic effects that reduce the possible routes of oncogenesis, and inform cancer research regarding viable trajectories of somatic evolution whose progression could be forestalled by precision medicine. Synergistic epistatic effects are also identified, most notably in the somatic genotype TP53 LRP1B for mutations in the KRAS gene, and in somatic genotypes containing KRAS or TP53 mutations for mutations in the STK11 gene. Large positive fluxes of KRAS variants were driven by large selection coefficients, whereas the flux toward LRP1B mutations was substantially aided by a large mutation rate for this gene. The approach enables inference of the most likely routes of site-specific variant evolution and estimation of the strength of selection operating on each step along the route, a key component of what we need to know to develop and implement personalized cancer therapies.

Dimensioning the pending interest table in content-centric networks

In chunk-based interest-driven content-centric networks, each interest packet forwarded upstream by a node on a face implies the return of at most one data chunk to the node from that face shortly after. As a consequence, the congestion of the downstream transmission buffer in the data path of the node is highly correlated with the occupancy of the pending interest table (PIT). Therefore a systematic study and analysis of the PIT occupancy are of paramount importance to understanding congestion in CCN. In particular, in this paper, we propose an analytical model to estimate the PIT occupancy distribution via a continuous-time Markov chain (CTMC) model that considers the effects of interest blocking, interest timeout and retries. To validate our model and assumptions, we invoke simulation with realistic traffic streams and show how the filtering effects of caching and interest aggregation make the Markov assumption reasonable in the nodes that are the most susceptible to experiencing congestion. To solve the model numerically, we use two alternative approximations, state space truncation and state aggregation, and then give some numerical results to demonstrate the accuracy of our approximations.

Aerial computing: Enhancing mobile cloud computing with unmanned aerial vehicles as data bridges—A Markov chain based dependability quantification

Aerial Computing, utilizing unmanned aerial vehicles (UAVs), has emerged as a promising solution to enhance mobile cloud computing (MCC) infrastructure for the Internet of Things (IoT). The continuous generation of vast amounts of data by IoT devices requires efficient processing and monitoring for timely decision-making. However, wireless connections between IoT devices and remote servers can be unreliable, resulting in data loss. UAVs, with their increasing processing power and autonomy, can act as bridges between IoT devices and remote servers such as edge or cloud computing. In that context, this paper proposes a continuous time Markov chain (CTMC) models for an aerial computing system to evaluate system dependability metrics including availability and reliability. Sensitivity analysis is conducted to provide extended CTMC models with improved system availability. The proposed advanced model reduces downtime by 62 h compared to the baseline model, showcasing the potential of UAVs in enhancing the availability and reliability of MCC infrastructures. The use of UAVs and MCC in aerial computing is believed to be a win–win solution for cost-effective and energy-saving communication and computation services in various environments.

Stochastic Modeling of a Measles Outbreak in Brazil

Development of mathematical models and its numerical implementations are essential tools in epidemiological modeling. Susceptible-Infected-Recovered (SIR) compartmental model, proposed by Kermack and McKendrick, in 1927, is a widely used deterministic model which serves as a basis for more involved mathematical models. In this work, we consider two stochastic versions of the SIR model for analysing a measles outbreak in Ilha Grande, Rio de Janeiro, in 1976; Continuous Time Markov Chain and Stochastic Differential Equations. The SIR Continuous Time Markov Chain model is used to extract specific information from the measles outbreak, obtaining results in excellent agreement with the reported epidemic values. Numerical simulations are performed in Python.