Continuous-time Markov Chain Research Articles

A Markovian Aging Process Forecasting Model: Predicting U.S. Mortality

In this study, we provide a novel finite-state Markov model for predicting death rates. The Markovian physiological age, which forms the basis of this model, is represented by the states in the underlying continuous-time Markov chain. This model forecasts mortality rates for the Markovian physiological age, for which mortality rates for calendar ages can be easily computed. A set of data from the U.S. population is used to calibrate the model. The data collection includes individuals aged 30 to 108 and covers the years 1970 to 2019. We train the model using data from 1970 to 2014 and then test it using data from 2015 to 2019. Based on metrics utilized for training and test datasets, the suggested model outperforms the models of both Lee and Carter (1992) and Renshaw and Haberman (2006). It is noteworthy that this method uses fewer model parameters than the comparable models. Forecasts of mortality rates and life expectancy are made using the findings. According to the findings, a 30-year-old’s life expectancy in 2040, 2060, and 2080 will be 55, 59, and 63 years, respectively, which is longer than the basic Lee-Carter model predicted. The model in this work, unlike the Lee Carter model, is identifiable.

Estimating the prevalence of terrorism under control policies. A statistical modelling approach

Measuring the intensity of the prevalence of radicalization plays a vital role in countering violent extremism. Hence, several mathematical models have been proposed aiming to describe the complex dynamics of this social phenomenon. It should be noted that existing literature mainly includes deterministic compartmental models, while the measuring of the extremism's intensity is based on the basic reproduction number R0. However, deterministic approaches can only provide an estimation of the mean tendency of the phenomenon, while the R0 index is accompanied by several drawbacks. In this paper, we propose a new compartmental scheme which considers control policies, containing susceptible, resistant, extremist and under-rehabilitation cases. The aforementioned model is combined with the methodology of continuous time Markov processes, leading to the quantification of not just the mean trend but the uncertainty of the phenomenon, too. Moreover, we introduce two new stochastic descriptors for the assessment of the radicalization's severity, namely the exact (Rε) and aggregate (Ra) reproduction numbers that address the limitation of the widely used R0 index. Emphasis is placed on the presentation of theorems and algorithms that describe the computation of their complete distribution functions. We derive formulas for the calculation of elasticities, and perform a perturbation analysis, shedding light on the overall impact of system's parameters on the selected stochastic descriptors. A series of illustrative examples is presented for the illustration of the derived theoretical properties, along with numerical comparisons between the three reproduction numbers. Finally, based on the proposed measures, we pinpoint the most impactful factors, and discuss control policies that facilitate the effective combating of the negative repercussions of terrorism.

Markov generators as non-Hermitian supersymmetric quantum Hamiltonians: spectral properties via bi-orthogonal basis and singular value decompositions

Continuity equations associated with continuous-time Markov processes can be considered as Euclidean Schrödinger equations, where the non-Hermitian quantum Hamiltonian H=divJ is naturally factorized into the product of the divergence operator div and the current operator J . For non-equilibrium Markov jump processes in a space of N configurations with M links and C=M−(N−1)⩾1 independent cycles, this factorization of the N × N Hamiltonian H=I†J involves the incidence matrix I and the current matrix J of size M × N, so that the supersymmetric partner H^=JI† governing the dynamics of the currents existing on the M links is of size M × M. To better understand the relations between the spectral decompositions of these two Hamiltonians H=I†J and H^=JI† with respect to their bi-orthogonal basis of right and left eigenvectors that characterize the relaxation dynamics towards the steady state and the steady currents, it is useful to analyze the properties of the singular value decomposition of the two rectangular matrices I and J of size M × N and the interpretations in terms of discrete Helmholtz decompositions. This general framework concerning Markov jump processes can be adapted to non-equilibrium diffusion processes governed by Fokker–Planck equations in dimension d, where the number N of configurations, the number M of links and the number C=M−(N−1) of independent cycles become infinite, while the two matrices I and J become first-order differential operators acting on scalar functions to produce vector fields.

Stochastic EM algorithm for partially observed stochastic epidemics with individual heterogeneity.

We develop a stochastic epidemic model progressing over dynamic networks, where infection rates are heterogeneous and may vary with individual-level covariates. The joint dynamics are modeled as a continuous-time Markov chain such that disease transmission is constrained by the contact network structure, and network evolution is in turn influenced by individual disease statuses. To accommodate partial epidemic observations commonly seen in real-world data, we propose a stochastic EM algorithm for inference, introducing key innovations that include efficient conditional samplers for imputing missing infection and recovery times which respect the dynamic contact network. Experiments on both synthetic and real datasets demonstrate that our inference method can accurately and efficiently recover model parameters and provide valuable insight at the presence of unobserved disease episodes in epidemic data.

Ray–Knight compactification of birth and death processes

A birth and death process is a continuous-time Markov chain with minimal state space N, whose transition matrix is standard and whose density matrix is a birth–death matrix. Birth and death process is unique if and only if ∞ is an entrance or natural. When ∞ is neither an entrance nor natural, there are two ways in the literature to obtain all birth and death processes. The first one is an analytic treatment proposed by Feller in 1959, and the second one is a probabilistic construction completed by Wang in 1958.In this paper we will give another way to study birth and death processes using the Ray–Knight compactification. This way has the advantage of both the analytic and probabilistic treatments above. By applying the Ray–Knight compactification, every birth and death process can be modified into a càdlàg Ray process on N∪{∞}∪{∂}, which is either a Doob processes or a Feller Q-process. Every birth and death process in the second class has a modification that is a Feller process on N∪{∞}∪{∂}. We will derive the expression of its infinitesimal generator, which explains its boundary behaviours at ∞. Furthermore, by using the killing transform and the Ikeda–Nagasawa–Watanabe piecing out procedure, we will also provide a probabilistic construction for birth and death processes. This construction relies on a triple determining the resolvent matrix introduced by Wang and Yang in their work (Wang and Yang, 1992).

Some Probabilistic Interpretations Related to the Next-Generation Matrix Theory: A Review with Examples

The fact that the famous basic reproduction number R0, i.e., the largest eigenvalue of the next generation matrix FV−1, sometimes has a probabilistic interpretation is not as well known as it deserves to be. It is well understood that half of this formula, −V, is a Markovian generating matrix of a continuous-time Markov chain (CTMC) modeling the evolution of one individual on the compartments. It has also been noted that the not well-enough-known rank-one formula for R0 of Arino et al. (2007) may be interpreted as an expected final reward of a CTMC, whose initial distribution is specified by the rank-one factorization of F. Here, we show that for a large class of ODE epidemic models introduced in Avram et al. (2023), besides the rank-one formula, we may also provide an integral renewal representation of R0 with respect to explicit “age kernels” a(t), which have a matrix exponential form.This latter formula may be also interpreted as an expected reward of a probabilistic continuous Markov chain (CTMC) model. Besides the rather extensively studied rank one case, we also provide an extension to a case with several susceptible classes.

Impact of dust and temperature on photovoltaic panel performance: A model-based approach to determine optimal cleaning frequency

Enhancing the reliability of photovoltaic (PV) systems is of paramount importance, given their expanding role in sustainable energy production, carbon emissions reduction, and supporting industrial growth. However, PV panels commonly encounter issues that significantly impact their performance. Specifically, the accumulation of dust and the rise in internal temperature lead to a drop in energy production efficiency. The primary issue addressed in this paper is using mathematical modeling to determine the optimal cleaning frequency. This paper first focuses on stochastic modeling for dust accumulation and temperature changes in PV panels, considering varying environmental conditions and proposing a model-based approach to determine the optimal cleaning frequency. Dust accumulation is described using a Non-homogeneous compound Poisson process (NHCPP), while temperature evolution is modeled using Markov chains. Within this framework, we consider the impact of wind speed and rainfall on dust accumulation and temperature. These factors, treated as covariates, are modeled using a two-dimensional time-continuous Markov chain with a finite state space. A Condition-based cleaning policy is proposed and assessed based on the degradation model. Optimal preventive cleaning thresholds and cleaning frequency (periodic and non-periodic) are determined to minimize the long-term average maintenance cost. The gain achieved by non-periodic inspections compared to periodic inspections ranges from 3.83% to 9.37%. Numerical experiments demonstrate the performance of the proposed cleaning policy, highlighting its potential to improve PV system efficiency and reliability.

Efficient valuation of variable annuities under regime-switching jump diffusion models with surrender risk and mortality risk

We present an efficient valuation approach for guaranteed minimum accumulation benefits (GMABs), guaranteed minimum death benefits (GMDBs), and surrender benefits (SBs) embedded in variable annuity (VA) contracts in a regime-switching jump diffusion model. We incorporate into the contract the risks of mortality and surrender, with these events generally monitored discretely over the life of the policy. Using a combination of the continuous-time Markov chain (CTMC) approximation and the Fourier cosine series expansion (COS) method, we determine that the valuation problem can be resolved within a regime-switching jump diffusion framework. Extensive numerical experiments showcase the efficiency of the proposed method, which proves to be more advantageous when compared to existing approaches like Monte Carlo (MC) simulation. The thorough analysis explores how model parameters affect the valuation outcomes.

Sampling and filtering with Markov chains

A new continuous-time Markov chain rate change formula is proven. This theorem is used to derive existence and uniqueness of novel filtering equations akin to the Duncan–Mortensen–Zakai equation and the Fujisaki–Kallianpur–Kunita equation but for Markov signals with general continuous-time Markov chain observations. The equations in this second theorem have the unique feature of being driven by both the observations and the process counting the observation transitions. A direct method of solving these filtering equations is also derived. Most results apply as special cases to the continuous-time Hidden Markov Models (CTHMM), which have become important in applications like disease progression tracking, The corresponding CTHMM results are stated as corollaries. Finally, application of our general theorems to Markov chain importance sampling, rejection sampling and branching particle filtering algorithms is also explained and these are illustrated by way of disease tracking simulations.

Reliability analysis for 5G industrial network with application and dynamic coupling

The analysis of reliability is vital in ensuring the operation of 5G industrial networks (5GIN). With 5GIN are no longer merely chains of physical components, they consist of different applications that are dynamically deployed on base platform. Such a sharing of infrastructure between applications complicates the fault logic and greatly impacts their operation. Current reliability analysis tends to treat system as static, fail to take applications and dynamic coupling into accounts. This paper proposes a model consisting of network evolution model of 5GIN (NEM5G) and reliability-based sensitivity analysis (SA). NEM5G depicts the interaction of physical components and applications, which provides a description of dynamic coupling within system and its effect on reliability. Furthermore, SA helps identify influential parameters within 5GIN to support reliability optimization. The simulation results show that, our model achieves steady-state reliability aligned with that of the Continuous Time Markov Chain (CTMC) model in a simplified intelligent optical network case. When facing complex steel processing scenario, simulation results illustrate the variation trend of 5GIN reliability, further presents an importance ranking of parameters and analyzes available design strategies.

Spectrum and Energy Efficiency in CRNs Using Dynamic Channel Reservation and Komodo Mlipir Optimization

Cognitive Radio Networks (CRNs) block new arriving services and remove existing services to increase the spectrum efficiency. Particularly, Secondary Users (SUs) experience a decline in service deprivation in the arrival of Primary Users (PUs). To overcome this, a Spectrum and energy Efficiency in Cognitive Radio Networks using Dynamic Channel Reservation and Komodo Mlipir Optimization (SE-KMOA-CRN) is proposed. The proposed method splits reserved and non-reserved bands from the accessible spectrum. Numerous channels from the non-reserved band are dynamically assigned to the reserved band to support services that experience disruptions when functioning on non-reserved band. From the non-reserved band, a count of channels dynamically assigned to the reserved band to support services that experience disruptions when functioning on the non-reserved band. It enables priority-based channel assignment and termination while providing dynamic access to the obtainable spectrum. Here, Allocation of channels to Reserved Band CRN using Dynamic Channel Reservation Algorithm (DCRA). Komodo Mlipir Optimization Algorithm (KMOA) is employed for Service selection and Channel restoration. The proposed SE-KMOA-CRN method is modelled utilizing Continuous Time Markov Chain (CTMC) with mathematical equations for some quality of service (QoS) factors is obtained. The SE-KMOA-CRN approach attains higher Throughput, lower End-to-End Delay, higher Energy Efficiency, greater spectral efficiency and higher Packet Delivery Ratio compared with existing methods, like Multiple objective optimization for spectrum along energy efficacy trade-off in IRS-assisted CRNs using NOMA (SE-MOO-CRN), Energy efficacy in CRN utilizing cooperative spectrum sensing under hybrid spectrum handoff (SE-TOA-CRN) and Energy-effectual cross-layer spectrum sharing on CR green IoT (SE-CRG-CRN) respectively.

Estimating sojourn time and sensitivity of screening for ovarian cancer using a Bayesian framework.

Ovarian cancer is among the leading causes of gynecologic cancer-related death. Past ovarian cancer screening trials using combination of cancer antigen 125 testing and transvaginal ultrasound failed to yield statistically significant mortality reduction. Estimates of ovarian cancer sojourn time-that is, the period from when the cancer is first screen detectable until clinical detection-may inform future screening programs. We modeled ovarian cancer progression as a continuous time Markov chain and estimated screening modality-specific sojourn time and sensitivity using a Bayesian approach. Model inputs were derived from the screening arms (multimodal and ultrasound) of the UK Collaborative Trial of Ovarian Cancer Screening and the Prostate, Lung, Colorectal and Ovarian cancer screening trials. We assessed the quality of our estimates by using the posterior predictive P value. We derived histology-specific sojourn times by adjusting the overall sojourn time based on the corresponding histology-specific survival from the Surveillance, Epidemiology, and End Results Program. The overall ovarian cancer sojourn time was 2.1 years (posterior predictive P value = .469) in the Prostate, Lung, Colorectal and Ovarian studies, with 65.7% screening sensitivity. The sojourn time was 2.0 years (posterior predictive P value = .532) in the United Kingdom Collaborative Trial of Ovarian Cancer Screening's multimodal screening arm and 2.4 years (posterior predictive P value = .640) in the ultrasound screening arm, with sensitivities of 93.2% and 64.5%, respectively. Stage-specific screening sensitivities in the Prostate, Lung, Colorectal and Ovarian studies were 39.1% and 82.9% for early-stage and advanced-stage disease, respectively. The histology-specific sojourn times ranged from 0.8 to 1.8 years for type II ovarian cancer and 2.9 to 6.6 years for type I ovarian cancer. Annual screening is not effective for all ovarian cancer subtypes. Screening sensitivity for early-stage ovarian cancers is not sufficient for substantial mortality reduction.

Mutual Linearity of Nonequilibrium Network Currents.

For continuous-time Markov chains and open unimolecular chemical reaction networks, we prove that any two stationary currents are linearly related upon perturbations of a single edge's transition rates, arbitrarily far from equilibrium. We extend the result to nonstationary currents in the frequency domain, provide and discuss an explicit expression for the current-current susceptibility in terms of the network topology, and discuss possible generalizations. In practical scenarios, the mutual linearity relation has predictive power and can be used as a tool for inference or model proof testing.

A genetic algorithm and particle swarm optimization for redundancy allocation problem in systems with limited number of non-cooperating repairmen

This article considered the redundancy allocation problem (RAP) in repairable heterogeneous systems composed of k-out-of-n: G subsystems with a limited number of repairmen. The unit costs of the binary-state components within the subsystems remain constant over time. Exponential characteristics of independent failure and repair processes were assumed. To evaluate subsystem and system availability, a Continuous Time Markov Chain (CTMC) adapted to three standby modes (cold, warm, and hot) was developed. The characteristics of the redundant system and the assumption of a limited number of repairmen contribute to the existing scientific literature on RAP. Two nature-inspired metaheuristic approaches were developed to maximize the availability of the coherent system as an objective function to optimize redundancy with a cost constraint on component purchase. The proposed approaches were tested on a 15-unit large-scale system. The effectiveness and performance of the algorithms was evaluated as a function of the number of generations/iterations and population/swarm size. The genetic algorithm (GA) proved to be highly effective in finding the global optimum and outperformed particle swarm optimization (PSO) in terms of computational performance. Analysis of metaheuristic algorithms can offer valuable insights into future research on redundancy optimization. The sensitivity analysis of the large-scale system clearly demonstrates that the number of repairmen and cost constraints (up to three times the sum of the value of active components) have a significant impact on the system’s availability. Above this level, increasing spending on system building, regardless of the number of repairmen, is not justified by considerations of system availability.

Stability of Queueing Systems with Impatience, Balking and Non-Persistence of Customers

The operation of many queueing systems is adequately described by the structured multidimensional continuous-time Markov chains. The most well-studied classes of such chains are level-independent Quasi-Birth-and-Death processes, GI/M/1 type and M/G/1 type Markov chains, generators of which have the block tri-diagonal, lower- and upper-Hessenberg structure, respectively. All these classes assume that the matrices of transition rates are quasi-Toeplitz. This property greatly simplifies their analysis but makes them inappropriate for the study of many important systems, e.g., retrial queues with a retrial rate depending on the number of customers in orbit, queues with impatient customers, etc. The importance of such systems attracts significant interest to their analysis. However, in the literature, there is a methodological gap relating to the ergodicity condition of the corresponding Markov chains. To fulfill this gap and facilitate the analysis of a wide range of such systems, we show that under non-restrictive assumptions, the following hold true: (i) if the customers can balk or are impatient or non-persistent, then the Markov chain describing the behavior of the system belongs to the class of asymptotically quasi-Toeplitz Markov chains; (ii) this chain is ergodic; (iii) known algorithms can be applied for the calculation of the stationary distribution of the corresponding queueing system.

Stochastic filtering of reaction networks partially observed in time snapshots

Stochastic reaction network models arise in intracellular chemical reactions, epidemiological models and other population process models, and are a class of continuous time Markov chains which have the nonnegative integer lattice as state space. We consider the problem of estimating the conditional probability distribution of a stochastic reaction network given exact partial state observations in time snapshots. We propose a particle filtering method called the targeting method. Our approach takes into account that the reaction counts in between two observation snapshots satisfy linear constraints and also uses inhomogeneous Poisson processes as proposals for the reaction counts to facilitate exact interpolation. We provide rigorous analysis as well as numerical examples to illustrate our method and compare it with other alternatives.

Quantum-embeddable stochastic matrices

The classical embeddability problem asks whether a given stochastic matrix T, describing transition probabilities of a d-level system, can arise from the underlying homogeneous continuous-time Markov process. Here, we investigate the quantum version of this problem, asking of the existence of a Markovian quantum channel generating state transitions described by a given T. More precisely, we aim at characterising the set of quantum-embeddable stochastic matrices that arise from memoryless continuous-time quantum evolution. To this end, we derive both upper and lower bounds on that set, providing new families of stochastic matrices that are quantum-embeddable but not classically-embeddable, as well as families of stochastic matrices that are not quantum-embeddable. As a result, we demonstrate that a larger set of transition matrices can be explained by memoryless models if the dynamics is allowed to be quantum, but we also identify a non-zero measure set of random processes that cannot be explained by either classical or quantum memoryless dynamics. Finally, we fully characterise extreme stochastic matrices (with entries given only by zeros and ones) that are quantum-embeddable.

VIX Options Valuation via Continuous-Time Markov Chain Approximation and Ito-Taylor Expansion

VIX Options Valuation via Continuous-Time Markov Chain Approximation and Ito-Taylor Expansion

Seasonal variability and stochastic branching process in malaria outbreak probability

Background: Malaria is the world’s most fatal and challenging parasitic disease, caused by the Plasmodium parasite, which is transmitted to humans by the bites of infected female mosquitoes. Bangladesh is the most vulnerable region to spread malaria because of its geographic position. In this paper, we have considered the dynamics of vector-host models and observed the stochastic behavior. This study elaborates on the seasonal variability and calculates the probability of disease outbreaks. Methods: We present a model for malaria disease transmission and develop its corresponding continuous-time Markov chain (CTMC) representation. The proposed vector-host models illustrate the malaria transmission model along with sensitivity analysis. The deterministic model with CTMC curves is depicted to show the randomness in real scenarios. Sequentially, we expand these studies to a time-varying stochastic vector-host model that incorporates seasonal variability. Phase plane analysis is conducted to explore the characteristics of the disease, examine interactions among various compartments, and evaluate the impact of key parameters. The branching process approximation is developed for the corresponding vector-host model to calculate the probability outbreak. Numerous numerical results are accomplished to observe the analytical investigation. Results: Seasonality and contact patterns affect the dynamics of disease outbreaks. The numerical illustration provides that the probability of a disease outbreak depends on the infected host or vector. Additionally, periodic transmission rates have a great influence on the probability outbreak. The basic reproduction number (R0) is derived, which is the main justification for studying the dynamical behavior of epidemic models. Conclusions: Seasonal variability significantly impacts malaria transmission, and the probability of disease outbreaks is influenced by time and the initial number of infected individuals. Moreover, the branching process approximation is applicable when the population size is large enough and the basic reproduction number is less than 1. In the future, such analysis can help decision-makers understand the impact of various parameters and their stochastic behavior in the vector-host model to prevent such types of disease outbreaks.

One-dimensional continuous-time quantum Markov chains: qubit probabilities and measures

Quantum Markov chains (QMCs) are positive maps on a trace-class space describing open quantum dynamics on graphs. Such objects have a statistical resemblance with classical random walks, while at the same time they allow for internal (quantum) degrees of freedom. In this work we study continuous-time QMCs on the integer line, half-line and finite segments, so that we are able to obtain exact probability calculations in terms of the associated matrix-valued orthogonal polynomials and measures. The methods employed here are applicable to a wide range of settings, but we will restrict ourselves to classes of examples for which the Lindblad generators are induced by a single positive map, and such that the Stieltjes transforms of the measures and their inverses can be calculated explicitly.