Continuous Markov Research Articles

Oncology Simulation Model: A Comprehensive and Innovative Approach to Estimate and Project Prevalence and Survival in Oncology.

We demonstrate a new model framework as an innovative approach to more accurately estimate and project prevalence and survival outcomes in oncology. We developed an oncology simulation model (OSM) framework that offers a customizable, dynamic simulation model to generate population-level, country-specific estimates of prevalence, incidence of patients progressing from earlier stages (progression-based incidence), and survival in oncology. The framework, a continuous dynamic Markov cohort model, was implemented in Microsoft Excel. The simulation runs continuously through a prespecified calendar time range. Time-varying incidence, treatment patterns, treatment rates, and treatment pathways are specified by year to account for guideline-directed changes in standard of care and real-world trends, as well as newly approved clinical treatments. Patient cohorts transition between defined health states, with transitions informed by progression-free survival and overall survival as reported in published literature. Model outputs include point prevalence and period prevalence, with options for highly granular prevalence predictions by disease stage, treatment pathway, or time of diagnosis. As a use case, we leveraged the OSM framework to estimate the prevalence of bladder cancer in the United States. The OSM is a robust model that builds upon existing modeling practices to offer an innovative, transparent approach in estimating prevalence, progression-based incidence, and survival for oncologic conditions. The OSM combines and extends the capabilities of other common health-economic modeling approaches to provide a detailed and comprehensive modeling framework to estimate prevalence in oncology using simulation modeling and to assess the impacts of new treatments on prevalence over time.

Graph-based algorithms for phase-type distributions

Phase-type distributions model the time until absorption in continuous or discrete-time Markov chains on a finite state space. The multivariate phase-type distributions have diverse and important applications by modeling rewards accumulated at visited states. However, even moderately sized state spaces make the traditional matrix-based equations computationally infeasible. State spaces of phase-type distributions are often large but sparse, with only a few transitions from a state. This sparseness makes a graph-based representation of the phase-type distribution more natural and efficient than the traditional matrix-based representation. In this paper, we develop graph-based algorithms for analyzing phase-type distributions. In addition to algorithms for state space construction, reward transformation, and moments calculation, we give algorithms for the marginal distribution functions of multivariate phase-type distributions and for the state probability vector of the underlying Markov chains of both time-homogeneous and time-inhomogeneous phase-type distributions. The algorithms are available as a numerically stable and memory-efficient open source software package written in C named ptdalgorithms. This library exposes all methods in the programming languages C and R. We compare the running time of ptdalgorithms to the fastest tools using a traditional matrix-based formulation. This comparison includes the computation of the probability distribution, which is usually computed by exponentiation of the sub-intensity or sub-transition matrix. We also compare time spent calculating the moments of (multivariate) phase-type distributions usually defined by inversion of the same matrices. The numerical results of our graph-based and traditional matrix-based methods are identical, and our graph-based algorithms are often orders of magnitudes faster. Finally, we demonstrate with a classic problem from population genetics how ptdalgorithms serves as a much faster, simpler, and completely general modeling alternative.

Spatial Markov Semigroups Admit Hudson-Parthasarathy Dilations

We present, for the first time, the result (from 2008) that (normal, strongly continuous) Markov semigroups on $\mathscr{B}(G)$ ($G$ a separable Hilbert space) admit a Hudson-Parthasarathy dilation (that is, a dilation to a cocycle perturbation of a noise) if and only if the Markov semigroup is spatial (that is, if it dominates an elementary CP-semigroup). The proof is by general abstract nonsense (taken from Arveson's classification of $E_0$-semigroups on $\mathscr{B}(H)$ by Arveson systems up to cocycle conjugacy) and not, as usual, by constructing the cocycle as a solution of a quantum stochastic differential equation in the sense of Hudson and Parthasarathy. All other results that make similar statements (especially, [Mem. Amer. Math. Soc. 240 (2016), vi+126 pages, arXiv:0901.1798]) for more general $C^*$-algebras) have been proved later by suitable adaptations of the methods exposed here. (They use Hilbert module techniques, which we carefully avoid here in order to make the result available without any appeal to Hilbert modules.)

Lamb wave-based damage assessment for CFRP composite structures using a CHMM-based damage localization algorithm and a damage quantitative expression

Accurately and quantitatively monitoring the damages in composite structures is very challenging, but important for assessing their integrity. In this paper, an innovative damage location identification and size quantification assessment method based on Lamb wave is proposed for composite laminates. A continuous hidden Markov model (CHMM)-based damage scaling factor together with a weighted average imaging algorithm is adopted in damage location identification. To quantify the degree of structural deviation from the healthy state, the inconspicuous change of guided wave (GW) signal is converted to a CHMM-based damage scaling factor with significant change. Based on the damage scaling factors of all GW sensing paths, a weighted average imaging algorithm is performed to identify the most probable damage location. In addition, a damage quantitative mathematical expression is proposed to approximately estimate the size of circular-shaped damage founded on several different damage-related characterizations. The implementation of the proposed quantitative methodology is comparatively simple and straightforward for damage detection. Experiments on three composite panels with multiple damage cases are conducted to substantiate the proposed method. The sensor layers with embedded piezoelectric transducer (PZT) sensor networks surf-mounted on the structures are employed to excite and receive Lamb wave signals since their superiority of flexibility, electrical stability and apt for complex structures. The results demonstrate that the proposed algorithm is capable of pinpointing both the location and size of the circular-hole damage in composite structures.

Adaptive Multiuser Cooperative NOMA Scheme With Energy Buffer-Aided Near-Users for High Spectral and Energy Efficiency

In this article, a multiuser cooperative nonorthogonal multiple access (NOMA) network is considered. In order to improve their battery lifetime, the near-user (NU) Internet of Things (IoT) nodes relay information to the far-user (FU) IoT nodes using only the energy harvested from the ambience. The harvested energy at all the NU IoT nodes is stored in energy buffers. To improve the throughput performance, the best NU and best FU (BNBF) user selection scheme is employed. Moreover, unlike conventional orthogonal multiple access (OMA) networks, an adaptive NOMA network is considered in this article that switches between direct NOMA, NOMA with relaying, and OMA modes to maximize throughput. To improve accuracy and reduce the computational complexity, the energy buffer states at the NU IoT nodes are modeled using a continuous state-space Markov chain (CSMC) instead of discrete state-space Markov chain (DSMC). We derive the limiting distributions of the stored energy in energy buffers with either the best effort policy (BEP) or the on–off policy (OOP) applied for buffer energy management. Expressions for throughput are derived for both harvest-store-use (HSU) and harvest-use (HU) architectures. The Monte Carlo simulations are used to validate the derived analytical expressions.

Lifting the Convex Conjugate in Lagrangian Relaxations: A Tractable Approach for Continuous Markov Random Fields

Dual decomposition approaches in nonconvex optimization may suffer from a duality gap. This poses a challenge when applying them directly to nonconvex problems such as MAP-inference in a Markov random field (MRF) with continuous state spaces. To eliminate such gaps, this paper considers a reformulation of the original nonconvex task in the space of measures. This infinite-dimensional reformulation is then approximated by a semi-infinite one, which is obtained via a piecewise polynomial discretization in the dual. We provide a geometric intuition behind the primal problem induced by the dual discretization and draw connections to optimization over moment spaces. In contrast to existing discretizations which suffer from a grid bias, we show that a piecewise polynomial discretization better preserves the continuous nature of our problem. Invoking results from optimal transport theory and convex algebraic geometry we reduce the semi-infinite program to a finite one and provide a practical implementation based on semidefinite programming. We show, experimentally and in theory, that the approach successfully reduces the duality gap. To showcase the scalability of our approach, we apply it to the stereo matching problem between two images.

Markovian Models for Microplastic Transport in Open‐Channel Flows

AbstractThe ubiquity of microplastics in marine environments is of growing concern and is increasingly receiving widespread attention. Due to the role of rivers and streams as suppliers of microplastics to the marine environment, it is essential to accurately capture their movements at these scales, but modeling and experimental knowledge in such settings is still limited. In this work, three Markov models, including a continuous time random walk model, Bernoulli model, and spatial Markov model (SMM), are implemented to investigate polyethylene particles transport in open‐channel flows. First, a three‐dimensional high‐resolution direct numerical simulation (DNS) fully resolves a canonical open‐channel flow, and particle transport is simulated using idealized point particles. Then, a series of laboratory transport experiments are conducted in a circulating water tank, and particle image velocimetry methods are used to obtain particle‐tracking data. We find that the correlated Bernoulli model and SMM can successfully reproduce the transport of both DNS and laboratory experiments, particularly in the prediction of measured breakthrough curves, which highlights the importance of correlation between the successive steps. A major benefit of these models is a computational cost that is several orders of magnitude less than, for example, DNS, which demonstrates their high‐efficiency and effectiveness. Therefore, this research offers new insights into the transport of microplastics in open‐channel systems like rivers and streams, which is necessary to prevent and reduce the environmental hazards of microplastics.

Application of Generation Adequacy Analysis for Reliability Evaluation of a Floating Production Storage and Offloading Platform Power System

This paper proposes the application of generation adequacy analysis for reliability evaluation of an insulated power generation system that supplies a FPSO (Floating Production Storage and Offloading) oil and gas production platform. The frequency and duration method was adopted for generating system reliability evaluation. The historical reliability data of the floating production storage and offloading platform power system and the continuous Markov process are used to determine the generator’s reliability model. The load model was also based on the platform daily peak load variation curve. The system risk indexes were obtained using Monte Carlo simulation. Two system scenarios were simulated using different failure data for one of the generators and the software PowerFactory© has been used as a tool for this simulation. For a complete generation system modeling, frequency and duration methods were developed to calculate the probabilities, frequencies and duration of the system states. Numerical results are presented and discussed to demonstrate the applicability of the proposed method.

Non-equilibrium thermodynamics of diffusion in fluctuating potentials

A positive rate of entropy production at steady-state is a distinctive feature of truly non-equilibrium processes. Exact results, while being often limited to simple models, offer a unique opportunity to explore the thermodynamic features of these processes in full detail. Here we derive analytical results for the steady-state rate of entropy production in single particle systems driven away from equilibrium by the fluctuations of an external potential of arbitrary shapes. Subsequently, we provide exact results for a diffusive particle in a harmonic trap whose potential stiffness varies in time according to both discrete and continuous Markov processes. In particular, studying the case of a fully intermittent potential allows us to introduce an effective model of stochastic resetting for which it is possible to obtain finite non-negative entropy production. Altogether, this work lays the foundation for a non-equilibrium thermodynamic theory of fluctuating potentials, with immediate applications to stochastic resetting processes, fluctuations in optical traps and fluctuating interactions in living systems.

Highway Lane-Changing Prediction Using a Hierarchical Software Architecture based on Support Vector Machine and Continuous Hidden Markov Model

Highway Lane-Changing Prediction Using a Hierarchical Software Architecture based on Support Vector Machine and Continuous Hidden Markov Model

1193-P: Natural History of Type 2 Diabetes in South Asians: Arrow of Time

Background: People of South Asian ancestry are at high risk for type 2 diabetes, but the time to disease progression in this population is not fully understood. Methods: We analyzed data from 2714 adults aged ≥20 years, enrolled in the Chennai site of the prospective Cardiometabolic risk reduction in South Asia (CARRS) study. Participants had 75g OGTT and HbA1c measurements. We defined normal glucose tolerance (NGT) , isolated impaired fasting glucose (iIFG) , impaired glucose tolerance (IGT) , and diabetes based on ADA criteria. We constructed continuous chain Markov models to estimate the transition through multiple states (NGT -> iIFG -> diabetes; NGT -> IGT -> diabetes) . The annual and five-year transition probability among states was estimated based on transition intensity. Results: Participants were, on average, 38.4 (range 20-89) years old, and 40.1% were male. Prevalence of iIFG and IGT was 16.9% and 7.3%, respectively. Over 5 years of follow up, the progression rates were: NGT to iIFG, 36.3 (95% CI: 32.5, 40.5) per 1,000 person-years (PYR) , NGT to IGT, 27.4 (95% CI: 24.1, 31.1) per 1,000 PYR, iIFG to diabetes 47.3 (95% CI: 40.4, 55.4) per 1,000 PYR, and IGT to diabetes 77.1 (95% CI: 65.1, 91.3) per 1,000 PYR. The estimated annual probability of remaining as NGT, iIFG, and IGT were 95.2% (94.7%, 95.8%) , 87.3% (85.5%, 89.0%) ;%) , and 81.9% (79.6%, 84.3%) %) , respectively. Assuming unidirectional transition, the annual probability of conversion from NGT to iIFG, and iIFG to diabetes were 4.5% (3.9%, 4.9%) , and 12.7% (10.9%, 14.5%) , respectively; and NGT to IGT, and IGT to diabetes were 3.8% (3.3%, 4.2%) and 18.1% (15.7%, 20.3%) . The estimated mean sojourn time (years) in each state were: NGT 20.5 (18.2, 23.2) , iIFG 7.4 (6.4, 8.6) , and IGT 5.0 (4.4, 5.8) , respectively. Conclusion: The course to diabetes in South Asians is rapid once prediabetes sets in, allowing a window of only 5-7.4 years to implement proven prevention. Innovative prevention efforts in normoglycemic people may be needed to prevent prediabetes in the first place. Disclosure K.Narayan: n/a. H.H.Chang: None. N.Tandon: None. V.Mohan: None. D.Kondal: None. L.R.Staimez: None. R.Anjana: None. U.Gujral: None. M.Deepa: None. S.A.Patel: Research Support; Johnson & Johnson. M.K.Ali: Advisory Panel; Bayer AG, Research Support; Merck & Co., Inc. D.Prabhakaran: None. Funding The National Heart, Lung, and Blood Institute (NHLBI) , National Institutes of Health (HHSN268200900026C)

A stationary Weibull process and its applications

In this paper we introduce a discrete-time and continuous state-space Markov stationary process , where has a two-parameter Weibull distribution, 's are dependent and there is a positive probability that . The motivation came from the gold price data where there are several instances for which . Hence, the existing methods cannot be used to analyze this data. We derive different properties of the proposed Weibull process. It is observed that the joint cumulative distribution function of and has a very convenient copula structure. Hence, different dependence properties and dependence measures can be obtained. The maximum likelihood estimators cannot be obtained in explicit forms, we have proposed a simple profile likelihood method to compute these estimators. We have used this model to analyze two synthetic data sets and one gold price data set of the Indian market, and it is observed that the proposed model fits quite well with the data set.

Perpetual American Double Lookback Options on Drawdowns and Drawups with Floating Strikes

We present closed-form solutions to the problems of pricing of the perpetual American double lookback put and call options on the maximum drawdown and the maximum drawup with floating strikes in the Black-Merton-Scholes model. It is shown that the optimal exercise times are the first times at which the underlying risky asset price process reaches some lower or upper stochastic boundaries depending on the current values of its running maximum or minimum as well as the maximum drawdown or maximum drawup. The proof is based on the reduction of the original double optimal stopping problems to the appropriate sequences of single optimal stopping problems for the three-dimensional continuous Markov processes. The latter problems are solved as the equivalent free-boundary problems by means of the smooth-fit and normal-reflection conditions for the value functions at the optimal stopping boundaries and the edges of the three-dimensional state spaces. We show that the optimal exercise boundaries are determined as either the unique solutions of the associated systems of arithmetic equations or the minimal and maximal solutions of the appropriate first-order nonlinear ordinary differential equations.

Optimal Double Stopping Problems for Maxima and Minima of Geometric Brownian Motions

We present closed-form solutions to some double optimal stopping problems with payoffs representing linear functions of the running maxima and minima of a geometric Brownian motion. It is shown that the optimal stopping times are th first times at which the underlying process reaches some lower or upper stochastic boundaries depending on the current values of its running maximum or minimum. The proof is based on the reduction of the original double optimal stopping problems to sequences of single optimal stopping problems for the resulting three-dimensional continuous Markov process. The latter problems are solved as the equivalent free-boundary problems by means of the smooth-fit and normal-reflection conditions for the value functions at the optimal stopping boundaries and the edges of the three-dimensional state space. We show that the optimal stopping boundaries are determined as the extremal solutions of the associated first-order nonlinear ordinary differential equations. The obtained results are related to the valuation of perpetual real double lookback options with floating sunk costs in the Black-Merton-Scholes model.

Multiple Sclerosis Progressive Courses: A Clinical Cohort Long-Term Disability Progression Study

ObjectivesImproving the understanding of multiple sclerosis (MS) mechanism and disability progression over time is essential to assess the value of healthcare interventions. Poor or no data on disability progression are available for progressive courses. This study aims to fill this gap. MethodsAn observational cohort study of patients with primary MS (PPMS) and secondary progressive MS (SPMS) was conducted on 2 Italian MS centers disease registries over an observational time of 34 years. Annual transition probabilities among Expanded Disability Status Scale (EDSS) states were estimated using continuous Markov models. A sensitivity analysis was performed in relation to clinical characteristic associated to disability progression. ResultsThe study cohort included 758 patients (274 PPMS and 434 SPMS) with a median follow-up of 8.2 years. Annual transition probability matrices of SPMS and PPMS reported different annual probabilities to move within EDSS levels. Excluding EDSS associated to relapse events or patient with relapses, the annual probability of staying stable in an EDSS level increased in both disease courses even not significantly. ConclusionsThis study provides estimates of annual disability progression as EDSS changes for PPMS and SPMS. These estimates could be a useful tool for healthcare decision makers and clinicians to properly assess impact of clinical interventions.

Continuous Hidden Markov Model Based Spectrum Sensing with Estimated SNR for Cognitive UAV Networks.

In this paper, to enhance the spectrum utilization in cognitive unmanned aerial vehicle networks (CUAVNs), we propose a cooperative spectrum sensing scheme based on a continuous hidden Markov model (CHMM) with a novel signal-to-noise ratio (SNR) estimation method. First, to exploit the Markov property in the spectrum state, we model the spectrum states and the corresponding fusion values as a hidden Markov model. A spectrum prediction is obtained by combining the parameters of CHMM and a preliminary sensing result (obtained from a clustered heterogeneous two-stage-fusion scheme), and this prediction can further guide the sensing detection procedure. Then, we analyze the detection performance of the proposed scheme by deriving its closed-formed expressions. Furthermore, considering imperfect SNR estimation in practical applications, we design a novel SNR estimation scheme which is inspired by the reconstruction of the signal on graphs to enhance the proposed CHMM-based sensing scheme with practical SNR estimation. Simulation results demonstrate the proposed CHMM-based cooperative spectrum sensing scheme outperforms the ones without CHMM, and the CHMM-based sensing scheme with the proposed SNR estimator can outperform the existing algorithm considerably.

Hidden Markov Model based Stochastic Resonance and its Application to Bearing Fault Diagnosis

Rolling bearings are crucial components in rotating machinery and the detection of damage at early stages is pivotal to ensure the life of the machine. Thus, developing accurate methods to identify the presence of faults captured using vibration measurements early on are necessary. Stochastic resonance (SR) is an interesting technique for bearing fault diagnosis, which by adding proper amount of noise to the signal the faulty signal can be extracted. However, traditional SR methods require that the analyst already know the characteristic frequency associated with the fault to be able to extract the corresponding weak signal. This paper overcomes this shortage by adaptively searching the SR system parameters by using Box-Cox sparse measures (BCSM) – based continuous hidden Markov model (CHMM) to determine the characteristic frequency directly from measured data. The method consists of a training phase that uses a simulated outer race fault to determine the optimal CHMM parameters and an application phase where the CHMM is used to determine the characteristic frequency directly from the unprocessed measured signal. The proposed method is verified using three well-known real bearing data sets and is found to be successful in all applications considered here.

Valuation of Annuity Guarantees Under a Self-Exciting Switching Jump Model

This article investigates the valuation of annuity guarantees under a regime-switching model when the dynamics of the underlying stock price follow a self-exciting switching jump-diffusion process. In this framework, we add a jump component to a regime-switching geometric Brownian for large shocks on the stock price. The intensity of shock arrivals is a Hawkes process modulated by a continuous time hidden Markov chain with a finite number of states. The interest rate used for discounting is stochastic and correlated to the stock market. In an incomplete market, we define an equivalent martingale measure to price a variable annuity contract that guarantees a minimum living or death benefit. Under this equivalent martingale measure, we propose closed-form approximation formulas using the inverse Fourier transform technique. A numerical implementation highlights the impact of self-exciting jumps and economic regimes on the valuation of guarantees.

Uncovering students’ problem-solving processes in game-based learning environments

As one of the most desired skills for contemporary education and career, problem-solving is fundamental and critical in game-based learning research. However, students' implicit and self-controlled learning processes in games make it difficult to understand their problem-solving behaviors. Observational and qualitative methods, such as interviews and exams, fail to capture students' in-process difficulties. By integrating data mining techniques, this study explored students' problem-solving processes in a puzzle-based game. First, we applied the Continuous Hidden Markov Model to identify students' problem-solving phases and the transition probabilities between these phases. Second, we employed sequence mining techniques to investigate problem-solving patterns and strategies facilitating students' problem-solving processes. The results suggested that most students were stuck in certain phases, with only a few able to transfer to systematic phases by applying efficient strategies. At the beginning of the puzzle, the most popular strategy was testing one dimension of the solution at each attempt. In contrast, the other two strategies (remove or add untested dimensions one by one) played pivotal roles in promoting transitions to higher problem-solving phases. The findings of this study shed light on when, how, and why students advanced their effective problem-solving processes. Using the Continuous Hidden Markov Model and sequence mining techniques, we provide considerable promise for uncovering students' problem-solving processes, which helps trigger future scaffolds and interventions to support students’ personalized learning in game-based learning environments.

How patients with multiple sclerosis acquire disability.

Patients with multiple sclerosis acquire disability either through relapse-associated worsening (RAW) or progression independent of relapse activity (PIRA). This study addresses the relative contribution of relapses to disability worsening over the course of the disease, how early progression begins and the extent to which multiple sclerosis therapies delay disability accumulation.Using the Novartis-Oxford multiple sclerosis (NO.MS) data pool spanning all multiple sclerosis phenotypes and paediatric multiple sclerosis, we evaluated ∼200 000 Expanded Disability Status Scale (EDSS) transitions from >27 000 patients with ≤15 years follow-up. We analysed three datasets: (i) A full analysis dataset containing all observational and randomized controlled clinical trials in which disability and relapses were assessed (n = 27 328); (ii) all phase 3 clinical trials (n = 8346); and (iii) all placebo-controlled phase 3 clinical trials (n = 4970). We determined the relative importance of RAW and PIRA, investigated the role of relapses on all-cause disability worsening using Andersen-Gill models and observed the impact of the mechanism of worsening and disease-modifying therapies on the time to reach milestone disability levels using time continuous Markov models.PIRA started early in the disease process, occurred in all phenotypes and became the principal driver of disability accumulation in the progressive phase of the disease. Relapses significantly increased the hazard of all-cause disability worsening events; following a year in which relapses occurred (versus a year without relapses), the hazard increased by 31–48% (all P < 0.001). Pre-existing disability and older age were the principal risk factors for incomplete relapse recovery. For placebo-treated patients with minimal disability (EDSS 1), it took 8.95 years until increased limitation in walking ability (EDSS 4) and 18.48 years to require walking assistance (EDSS 6). Treating patients with disease-modifying therapies delayed these times significantly by 3.51 years (95% confidence limit: 3.19, 3.96) and 3.09 years (2.60, 3.72), respectively. In patients with relapsing-remitting multiple sclerosis, those who worsened exclusively due to RAW events took a similar length of time to reach milestone EDSS values compared with those with PIRA events; the fastest transitions were observed in patients with PIRA and superimposed relapses.Our data confirm that relapses contribute to the accumulation of disability, primarily early in multiple sclerosis. PIRA begins in relapsing-remitting multiple sclerosis and becomes the dominant driver of disability accumulation as the disease evolves. Pre-existing disability and older age are the principal risk factors for further disability accumulation. The use of disease-modifying therapies delays disability accrual by years, with the potential to gain time being highest in the earliest stages of multiple sclerosis.