Continuous-time semi-Markov Chain Research Articles

On the basic reproduction number in semi-Markov switching networks

ABSTRACT Basic reproduction number in network epidemic dynamics is studied in the case of stochastic regime-switching networks. For generality, the dependence between successive networks is considered to follow a continuous time semi-Markov chain. is the weighted average of the basic reproduction numbers of deterministic subnetworks. Its position with respect to 1 can determine epidemic persistence or extinction in theories and simulations.

Modeling ageing effects for multi-state systems with multiple components subject to competing and dependent failure processes

This paper explores how the ageing process affects the two dependent and competing failure processes in the context of multi-state systems with multiple components. Ageing process is assumed to result in the decline of the product's intrinsic capacity to resist the exposed stresses. The state residence time is explicitly modeled with the consideration of the potential transition gap (the total number of states that the state transition process needs to stride across) and the cumulative ageing probability which is a measure of the likelihood that the product strength degrades. The random shock is assumed to cause abrupt changes in the one-step transition probabilities. The damage size is jointly determined by the shock magnitude and the cumulative ageing probability. Reliability analysis is conducted based on the continuous-time semi-Markov chain. Reliability functions for the degradation, abrupt failure, and competing failure processes are obtained respectively. An illustrative example of a multi-state transformer with four multi-state components is studied to demonstrate how the proposed method can be applied to the engineering practice.

The Semimartingale Dynamics and Generator of a Continuous Time Semi-Markov Chain

The Semimartingale Dynamics and Generator of a Continuous Time Semi-Markov Chain

Condition-based maintenance for long-life assets with exposure to operational and environmental risks

This paper presents a new condition-based maintenance (CBM) model for long-life assets to address the potential risk caused by the decline of the operating environment. Two types of maintenance are formulated in the CBM model. Minor maintenance can mitigate the operational and environmental risk, and major maintenance can eliminate the accumulated damage within the asset. A continuous-time semi-Markov chain (CTSMC) is used for modeling the aging of the asset as well as the stochastic decline of the operating environment. To optimize the CBM policy in a mathematically tractable manner, we introduce a hypo-exponential approximation approach to match the first four moments of the sojourn time distribution of CTSMC. This approach guarantees a minimum representation of the CTSMC with non-fictitious surrogated Markov chain. The model provides both good mathematical tractability and sufficient generalizability. The practical impact of this research is demonstrated by applying it to a real industrial case of concrete bridge maintenance. It is observed that this approach results in a CBM plan with a lower asset lifecycle cost compared to current techniques.

Optimal Bayesian early fault detection for CNC equipment using hidden semi-Markov process

Abstract The goal of a computer numerically controlled (CNC) equipment is to guarantee high specified performance and to maintain it effectively over its life cycle time. Thus, the assessment of health condition is crucial for industrial systems. However, very few papers have dealt with the cost-optimal early fault detection and remaining useful life prediction of CNC equipment using multivariate positioning data. The novel approach presented here is based on vector autoregressive (VAR) degradation modeling and hidden semi-Markov modeling using the optimal Bayesian control technique. System condition is modeled using a continuous time semi-Markov chain with three states, i.e. unobservable healthy state 1, unobservable warning state 2 and observable failure state 3. Model parameter estimates are calculated using the expectation-maximization (EM) algorithm. The optimal control policy for the three-state model is represented by a Bayesian control chart for a multivariate observation process. The optimization problem is formulated and solved in the semi-Markov decision process (SMDP) framework. A formula for the mean residual life (MRL) is also derived based on Bayesian approach, which enables the estimation of the remaining useful life and early maintenance planning based on the observed data. The validation of the proposed methodologies is carried out using actual multivariate degradation data obtained from a CNC equipment. A comparison with the multivariate Bayesian control chart based on a hidden Markov model (HMM) is given, which illustrates the effectiveness of the proposed approach.

Opportunistic Capacity-Based Resource Allocation for Chunk-Based Multi-Carrier Cognitive Radio Sensor Networks.

The spectrum allocation for cognitive radio sensor networks (CRSNs) has received considerable research attention under the assumption that the spectrum environment is static. However, in practice, the spectrum environment varies over time due to primary user/secondary user (PU/SU) activity and mobility, resulting in time-varied spectrum resources. This paper studies resource allocation for chunk-based multi-carrier CRSNs with time-varied spectrum resources. We present a novel opportunistic capacity model through a continuous time semi-Markov chain (CTSMC) to describe the time-varied spectrum resources of chunks and, based on this, a joint power and chunk allocation model by considering the opportunistically available capacity of chunks is proposed. To reduce the computational complexity, we split this model into two sub-problems and solve them via the Lagrangian dual method. Simulation results illustrate that the proposed opportunistic capacity-based resource allocation algorithm can achieve better performance compared with traditional algorithms when the spectrum environment is time-varied.

An adaptive mode control algorithm of a scalable intrusion tolerant architecture

In this paper we consider an intrusion tolerant system with two detection modes; automatic detection mode and manual detection mode for intrusions, which is called SITAR (Scalable Intrusion Tolerant Architecture), and describe the dynamic transition behavior by a continuous-time semi-Markov chain (CTSMC). Based on the embedded Markov chain (EMC) approach, we derive the steady-state probability of the CTSMC, the steady-state system availability and the mean time to security failure (MTTSF). Especially, we show necessary and sufficient conditions to exist the optimal switching time from an automatic detection mode to a manual detection mode, which maximizes the steady-state system availability. Next, we develop an adaptive mode control scheme to estimate the optimal switching time without specifying the associated probability distribution function, whose idea behind is based on a statistically non-parametric algorithm by means of the total time on test concept. Numerical examples through a comprehensive simulation study are presented for illustrating the optimal switching of detection mode, and investigating the asymptotic property of the resulting adaptive mode control scheme.

Busy periods in fluid queues with multiple emptying input states

A semi-numerical method is derived to compute the Laplace transform of the equilibrium busy period probability density function in a fluid queue with constant output rate when the buffer is nonempty. The input process is controlled by a continuous-time semi-Markov chain (CTSMC) with n states such that in each state the input rate is constant. The holding time in states with net positive output rate - so-called emptying states - is assumed to be an exponentially distributed random variable, whereas in states with net positive input rate - so-called filling states - it may have an arbitrary probability distribution. The result is demonstrated by applying it to various systems, including fluid queues with two on-off input sources. The latter exercise in part shows consistency with prior results but also solves the problem in the case where there are two emptying states. Numerical results are presented for selected examples which expose discontinuities in the busy period distribution when the number of emptying states changes, e.g. as a result of increasing the fluid arrival rate in one or more states of the controlling CTSMC.

Busy periods in fluid queues with multiple emptying input states

A semi-numerical method is derived to compute the Laplace transform of the equilibrium busy period probability density function in a fluid queue with constant output rate when the buffer is nonempty. The input process is controlled by a continuous-time semi-Markov chain (CTSMC) with n states such that in each state the input rate is constant. The holding time in states with net positive output rate - so-called emptying states - is assumed to be an exponentially distributed random variable, whereas in states with net positive input rate - so-called filling states - it may have an arbitrary probability distribution. The result is demonstrated by applying it to various systems, including fluid queues with two on-off input sources. The latter exercise in part shows consistency with prior results but also solves the problem in the case where there are two emptying states. Numerical results are presented for selected examples which expose discontinuities in the busy period distribution when the number of emptying states changes, e.g. as a result of increasing the fluid arrival rate in one or more states of the controlling CTSMC.