Continuous-time semi-Markov Process Research Articles

ROCOF of higher order for semi-Markov processes

In this paper, we study the rate of occurrence of failures (ROCOF) of higher order for continuous time semi-Markov processes (SMP). This indicator gives information on whether there are a lot of failures or only a few within a time interval. It also considers the relative positioning of tuples of failures in time. Here, we consider SMP with a mixed probability distribution for the initial law of the system taking into account the possible random starting from any state with any duration. Furthermore, under suitable assumptions on the transition rates, we determine an explicit formula for the ROCOF of higher order and we recover as particular cases previous results obtained in the literature. A numerical example demonstrates the possibility of using this index in real applications. A possible application in the field of mathematical finance is also discussed.

Defining state-transition diagrams as part of technical state monitoring

State-transition diagrams are given for the situation of monitoring that takes into consideration explicit and hidden failures.Aim. To generate state-transition diagrams that are used in the development of the dependability model and associated calculation methods under conditions of periodic monitoring.Methods. The method is based on the classification of failures by detection, i.e., explicit and hidden failures. Accordingly, facility condition inspection may be continuous and/or periodical. Periodical inspection is conducted at equal intervals. The state-transition diagram is generated taking into consideration the relations between the states and events, i.e., each state can be the cause of a certain event and, at the same time, a state is the consequence of an event. Similarly, each event is the cause of a state change, while at the same time being a consequence of a state. Within the time between two periodic inspections, continuous-time transitions take place due to failures. This process is described by the theory of continuous-time Markov processes. Upon the completion of a periodic inspection operation, the up state ensues if an operable item is subjected to inspection. If an item proves to be down, it is submitted to maintenance. Such transitions in discrete moments in time are described by semi-Markov processes. For a better understanding of the information set forth in the paper, a list is provided of the used terminology referring to state standards.Results. Under the adopted failure modes within a single inspection period, state transitions are described in continuous time, while after the inspection operation they are described in discrete time. Within each inspection period, a system of differential equations is defined and solved based on the assumption of initial up state. Based on the deduced probabilities, the probabilities of a period without failures and with a failure are calculated along with the mean time of up and down states within the period with a failure. Should failures be detected, maintenance state is initiated. The probability of such transition is within the theory of semi-Markov processes. Based on the probabilities of the semi-Markov process, the mean failure-free periods are calculated. Such model adequately reflects the dependability processes of operated equipment. If the method is not fully observed, the dependability indicators may take significantly different values. Three examples of state-transition diagrams are given: taking into account only explicit failures, only hidden failures and both.Discussion and conclusions. The presented approach with a constant (regular) inspection period allows building models that adequately reflect processes occurring in actual systems. All operations involved in the implementation of simulated processes are performed based on the theory of continuous-time Markov processes and semi-Markov processes. The operations are performed in matrix form. Such method enables mathematical operations using computers. The presented approach can be used for improving dependability models of technical systems.

On the asymptotic properties of some kernel estimators for continuous-time semi-Markov processes

This paper provides kernel estimators of the main characteristics of a continuous-time semi-Markov process, like conditional and unconditional sojourn times in a state, semi-Markov kernel, etc. The main goal of this paper is to establish asymptotic properties of the semi-Markov kernel estimators and of the sojourn time distribution estimators (conditional and unconditional), as well as of the estimators of the associated Radon-Nikodym derivatives, when the sample size becomes large. The approach is illustrated by considering a three state example with detailed calculus and numerical evaluations.

Predictive Maintenance and Sensitivity Analysis for Equipment with Multiple Quality States

This paper discusses the predictive maintenance (PM) problem of a single equipment system. It is assumed that the equipment has deteriorating quality states as it operates, resulting in multiple yield levels represented as system observation states. We cast the equipment deterioration as discrete-state and continuous-time semi-Markov decision process (SMDP) model and solve the SMDP problem in reinforcement learning (RL) framework using the strategy-based method. In doing so, the goal is to maximize the system average reward rate (SARR) and generate the optimal maintenance strategy for given observation states. Further, the PM time is capable of being produced by a simulation method. In order to prove the advantage of our proposed method, we introduce the standard sequential preventive maintenance algorithm with unequal time interval. Our proposed method is compared with the sequential preventive maintenance algorithm in a test objective of SARR, and the results tell us that our proposed method can outperform the sequential preventive maintenance algorithm. In the end, the sensitivity analysis of some parameters on the PM time is given.

Using Semi-Markov Chains to Solve Semi-Markov Processes

This article provides a novel method to solve continuous-time semi-Markov processes by algorithms from discrete-time case, based on the fact that the Markov renewal function in discrete-time case is a finite series. Bounds of approximate errors due to discretization for the transition function matrix of the continuous-time semi-Markov process are investigated. This method is applied to a reliability problem which refers to the availability analysis of the system subject to sequential cyber-attacks. Two cases where sojourn times follow exponential and Weibull distributions are considered and computed in order to verify and illustrate the proposed method.

On discrete-time semi-Markov processes

In the last years, several authors studied a class of continuous-time semi-Markov processes obtained by time-changing Markov processes by hitting times of independent subordinators. Such processes are governed by integro-differential convolution equations of generalized fractional type. The aim of this paper is to develop a discrete-time counterpart of such a theory and to show relationships and differences with respect to the continuous time case. We present a class of discrete-time semi-Markov chains which can be constructed as time-changed Markov chains and we obtain the related governing convolution type equations. Such processes converge weakly to those in continuous time under suitable scaling limits.

Industrial equipment reliability estimation: A Bayesian Weibull regression model with covariate selection

A three-state continuous-time semi-Markov process is used to model the degradation of an industrial equipment. The transition times are assumed Weibull-distributed and influenced by a set of covariates. A Weibull Regression Model is developed within the Bayesian probability framework, to account for the influence of these covariates and estimate the model parameters with the related uncertainty, on the basis of few data and expert judgment. The number of covariates is reduced by a two-step selection procedure derived from the condition monitoring engineering practice. The developed model enables estimating reliability and time-dependent state probabilities for a component degrading in given operational and ambient conditions, represented by a vector of covariates. The model is illustrated by way of a real case study concerning the degradation process affecting diaphragm valves used in the biopharmaceutical industry.

Extended dissipative synchronization for singularly perturbed semi-Markov jump neural networks with randomly occurring uncertainties

This paper concentrates on the synchronization problem for singularly perturbed neural networks with semi-Markov jump parameters and randomly occurring uncertainties. A continuous-time semi-Markov process is utilized to model the stochastic switching of the parameters. An independent singularly perturbed parameter is separated through the use of singularly perturbed slow-fast decomposition method. Some sufficient conditions are deduced to ensure that the error system is synchronized and meets the extended dissipative property. In particular, the uncertainty of the networks is considered to occur randomly, which is more realistic than the existing work. Moreover, the efficiency of the presented method is demonstrated by a numerical example.

Semi-additive functionals of semi-Markov processes and measure-valued Poisson equation

In this paper, we consider a continuous-time semi-Markov process (SMP) in Polish spaces. We first introduce the semi-additive functional in semi-Markov cases, a natural generalization of the additive functional of Markov process (MP). The main contribution of this paper is the properties of semi-additive functional aforementioned. First, semi-additive functionals of SMPs are characterized in terms of a càdlàg function with zero initial value and a measurable function. Second, the necessary and sufficient conditions are investigated under which a semi-additive functional of SMP is a semimartingale, a local martingale, or a special semimartingale respectively. By the way, the Itô type formula is given. Finally, we study the expected cumulative discounted value of the semi-additive functional of an SMP. We prove that it solves a measure-valued Poisson equation and give the uniqueness conditions.

Kemeny's function for Markov chains and Markov renewal processes

Extensions of Kemeny's constant, as derived for irreducible finite Markov chains in discrete-time, to Markov renewal processes and Markov chains in continuous-time are discussed. Three alternative Kemeny's functions and their variants are considered. Typically, they lead to a constant if and only if the mean holding times between the states in the Markov renewal process are constant. However one particular variant, that arises under expected time to hitting of a state chosen according the continuous-time stationary distribution, leads to a constant, analogous to the discrete-time Markov chain result. Specifically, if the state space S is finite, the sum, j∈S of each mean first passage time mij (omitting the mean return time mjj when i=j) weighted by the stationary probability ϖj associated with the continuous-time semi-Markov process, is a constant, independent of i, for any Markov Renewal Process. Expressions for the Kemeny's functions and the relevant constants are derived for Markov Renewal Processes and special cases involving continuous-time Markov chains and Birth–Death Processes.

Features of Load and Wear of Main Propulsion Devices on Sea-Going Ships with Piston Combustion Engines and Their Impact on Changes in Technical States of the Systems

Abstract The paper presents the specificity of operation of propulsion systems of seagoing ships which causes the need to control the load on them, especially on their engines called main engines. The characteristics of the load on the propulsion systems, especially on the main engines as well as on the shaft lines and propellers driven by the engines, along with the process of wear in tribological joints (sliding tribological systems) of the machines have been described herein. Using examples of typical types of wear (both linear and volumetric) for the tribological systems of this sort, interpretation of states of their wear has been provided with regards to the wear levels defined as acceptable, unacceptable and catastrophic. The following hypotheses have been formulated: 1) hypothesis explaining necessity to consider the loads on the systems under operation as stochastic processes; 2) hypothesis explaining a possibility of considering the processes as stationary; and 3) hypothesis explaining why it can be assumed that a given technical state of any tribological system can be considered as dependent only on the directly preceding state and stochastically independent of the states that existed earlier. Accepting the hypotheses as true, a four-state continuous-time semi-Markov process has been proposed in the form of a model of changes in condition of a propulsion system (PS) of any ship. The model includes the most significant states affecting safety of a ship at sea, such as: s0 - PS ability state, s1 - PS disability state due to damage to the main engine (ME), s2 - PS disability state due to damage to the shaft line (SL) and s3 - PS disability state due to damage to the propeller (P). Probability of occurrence (changes) of the states has also been demonstrated.

Data-Driven Risk Assessment from Small Scale Epidemics: Estimation and Model Choice for Spatio-Temporal Data with Application to a Classical Swine Fever Outbreak.

Livestock epidemics have the potential to give rise to significant economic, welfare, and social costs. Incursions of emerging and re-emerging pathogens may lead to small and repeated outbreaks. Analysis of the resulting data is statistically challenging but can inform disease preparedness reducing potential future losses. We present a framework for spatial risk assessment of disease incursions based on data from small localized historic outbreaks. We focus on between-farm spread of livestock pathogens and illustrate our methods by application to data on the small outbreak of Classical Swine Fever (CSF) that occurred in 2000 in East Anglia, UK. We apply models based on continuous time semi-Markov processes, using data-augmentation Markov Chain Monte Carlo techniques within a Bayesian framework to infer disease dynamics and detection from incompletely observed outbreaks. The spatial transmission kernel describing pathogen spread between farms, and the distribution of times between infection and detection, is estimated alongside unobserved exposure times. Our results demonstrate inference is reliable even for relatively small outbreaks when the data-generating model is known. However, associated risk assessments depend strongly on the form of the fitted transmission kernel. Therefore, for real applications, methods are needed to select the most appropriate model in light of the data. We assess standard Deviance Information Criteria (DIC) model selection tools and recently introduced latent residual methods of model assessment, in selecting the functional form of the spatial transmission kernel. These methods are applied to the CSF data, and tested in simulated scenarios which represent field data, but assume the data generation mechanism is known. Analysis of simulated scenarios shows that latent residual methods enable reliable selection of the transmission kernel even for small outbreaks whereas the DIC is less reliable. Moreover, compared with DIC, model choice based on latent residual assessment correlated better with predicted risk.

Nonparametric Estimation for Non-Homogeneous Semi-Markov Processes: An Application to Credit Risk

We propose procedures for estimating the time-dependent transition matrices for the general class of finite nonhomogeneous continuous-time semi-Markov processes. We prove the existence and uniqueness of solutions for the system of Volterra integral equations defining the transition matrices, therefore showing that these empirical transition probabilities can be estimated from window censored event-history data. An implementation of the method is presented based on nonparametric estimators of the hazard rate functions in the general and separable cases. A Monte Carlo study is performed to assess the small sample behavior of the resulting estimators. We use these new estimators for dealing with a central issue in credit risk. We consider the problem of obtaining estimates of the historical corporate default and rating migration probabilities using a dataset on credit ratings from Standard & Poor's.

Development of a Bayesian multi-state degradation model for up-to-date reliability estimations of working industrial components

We consider a three-state continuous-time semi-Markov process with Weibull-distributed transition times to model the degradation mechanism of an industrial equipment. To build this model, an original combination of techniques is proposed for building a semi-Markov degradation model based on expert knowledge and few field data within the Bayesian statistical framework. The issues addressed are: i) the prior elicitation of the model parameters values from experts, avoiding possible information commitment; ii) the development of a Markov-Chain Monte Carlo algorithm for sampling from the posterior distribution; iii) the posterior inference of the model parameters values and, on this basis, the estimation of the time-dependent state probabilities and the prediction of the equipment remaining useful life. The developed Bayesian model offers the possibility of updating the system reliability estimation every time a new evidence is gathered. The application of the modeling framework is illustrated by way of a real industrial case study concerning the degradation of diaphragms installed in a production line of a biopharmaceutical industry.

Semi-Markov Model for the Oxidation Degradation Mechanism in Gas Turbine Nozzles

This paper presents the statistical characterization of the oxidation degradation mechanism affecting the nozzles of turbines operated in Oil&Gas utilities. The degradation mechanism is modeled as a four-state, continuous-time semi-Markov process with Weibull distributed transition times. Maximum likelihood estimation is used to infer the parameters of the model from an available set of field data, whereas a numerical approach to estimate the Fisher information matrix is used to characterize the uncertainty in the estimates. The estimates obtained are, then, utilized to compute the probabilities of occupying the four degradation states over time and the corresponding uncertainties. A case study is shown, dealing with real field data.

Analysis of System Reliability with Control, Dependent Failures, and Arbitrary Repair Times

This work is motivated by modelling of real information systems. Parallel and series reliability models or their combinations are usually used for these tasks. Common assumptions for such models are independent failures, exponentially distributed failures and recoveries. These assumptions simplify a system modelling significantly, but often give a very rude approximation for it. So there are a lot of restrictions for an application of these models to practical tasks. This study presents a system with more general assumptions: dependent failures, arbitrary failures and repairs, and a system with control. We apply a continuous-time semi-Markov process to evaluate the reliability and the mean time to system failure (MTTF) for a system under these assumption. The repair time of each component is assumed to have an arbitrary distribution function (e.g., Weibull, Poisson or exponential). Kolmogorov equations method and the Laplace transform are used to derive generalised expressions for system state probabilities, reliability and MTTF. A numerical example is presented in order to illustrate the performance analysis of the model.

Reliability of maintained systems under a semi-Markov setting

A semi-Markov setting is considered in order to study the main dependability measures of a repairable continuous time system under the hypothesis that the evolution in time of its components is described by a continuous time semi-Markov process. Moreover, the main dependability measures of a periodically maintained system are studied. Finally, all the above systems are compared with the corresponding Markov systems where the general repair time distribution is replaced by the exponential distribution with the same mean which is the most commonly used approximation of the original system in practice.

Ruin Probabilities in a Finite-Horizon Risk Model with Investment and Reinsurance

A finite-horizon insurance model is studied where the risk/reserve process can be controlled by reinsurance and investment in the financial market. Our setting is innovative in the sense that we describe in a unified way the timing of the events, that is, the arrivals of claims and the changes of the prices in the financial market, by means of a continuous-time semi-Markov process which appears to be more realistic than, say, classical diffusion-based models. Obtaining explicit optimal solutions for the minimizing ruin probability is a difficult task. Therefore we derive a specific methodology, based on recursive relations for the ruin probability, to obtain a reinsurance and investment policy that minimizes an exponential bound (Lundberg-type bound) on the ruin probability.

Ruin Probabilities in a Finite-Horizon Risk Model with Investment and Reinsurance

A finite-horizon insurance model is studied where the risk/reserve process can be controlled by reinsurance and investment in the financial market. Our setting is innovative in the sense that we describe in a unified way the timing of the events, that is, the arrivals of claims and the changes of the prices in the financial market, by means of a continuous-time semi-Markov process which appears to be more realistic than, say, classical diffusion-based models. Obtaining explicit optimal solutions for the minimizing ruin probability is a difficult task. Therefore we derive a specific methodology, based on recursive relations for the ruin probability, to obtain a reinsurance and investment policy that minimizes an exponential bound (Lundberg-type bound) on the ruin probability.

Monounireducible Nonhomogeneous Continuous Time Semi-Markov Processes Applied to Rating Migration Models

Monounireducible nonhomogeneous semi- Markov processes are defined and investigated. The mono- unireducible topological structure is a sufficient condition that guarantees the absorption of the semi-Markov process in a state of the process. This situation is of fundamental importance in the modelling of credit rating migrations because permits the derivation of the distribution function of the time of default. An application in credit rating modelling is given in order to illustrate the results.