Markov Renewal Equation Research Articles

A comparative study of numerical methods for reliability assessment based on semi-Markov processes

The semi-Markov process, renowned for its versatile applications, has garnered significant attention in recent years. However, deriving closed-form expressions for computing reliability metrics proves challenging when sojourn times deviate from exponential distributions. This paper investigates three numerical techniques for computing the transition function matrix of the semi-Markov process, including the algebraic method, the truncated method, and the iterative method. It delves into their truncation and discretization errors, as well as their computational complexities. Additionally, the Laplace-based method and the semi-Markov-chain-based method are discussed to contrast their effectiveness with the three devised numerical approaches. Building on the linkage between the Markov renewal equation and system reliability metrics, five computational methods are applied to the evaluation of system reliability and availability. A case study on sequential cyber-attacks is presented to illustrate the applicability of these methods, considering sojourn times that follow exponential, gamma, lognormal, and Weibull distributions respectively. The results reveal that the three proposed numerical methods not only achieve high precision and rapid speed but also address scenarios beyond the capability of the Laplace-based method.

Maintenance policy optimization for a cold standby system considering multiple failure types

Purpose The purpose of this paper is to develop novel preventive maintenance (PM) modeling methods for a cold standby system subject to two types of failures: random failure and deterioration failure. Design/methodology/approach The system consists of two components and a single repair shop, assuming that the repair shop can only service for one component at a time. Based on semi-Markov theory, transition probabilities between all possible system states are discussed. With the transition probabilities, Markov renewal equations are established at regenerative points. By solving the Markov regenerative equations, the mean time from the initial state to system failure (MTSF) and the steady state availability (SSA) are formulated as two reliability measures for different reliability requirements of systems. The optimal PM policies are obtained when MTSF and SSA are maximized. Findings The result of simulation experiments verifies that the derived maintenance models are effective. Sensitivity analysis revealed the significant influencing factors for optimal PM policy for cold standby systems when different system reliability indexes (i.e. MTSF and SSA) are considered. Furthermore, the results show that the repair for random failure has a tremendous impact on prolonging the MTSF of cold standby system and PM plays a greater role in promoting the system availability of a cold standby system than it does in prolonging the MTSF of system. Practical implications In practical situations, system not only suffers normal deterioration caused by internal factors, but also undergoes random failures influenced by random shocks. Therefore, multiple failure types are needed to be considered in maintenance modeling. The result of the sensitivity analysis has an instructional role in making maintenance decisions by different system reliability indexes (i.e. MTSF and SSA). Originality/value This paper presents novel PM modeling methods for a cold standby system subject to two types of failures: random failure and deterioration failure. The sensitivity analysis identifies the significant influencing factors for optimal maintenance policy by different system reliability indexes which are useful for the managers for further decision making.

Research of reliability and efficiency of technological processes of mechanical assembly production on the basis of the common semi-Markov model

In the article a common semi-Markov mathematical model is considered that allows one to investigate the productivity and reliability of various technological processes of mechanical assembly production. The proposed model allows to study, inter alia, technological processes of manufacturing parts with screw and assemblies of threaded connections. Mathematical apparatus of the research is the theory of semi-Markov processes with a common phase space, which operates with a common kind of random variables distribution functions. If the considering process in the system is a subsystem located on a higher level of hierarchy, the hierarchical model for compatibility with each other levels as output simulation parameters required distribution functions. In the proposed model, based on the decision of the Markov renewal equations depend not only on the torque characteristics, but also the distribution function of time per unit of output service according to different kinds of undervalued failures.

On the Existence and Uniqueness of Solution of MRE and Applications

In this paper, we study the existence and uniqueness of the solution for Markov renewal equation (MRE) of a semi-Markov process with countable state space. This method and its proof are based on an iterative scheme. A numerical solution is also given as well as a case study on system reliability assessment.

The roles of coupling and the deviation matrix in determining the value of capacity in M/M/1/C queues

In an M/M/1/C queue, customers are lost when they arrive to find C customers already present. Assuming that each arriving customer brings a certain amount of revenue, we are interested in calculating the value of an extra waiting place in terms of the expected amount of extra revenue that the queue will earn over a finite time horizon [0, t]. There are different ways of approaching this problem. One involves the derivation of Markov renewal equations, conditioning on the first instance at which the state of the queue changes; a second involves an elegant coupling argument; and a third involves expressing the value of capacity in terms of the entries of a transient analogue of the deviation matrix. In this paper, we shall compare and contrast these approaches and, in particular, use the coupling analysis to explain why the selling price of an extra unit of capacity remains the same when the arrival and service rates are interchanged when the queue starts at full capacity.

Quasistochastic matrices and Markov renewal theory

Let 𝓈 be a finite or countable set. Given a matrix F = (Fij)i,j∈𝓈 of distribution functions on R and a quasistochastic matrix Q = (qij)i,j∈𝓈, i.e. an irreducible nonnegative matrix with maximal eigenvalue 1 and associated unique (modulo scaling) positive left and right eigenvectors u and v, the matrix renewal measure ∑n≥0Qn ⊗ F*n associated with Q ⊗ F := (qijFij)i,j∈𝓈 (see below for precise definitions) and a related Markov renewal equation are studied. This was done earlier by de Saporta (2003) and Sgibnev (2006, 2010) by drawing on potential theory, matrix-analytic methods, and Wiener-Hopf techniques. In this paper we describe a probabilistic approach which is quite different and starts from the observation that Q ⊗ F becomes an ordinary semi-Markov matrix after a harmonic transform. This allows us to relate Q ⊗ F to a Markov random walk {(Mn, Sn)}n≥0 with discrete recurrent driving chain {Mn}n≥0. It is then shown that renewal theorems including a Choquet-Deny-type lemma may be easily established by resorting to standard renewal theory for ordinary random walks. The paper concludes with two typical examples.

Quasistochastic matrices and Markov renewal theory

Let 𝓈 be a finite or countable set. Given a matrix F = (F ij ) i,j∈𝓈 of distribution functions on R and a quasistochastic matrix Q = (q ij ) i,j∈𝓈 , i.e. an irreducible nonnegative matrix with maximal eigenvalue 1 and associated unique (modulo scaling) positive left and right eigenvectors u and v, the matrix renewal measure ∑ n≥0 Q n ⊗ F *n associated with Q ⊗ F := (q ij F ij ) i,j∈𝓈 (see below for precise definitions) and a related Markov renewal equation are studied. This was done earlier by de Saporta (2003) and Sgibnev (2006, 2010) by drawing on potential theory, matrix-analytic methods, and Wiener-Hopf techniques. In this paper we describe a probabilistic approach which is quite different and starts from the observation that Q ⊗ F becomes an ordinary semi-Markov matrix after a harmonic transform. This allows us to relate Q ⊗ F to a Markov random walk {(M n , S n )} n≥0 with discrete recurrent driving chain {M n } n≥0. It is then shown that renewal theorems including a Choquet-Deny-type lemma may be easily established by resorting to standard renewal theory for ordinary random walks. The paper concludes with two typical examples.

The Markov Additive Risk Process Under an Erlangized Dividend Barrier Strategy

In this paper, we consider a Markov additive insurance risk process under a randomized dividend strategy in the spirit of Albrecher et al. (2011). Decisions on whether to pay dividends are only made at a sequence of dividend decision time points whose intervals are Erlang(n) distributed. At a dividend decision time, if the surplus level is larger than a predetermined dividend barrier, then the excess is paid as a dividend as long as ruin has not occurred. In contrast to Albrecher et al. (2011), it is assumed that the event of ruin is monitored continuously (Avanzi et al. (2013) and Zhang (2014)), i.e. the surplus process is stopped immediately once it drops below zero. The quantities of our interest include the Gerber-Shiu expected discounted penalty function and the expected present value of dividends paid until ruin. Solutions are derived with the use of Markov renewal equations. Numerical examples are given, and the optimal dividend barrier is identified in some cases.

On the Modelling of Imperfect Repairs for a Continuously Monitored Gamma Wear Process Through Age Reduction

A continuously monitored system is considered, which is subject to accumulating deterioration modelled as a gamma process. The system fails when its degradation level exceeds a limit threshold. At failure, a delayed replacement is performed. To shorten the down period, a condition-based maintenance strategy is applied, with imperfect repair. Mimicking virtual age models used for recurrent events, imperfect repair actions are assumed to lower the system degradation through a first-order arithmetic reduction of age model. Under these assumptions, Markov renewal equations are obtained for several reliability indicators. Numerical examples illustrate the behaviour of the system.

On the Modelling of Imperfect Repairs for a Continuously Monitored Gamma Wear Process Through Age Reduction

A continuously monitored system is considered, which is subject to accumulating deterioration modelled as a gamma process. The system fails when its degradation level exceeds a limit threshold. At failure, a delayed replacement is performed. To shorten the down period, a condition-based maintenance strategy is applied, with imperfect repair. Mimicking virtual age models used for recurrent events, imperfect repair actions are assumed to lower the system degradation through a first-order arithmetic reduction of age model. Under these assumptions, Markov renewal equations are obtained for several reliability indicators. Numerical examples illustrate the behaviour of the system.

A novel optimal preventive maintenance policy for a cold standby system based on semi-Markov theory

A novel optimal preventive maintenance policy for a cold standby system consisting of two components and a repairman is described herein. The repairman is to be responsible for repairing either failed component and maintaining the working components under certain guidelines. To model the operational process of the system, some reasonable assumptions are made and all times involved in the assumptions are considered to be arbitrary and independent. Under these assumptions, all system states and transition probabilities between them are analyzed based on a semi-Markov theory and a regenerative point technique. Markov renewal equations are constructed with the convolution of the cumulative distribution function of system time in each state and corresponding transition probability. By using the Laplace transform to solve these equations, the mean time from the initial state to system failure is derived. The optimal preventive maintenance policy that will provide the optimal preventive maintenance cycle is identified by maximizing the mean time from the initial state to system failure, and is determined in the form of a theorem. Finally, a numerical example and simulation experiments are shown which validated the effectiveness of the policy.

A unified analysis of claim costs up to ruin in a Markovian arrival risk model

An insurance risk model where claims follow a Markovian arrival process (MArP) is considered in this paper. It is shown that the expected present value of total operating costs up to default H, as a generalization of the classical Gerber–Shiu function, contains more non-trivial quantities than those covered in Cai et al. (2009), such as all moments of the discounted claim costs until ruin. However, it does not appear that the Gerber–Shiu function ϕ with a generalized penalty function which additionally depends on the surplus level immediately after the second last claim before ruin (Cheung et al., 2010a) is contained in H. This motivates us to investigate an even more general function Z from which both H and ϕ can be retrieved as special cases. Using a matrix version of Dickson–Hipp operator (Feng, 2009b), it is shown that Z satisfies a Markov renewal equation and hence admits a general solution. Applications to other related problems such as the matrix scale function, the minimum and maximum surplus levels before ruin are given as well.

Analysis of Markov renewal reward process on VLSI cell partitioning

It is well known that various characteristics in risk and queuing process can be formulated as Markov Renewal function. We study the max-position of finitely many Markov Renewal Reward process with countable state space. We define Markov Renewal equation associated with max-posed process. The solutions of the Markov Renewal Reward equation are derived and the asymptotic behaviors of the equations are studied. Also we discuss the idea of delayed Markov renewal reward with state space.

The Geometric Markov Renewal Processes with Application to Finance

We introduce the geometric Markov renewal processes as a model for a security market and study this processes in a series scheme. We consider its approximations in the form of averaged, merged and double averaged geometric Markov renewal processes. Weak convergence analysis and rates of convergence of ergodic geometric Markov renewal processes are presented. Martingale properties, infinitesimal operators of geometric Markov renewal processes are presented and a Markov renewal equation for expectation is derived. As an application, we consider the case of two ergodic classes. Moreover, we consider a generalized binomial model for a security market induced by a position dependent random map as a special case of a geometric Markov renewal process.

Age-usage semi-Markov models

This paper presents a non-homogeneous age-usage semi-Markov model with a measurable state space. Several probability functions useful to assess the system’s reliability are investigated. They satisfy the same family of equations we call indexed Markov renewal equations. Sufficient conditions to assure the existence and uniqueness of their solutions are provided. The numerical analysis of these equations is executed through the construction of a process discrete in time and space, which is shown to converge to the continuous one in the Skorohod topology. An algorithm useful for solving the discretized system of equations is presented by using a matrix representation.

A method to compute the transition function of a piecewise deterministic Markov process with application to reliability

We study the time evolution of an increasing stochastic process governed by a first-order stochastic differential system. This defines a particular piecewise deterministic Markov process (PDMP). We consider a Markov renewal process (MRP) associated to the PDMP and its Markov renewal equation (MRE) which is solved in order to obtain a closed-form solution of the transition function of the PDMP. It is then applied in the framework of survival analysis to evaluate the reliability function of a given system. We give a numerical illustration and we compare this analytical solution with the Monte Carlo estimator.

Numerical Bounds for Semi-Markovian Quantities and Application to Reliability

We propose new easily computable bounds for different quantities which are solutions of Markov renewal equations linked to some continuous-time semi-Markov process (SMP). The idea is to construct two new discrete-time SMP which bound the initial SMP in some sense. The solution of a Markov renewal equation linked to the initial SMP is then shown to be bounded by solutions of Markov renewal equations linked to the two discrete time SMP. Also, the bounds are proved to converge. To illustrate the results, numerical bounds are provided for two quantities from the reliability field: mean sojourn times and probability transitions.

Upper and lower bounds for the solutions of Markov renewal equations

Upper and lower bounds are studied for the solutions of Markov renewal equations. Some of their special cases are derived under specific marginal conditons and in an alternating environment. The method to construct the bounds is also explained in detail. At the end, these bounds are applied to a shock model and an age-dependent branching process under Markovian environment.

The stationary tail asymptotics in the GI/G/1-type queue with countably many background states

We consider the asymptotic behaviour of the stationary tail probabilities in the discrete-time GI/G/1-type queue with countable background state space. These probabilities are presented in matrix form with respect to the background state space, and shown to be the solution of a Markov renewal equation. Using this fact, we consider their decay rates. Applying the Markov renewal theorem, it is shown that certain reasonable conditions lead to the geometric decay of the tail probabilities as the level goes to infinity. We exemplify this result using a discrete-time priority queue with a single server and two types of customer.

The stationary tail asymptotics in the GI/G/1-type queue with countably many background states

We consider the asymptotic behaviour of the stationary tail probabilities in the discrete-time GI/G/1-type queue with countable background state space. These probabilities are presented in matrix form with respect to the background state space, and shown to be the solution of a Markov renewal equation. Using this fact, we consider their decay rates. Applying the Markov renewal theorem, it is shown that certain reasonable conditions lead to the geometric decay of the tail probabilities as the level goes to infinity. We exemplify this result using a discrete-time priority queue with a single server and two types of customer.