Markov Processes

Abstract

A multi-state system is often applied for system degradation process modeling because it offers the possibility of describing the degradation state of the components and system by a number of consecutive levels from perfect working to complete failure. To model the dynamics and transitions of such a multi-state degradation process, Markov models have often been used. In doing this, the rates of transition among the degradation states are typically assumed to be constant, which implies that the degradation process is memoryless. The resulting stochastic process is called a homogeneous continuous time Markov chain (HCTMC). The inhomogeneous continuous time Markov chain is more suited than HCTMC for modeling the degradation process. An semi-Markov process is one that changes states in accordance with a Markov chain but takes arbitrary amounts of time between changes. Piecewise-deterministic Markov process is adopted to treat the system dynamics and the degradation dependence in multistate physics systems.