Regenerative randomization

Abstract

Randomization is a popular method for the transient solution of continuous-time Markov models. Its primary advantages over other methods (i.e., ODE solvers) are robustness and ease of implementation. It is however well-known that the performance of the method deteriorates with the "stiffness" of the model: the number of required steps to solve the model up to time t tends to Λ t for Λ t → ∞. In this paper we present a new method called regenerative randomization and apply it to the computation of two transient measures for rewarded irreducible Markov models. Regarding the number of steps required in regenerative randomization we prove that: 1) it is smaller than the number of steps required in standard randomization when the initial distribution is concentrated in a single state, 2) for Λ t → ∞, it is upper bounded by a function O (log(Λ t /ε)), where ε is the desired relative approximation error bound. Using dependability and performability examples we analyze the performance of the method.