Numerical analysis of large Markov reward models

Abstract

A first analysis of Markov Reward Models (MRM) resulted in a double transform expression, whose numerical solution is based on the inverse transformations both in time and reward variable domain. Better numerical methods were proposed based on the time domain properties of these models, such as the set of partial differential equations describing the process evolution in time. This paper introduces an effective numerical method for the analysis of MRMs based on the transform domain description of the system, which allows the evaluation of models with large state space (∼10 6 states). The proposed method provides the moments of reward measures on the same computational cost and memory requirement as the transient analysis of the underlying Continuous Time Markov Chain and benefits from the advantages of the randomization method, which avoids numerical instabilities and provides global error bound in advance of the computation. Implementation notes and numerical examples demonstrate the numerical properties of the proposed method are also provided.