Transient Solutions of Time-Inhomogeneous Markov Reward Models with Discontinuous Rates

Abstract

A time-inhomogeneous Markov reward model allows globally time-dependent transitions and incorporates cumulative reward measures to provide figures of merit for work performed. Transient solutions of such models are useful in combined reliability and performance evaluation. When this approach is generalized to the modeling of phased-mission systems, phase changes appear in the model as transitions with discontinuous rates. This paper discusses the solution of such models using an ordinary differential equation solver in combination with event queue and recalculation list control, adaptive step size techniques, and explicit reward measure integration for calculating the expected value of accumulated reward. A solution tool, called PUMA, has been implemented to demonstrate these concepts.