Markovian imprecise jump processes: Extension to measurable variables, convergence theorems and algorithms

Abstract

The existing framework of Markovian imprecise jump processes, also known as imprecise continuous-time Markov chains, is limited to bounded real variables that depend on the state of the system at a finite number of (future) time points. This is an issue in many applications, because typically the variables of interest depend on the state of the system at all time points in some – possibly unbounded – (future) interval, and they can be unbounded or even extended real valued; examples of such variables are temporal averages, the number of (selected) jumps in some interval and hitting times. To eliminate this shortcoming, we assume that the sample paths are càdlàg and use measure theory to extend the domain of Markovian imprecise jump processes to extended real-valued variables that may depend on the state of the system at all (future) time points – that is, the extended real variables that are bounded below or above and are measurable with respect to the σ-algebra generated by the cylinder events. We investigate the continuity properties of the extended lower and upper expectations with respect to point-wise convergent sequences, and this yields generalisations of the Monotone Convergence Theorem and Lebesgue's Dominated Convergence Theorem. For two particular classes of variables, we strengthen these convergence theorems and present an iterative scheme to approximate their lower and upper expectations. The first class is the number of selected jumps in some interval, and the second class are real variables that take the form of a Riemann integral over some interval; this second class includes temporal averages and occupancy times.