Verifying infinite Markov chains with a finite attractor or the global coarseness property

Abstract

We consider infinite Markov chains which either have a finite attractor or satisfy the global coarseness property. Markov chains derived from probabilistic lossy channel systems (PLCS) or probabilistic vector addition systems with states (PVASS) are classic examples for these types, respectively. We consider three different variants of the reachability problem and the repeated reachability problem: the qualitative problem, i.e., deciding if the probability is one (or zero); the approximate quantitative problem, i.e., computing the probability up-to arbitrary precision; the exact quantitative problem, i.e., computing probabilities exactly. We express the qualitative problem in abstract terms for Markov chains with a finite attractor and for globally coarse Markov chains, and show an almost complete picture of its decidability of PLCS and PVASS. We also show that the path enumeration algorithm of (P. Iyer et al., 1997) terminates for our types of Markov chain and can thus be used to solve the approximate quantitative reachability problem. Furthermore, a modified variant of this algorithm can solve the approximate quantitative repeated reachability problem for Markov chains with a finite attractor. Finally, we show that the exact probability of (repeated) reachability cannot be effectively expressed in the first-order theory of the reals (R,+,*,/spl les/) for either PLCS or PVASS (unlike for other probabilistic models, e.g., probabilistic pushdown automata (J. Esparza et al., 2004, K. Etessami et al., 2005, J. Esparza et al., 2004).