Abstract
We consider Markov decision processes (MDPs) with Büchi (liveness) objectives. We consider the problem of computing the set of almost-sure winning states from where the objective can be ensured with probability 1. Our contributions are as follows: First, we present the first subquadratic symbolic algorithm to compute the almost-sure winning set for MDPs with Büchi objectives; our algorithm takes \(O(n \cdot\sqrt{m})\) symbolic steps as compared to the previous known algorithm that takes O(n 2) symbolic steps, where n is the number of states and m is the number of edges of the MDP. In practice MDPs have constant out-degree, and then our symbolic algorithm takes \(O(n \cdot\sqrt{n})\) symbolic steps, as compared to the previous known O(n 2) symbolic steps algorithm. Second, we present a new algorithm, namely win-lose algorithm, with the following two properties: (a) the algorithm iteratively computes subsets of the almost-sure winning set and its complement, as compared to all previous algorithms that discover the almost-sure winning set upon termination; and (b) requires \(O(n \cdot\sqrt{K})\) symbolic steps, where K is the maximal number of edges of strongly connected components (scc’s) of the MDP. The win-lose algorithm requires symbolic computation of scc’s. Third, we improve the algorithm for symbolic scc computation; the previous known algorithm takes linear symbolic steps, and our new algorithm improves the constants associated with the linear number of steps. In the worst case the previous known algorithm takes 5⋅n symbolic steps, whereas our new algorithm takes 4⋅n symbolic steps.
Highlights
The qualitative analysis of an Markov decision processes (MDPs) with a parity objective with d priorities can be achieved by O(d) calls to an algorithm for qualitative analysis of MDPs with Buchi objectives, and we focus on the qualitative analysis of MDPs with Buchi objectives
Given an MDP with a Buchi objective, the WINLOSE algorithm iteratively computes the subsets of the almost-sure winning set and its complement, and in the end correctly computes the set 1 almost (Buchi(T )) and the algorithm runs in time O(KS · m), where KS is the maximum number of states in an scc of the graph of the MDP
Given an MDP with a Buchi objective, the IMPRWINLOSE algorithm iteratively computes the subsets of the almost-sure winning set and its complement, and in the en√d correctly computes the set 1 almost (Buchi(T )) and the algorithm runs in time O( KE · m), where KE is the maximum number of edges in an scc of the graph of the MDP
Summary
One of the key challenges in probabilistic verification is to obtain efficient and symbolic algorithms for qualitative analysis of MDPs with parity objectives, as symbolic algorithms allow to handle MDPs with a large state space. The classical algorithm for qualitative analysis for MDPs with Buchi objectives works in O(n·m) time, where n is the number of states, and m is the number of edges of the MDP [6, 7]. 2. All previous algorithms for the qualitative analysis of MDPs with Buchi objectives iteratively discover states that are guaranteed to be not almost-sure winning, and only when the algorithm terminates the almost-sure winning set is discovered. Our experimental results show that our new algorithms perform better than the previous known algorithms both for qualitative analysis of MDPs with Buchi objectives and symbolic scc computation
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
