Abstract
Simple stochastic games can be solved by value iteration (VI), which yields a sequence of under-approximations of the value of the game. This sequence is guaranteed to converge to the value only in the limit. Since no stopping criterion is known, this technique does not provide any guarantees on its results. We provide the first stopping criterion for VI on simple stochastic games. It is achieved by additionally computing a convergent sequence of over-approximations of the value, relying on an analysis of the game graph. Consequently, VI becomes an anytime algorithm returning the approximation of the value and the current error bound. As another consequence, we can provide a simulation-based asynchronous VI algorithm, which yields the same guarantees, but without necessarily exploring the whole game graph.
Highlights
Simple stochastic game (SG) [Con92] is a zero-sum two-player game played on a graph by Maximizer and Minimizer, who choose actions in their respective vertices
We implemented both our algorithms as an extension of PRISM-games [CFK+ 13a], a branch of PRISM [KNP11] that allows for modelling SGs, utilizing previous work of [BCC+ 14,Ujm15] for Markov decision processes (MDP) and SG with single-player end component (EC)
We have provided the first stopping criterion for value iteration on simple stochastic games and an anytime algorithm with bounds on the current error
Summary
Simple stochastic game (SG) [Con92] is a zero-sum two-player game played on a graph by Maximizer and Minimizer, who choose actions in their respective vertices ( called states). Stochastic games constitute a fundamental problem for several reasons. The task of solving SG is polynomial-time equivalent to solving perfect information Shapley, Everett and Gillette games [AM09]. SG can model reactive systems, with players corresponding to the controller of the system and to its environment, where quantified uncertainty is explicitly modelled. This is useful in many application domains, ranging from smart energy management [CFK+ 13c] to autonomous urban driving [CKSW13], robot
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
