Planning in stochastic domains for multiple agents with individual continuous resource state-spaces

Abstract

An approximation method is proposed that solves a class of Decentralized hybrid Markov Decision Processes (DEC-HMDPs). These DEC-HMDPs have both discrete and continuous state variables and represent individual agents with continuous measurable state-spaces, such as resources. Adding to the natural complexity of decentralized problems, continuous state variables lead to a blowup in potential decision points. Representing value functions as Rectangular Piecewise Constant (RPWC) functions, we formalize and detail an extension to the Coverage Set algorithm (CSA) (Becker et al., J Artif Intell Res, 22, 2004) that solves transition independent DEC-HMDPs with controlled error. The resource constraints of each agent lead to problems that are over-subscribed in the number of agents, that is where some agents have no role to play. Based on our extension to the CSA, two heuristics are proposed that allow A*-like search to find the minimal optimal team of agents that is solution to a given problem. We apply and test our algorithms on a range of multi-robot exploration problems with continuous resource constraints.