Multirobot Navigation Using Partially Observable Markov Decision Processes with Belief-Based Rewards

Abstract

For the safe deployment of multirobot systems, accurate position estimation is necessary. We present a multirobot navigation algorithm that accomplishes the dual objective of reaching designated goal positions and maintaining low position uncertainty. We pose the problem as a -partially observable Markov decision process (POMDP) to capture position uncertainty through a belief-based reward. The multirobot system employs a centralized extended Kalman filter (EKF) for state estimation, whose output is used as the observation as well as the belief in the POMDP. Using the EKF output as the observation and belief allows for the integration of the EKF in the action planning module of the system. To solve the POMDP, we adapt the online information particle filter tree algorithm to be compatible with the EKF closed-form output being used as both the observation and belief. We present results for the proposed navigation algorithm that show emergent behaviors for a multirobot system, where the robots move to provide inter-ranging measurements that minimize position uncertainty for all robots in the system. We also demonstrate that the proposed method outperforms a state-of-the-art solver (partially observable Monte Carlo planning with observation widening) and a direct-to-goal planner in terms of minimizing position uncertainty.