Multiagent Graphical Games With Inverse Reinforcement Learning

Abstract

This work investigates inverse reinforcement learning (RL) for multiagent systems (MAS) defined by Graphical Apprentice Games. These games are solved by a learner MAS finding the unknown cost functions of an expert MAS from its demonstrated behavior. We begin by developing a model-based inverse RL algorithm including two update loops:1) an inner-loop optimal control update and 2) an outer-loop inverse optimal control (IOC) update. We then introduce a model-free inverse RL algorithm that uses online behaviors of the expert and learner MAS without knowing their dynamics. The optimal control and IOC are solved as subproblems in both proposed inverse RL algorithms. The reward functions that the learner MAS finds are proven to be both stabilizing and nonunique. Simulated case studies validate the effectiveness of the proposed inverse RL algorithms.