Learning data-driven decision-making policies in multi-agent environments for autonomous systems

Abstract

Autonomous systems such as Connected Autonomous Vehicles (CAVs), assistive robots are set improve the way we live. Autonomous systems need to be equipped with capabilities to Reinforcement Learning (RL) is a type of machine learning where an agent learns by interacting with its environment through trial and error, which has gained significant interest from research community for its promise to efficiently learn decision making through abstraction of experiences. However, most of the control algorithms used today in current autonomous systems such as driverless vehicle prototypes or mobile robots are controlled through supervised learning methods or manually designed rule-based policies. Additionally, many emerging autonomous systems such as driverless cars, are set in a multi-agent environment, often with partial observability. Learning decision making policies in multi-agent environments is a challenging problem, because the environment is not stationary from the perspective of a learning agent, and hence the Markov properties assumed in single agent RL does not hold. This paper focuses on learning decision-making policies in multi-agent environments, both in cooperative settings with full observability and dynamic environments with partial observability. We present experiments in simple, yet effective, new multi-agent environments to simulate policy learning in scenarios that could be encountered by an autonomous navigating agent such as a CAV. The results illustrate how agents learn to cooperate in order to achieve their objectives successfully. Also, it was shown that in a partially observable setting, an agent was capable of learning to roam around its environment without colliding in the presence of obstacles and other moving agents. Finally, the paper discusses how data-driven multi-agent policy learning can be extended to real-world environments by augmenting the intelligence of autonomous vehicles.