Abstract
Cooperation and coordination are major issues in studies on multi-agent systems because the entire performance of such systems is greatly affected by these activities. The issues are challenging however, because appropriate coordinated behaviors depend on not only environmental characteristics but also other agents’ strategies. On the other hand, advances in multi-agent deep reinforcement learning (MADRL) have recently attracted attention, because MADRL can considerably improve the entire performance of multi-agent systems in certain domains. The characteristics of learned coordination structures and agent’s resulting behaviors, however, have not been clarified sufficiently. Therefore, we focus here on MADRL in which agents have their own deep Q-networks (DQNs), and we analyze their coordinated behaviors and structures for the pickup and floor laying problem, which is an abstraction of our target application. In particular, we analyze the behaviors around scarce resources and long narrow passages in which conflicts such as collisions are likely to occur. We then indicated that different types of inputs to the networks exhibit similar performance but generate various coordination structures with associated behaviors, such as division of labor and a shared social norm, with no direct communication.
Highlights
Cooperation and coordination are important issues in the study of multi-agent systems, because they are essential to achieve the desired autonomous control to improve the overall efficiency in sophisticated cooperative tasks
Because we focus on multiagent deep reinforcement learning (MADRL) for a multi-agent pickup and floor laying problem, the agents individually learn the Q-values and associated policies by using their own deep Q-networks (DQNs) to identify cooperative and coordinated behaviors without direct communication
We evaluated the performance on the pickup and floor laying problem by agents with the proposed input information (RVs, historic relative view (HRV), and Local view (LV)) and various observable
Summary
Cooperation and coordination are important issues in the study of multi-agent systems, because they are essential to achieve the desired autonomous control to improve the overall efficiency in sophisticated cooperative tasks It is difficult and complicated to identify appropriate coordination structures and behavioral rules or strategies, such as social norms and division of work areas. Our experimental results indicate that the agents in distributed MADRL could generate a number of coordinated structures for cooperative work, but the structures were quite different depending on the input information fed to the agents’ own DQNs. For example, when the agents had absolute locations in the environment, they formed divisional cooperation by spatial segmentation. We discuss the relationships between the agents’ coordination structures and performance in the pickup and floor laying problem
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
