Design of Self-Organizing Systems Using Multi-Agent Reinforcement Learning and the Compromise Decision Support Problem Construct

Abstract

Abstract How can multi-robot self-organizing systems be designed so that they show the desired behaviors and are able to perform tasks specified by designers? Multi-robot self-organizing systems, e.g., swarm robots, have great potential for adapting when performing complex tasks in a changing environment. However, such systems are difficult to design due to the stochasticity of the system performance and the non-linearity between the local actions/interaction and the desired global behavior. In order to address this, in this paper we propose a framework for designing self-organizing systems using Multi-Agent Reinforcement Learning (MARL) and the compromise Decision-Support Problem (cDSP) construct. The design framework consists of two stages — preliminary design and design improvement. In the preliminary design stage, MARL is used to help designers train the robots so that they show stable group behavior for performing the task. In the design improvement stage, the cDSP construct is used to enable designers to explore the design space and identify satisfactory solutions considering several performance indicators based on the trained system established in the previous stage. Between the two stages, surrogate models are used to map the relationship between local parameters and global performance indicators utilizing the data generated in preliminary design. The surrogate models represent the goal functions in the cDSP. A multi-robot box-pushing problem is used as an example to test the efficacy of the proposed framework. The framework is general and can be extended to design other multi-robot self-organizing systems. Our focus in this paper is in describing the framework.