Option-Based Multi-Agent Reinforcement Learning for Painting With Multiple Large-Sized Robots

Abstract

To ensure an efficient and conflict-free task allocation, this paper presents an option-based multi-agent reinforcement learning (OMARL) method for a cooperative multi-station multi-robot system in aircraft painting application. The problem can be described as a fully-connected graph, whose nodes and edges represent stations and trajectories between stations, respectively. The aim is to minimize the maximum execution time of conflict-free aircraft painting with multiple robots. Traditional methods cannot handle mutual collision avoidance well when robots are in large size. Therefore, this work solved the planning problem with multi-agent reinforcement learning algorithm based on option framework. The moving trajectories were pre-planned by a simplified 2D A* algorithm, which is fast and reduces rotations. The agents would finish the whole task with less time, and potential collisions were punished at every time step. Hierarchical option-based model with shared network parameters greatly reduced the complexity of training for painting planning scenarios. A fuzzy clustering method was introduced, and the clustering scores of stations would contribute to greedy agents. Comparisons were made on static and dynamic situations among several planning methods. OMARL can give optimal schedules as well as, or better than the most efficient traditional method does. Furthermore, OMARL can provide satisfactory adjustments in real time when practical process is different from plan or there is one broken robot. In contrast, traditional methods take lots of time to recalculate the plan. The results indicate that OMARL can achieve better performance in multi-station multi-robot task allocation when the collision between robots cannot be ignored.