Multi-agent cooperative operation based on cross-domain zero-shot learning

Abstract

Multi-agent systems (MAS) based on reinforcement learning have shown impressive performance in complex tasks. However, they are still weak when it comes to addressing tasks without any samples. In this paper, we design a 2D ground and a 3D underwater cooperative task, which serve as the source and target tasks, respectively. The MAS requires learning prior knowledge that can be generalized to the target task from the source task. To solve this problem, we propose cross-domain zero-shot learning (CDZSL) for MAS cooperative operation. Specifically, we first construct a state transition graph by decomposing the source tasks into multiple state phases. Subsequently, we describe the task description document for MAS and construct the task-related semantic embedding space, which effectively combats the semantic gap. Then, we propose zero-shot learning based on feature synthesis mapping (FSM), which synthesizes the feature centers of the unseen classes by the features of seen classes. During knowledge transfer, FSM is employed to assist MAS's decisions-making in the target task. Finally, extensive experiments show that CDZSL, compared with MAX-Q, improves the adaptability of MAS, where the cost of physical time for the first time to complete the new task is reduced by eight times.