A GNN-based teacher–student framework with multi-advice

Abstract

Multi-agent learning involves the interaction of multiple agents with the environment to learn an optimal policy. To enhance learning performance, a commonly used approach is the teacher–student framework, which enables agents to seek advice from other agents. However, the current literature suffers from a common limitation, wherein a student agent can only receive advice from a single teacher agent at each time step, thus constraining the knowledge acquired. Although some methods allow advice from multiple teachers, they require pre-training of the teachers, which is impractical when agents concurrently learn from scratch simultaneously. Additionally, the importance of the agents’ connection structures is often disregarded, despite its critical role in the advice-giving process. To overcome these limitations, we propose a novel advising approach utilizing graph neural networks (GNNs). This method models the agents’ connection structures, learns the weight of advice, and aggregates inputs from multiple teachers to generate refined advice. Our experiments show that this proposed approach has superior learning performance compared to advising baseline methods.