Off-policy reinforcement learning for distributed output synchronization of linear multi-agent systems

Abstract

In this paper, off-policy reinforcement learning (RL) is used to find a model-free optimal solution to the H ∞ output synchronization of heterogeneous multi-agent discrete-time systems. First, the output synchronization problem is formulated as a set of local optimal tracking problems. It is shown that optimal local synchronization control protocols can be found by solving augmented game algebraic Riccati equations (GAREs). The solutions to the GAREs require the state of the leader for all agents and the knowledge of agent dynamics. To obviate this requirement, a distributed adaptive observer is designed to estimate the leader state for all agents without requiring complete knowledge of the leader dynamics. Moreover, off-policy RL algorithm is used to learn the solution to the GAREs using only measured data and without requiring the knowledge of the agent or the leader dynamics. In the proposed approach, in contrast to other model free approaches, the disturbance input does not need to be adjusted in a specific manner. A simulation example is given to show the effectiveness of the proposed method.