Distributed Input Design with Combination of System Identification and Reinforcement Learning

Abstract

The optimal input design of unknown large-scale network systems encounters difficulties when only local observation is available. Traditional system identification methods cannot guarantee to obtain an accurate system model because it cannot access the global information. Meanwhile, the direct use of model-free methods such as reinforcement learning lacks convergence guarantees and efficiency due to lack of prior knowledge of the large scale of the network. To tackle these problems, in this paper, we propose a distributed optimal input design method for large-scale networks by combining the system identification method with the reinforcement learning. First, we propose a distributed system identification method based on the least square method. Then, a distributed optimal input design algorithm is proposed based on deep reinforcement learning to update the results of system identification and minimize the cost function of optimal control at the same time. The identified system model is used as prior knowledge to determine the environment reward to contribute to the convergence of the reinforcement learning algorithm. It is proved that our method can obtain the identification results of system and the optimal input signal via local observation. Simulations demonstrate the effectiveness of the proposed method.