Distributed Adaptive Linear Quadratic Control using Distributed Reinforcement Learning

Abstract

Abstract In this paper distributed adaptive linear quadratic control of discrete-time linear large-scale systems with unknown dynamics using distributed reinforcement learning is studied. Linear quadratic control based on dynamic programming (specifically policy iteration) and adaptive linear quadratic control based on reinforcement learning (especially Q learning) are reviewed first. Then distributed adaptive linear quadratic control is addressed. Two Q functions exploiting the quadratic structure of the value function and leading to a decentralized and a distributed policy are proposed and a decentralized as well as a distributed Q learning algorithm are presented. Finally the concepts are evaluated in a simulation study. The simulation results indicate that the distributed policy is near-optimal.