Reinforcement Learning of Structured Stabilizing Control for Linear Systems With Unknown State Matrix

Abstract

This article delves into designing stabilizing feedback control gains for continuous-time linear systems with unknown state matrix, in which the control gain is subjected to a structural constraint. We bring forth the ideas from reinforcement learning (RL) in conjunction with sufficient stability and performance guarantees in order to design these structured gains using the trajectory measurements of states and controls. We first formulate a model-based linear quadratic regulator (LQR) framework to compute the structured control gain. Subsequently, we transform this model-based LQR formulation into a data-driven RL algorithm to remove the need for knowing the system state matrix. Theoretical guarantees are provided for the stability of the closed-loop system and the convergence of the structured RL (SRL) algorithm. A remarkable application of the proposed SRL framework is in designing distributed static feedback control, which is necessary for automatic control of many large-scale cyber–physical systems. As such, we validate our theoretical results with numerical simulations on a multiagent networked linear time-invariant dynamical system.