Value Iteration and Data-Driven Optimal Output Regulation of Linear Continuous-Time Systems

Abstract

This paper investigates the linear optimal output regulation problem (LO <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> RP). We propose a reinforcement learning (RL) based approach to learn the optimal regulator. This problem is solved by tackling two optimization problems, a static constrained optimization problem to find the optimal solution to the output regulation equations and a dynamic programming to obtain the optimal feedback control gain. Instead of relying on the prior knowledge of the system dynamics and an initial stabilizing feedback control gain, a novel online value iteration (VI) algorithm is proposed, which can learn the optimal feedback control gain and feedforward control gain using measurable data. Finally, numerical analysis is provided to show that the proposed approach results in desired disturbance rejection and tracking performance.