Policy iteration-based indirect adaptive optimal control for completely unknown continuous-time LTI systems

Abstract

This paper proposes a novel indirect adaptive optimal controller (AOC) for completely unknown continuous-time (CT) linear time invariant (LTI) systems using the policy iteration (PI) technique. The algorithm builds on the Kleinman's method of iteratively solving the algebraic Riccati equation (ARE). However, the actual system and control matrices information, required by the Kleinman's algorithm, is replaced by their CT online estimates using uniform sampling. A gradient-based online system identifier is developed using a low pass filter, which strategically eliminates the need for state derivative information, while the system identifier exponentially converges to the actual plant-parameter vector under the assumption of persistence of excitation (PE). The proposed online identifier based Kleinman's algorithm is shown to converge to the optimal control policy while preserving the stabilizability of the intermediate policies for the unknown CT LTI systems as validated through simulation studies on multi-input-multi-output (MIMO) LTI systems. The designed indirect AOC is argued to be computationally less intricate as compared to the past literature on direct AOC.