Adaptive Critic Design Using Policy Iteration Technique for LTI Systems: A Comprehensive Performance Analysis

Abstract

This paper presents adaptive critic design (ACD) using policy iteration technique for adaptive optimal control of continuous-time linear time-invariant (LTI) dynamical systems. A comprehensive performance analysis of control scheme is presented in this paper. ACD using online policy iteration technique provides an adaptive optimal control solution for an infinite horizon problem subject to the real-time dynamics of a continuous-time system. In general, ACD involves a critic and an actor, where the critic evaluates the performance of present control policy and generates a critic signal to update the control action for performance improvement, and the actor provides the control input to the system being controlled. Policy iteration technique which is based on actor–critic structure consists of two-step iteration: policy evaluation and policy improvement. In this paper, the application of control scheme is implemented for a practical example of LTI systems–load frequency control of power system. The modeling, simulation results, and analysis are presented for both of system models without and with integral control. The comparative performance investigation of adaptive critic control scheme and linear quadratic regulator is also presented.