Filter based Explorized Policy Iteration Algorithm for On-Policy Approximate LQR

Abstract

A filter-based policy iteration (PI) algorithm has been proposed to design an adaptive optimal controller (AOC) for uncertain continuous linear time invariant (LTI) systems. A novel two-layered filtering architecture is introduced in the PI algorithm- the first layer filters tactically eliminate the need for state derivative knowledge and finite window integrals (FWI), while the second layer filters provide suitable algebraic relations which obviates the requirement of intelligent data-storage. In addition to promising the intermediate policies to be stabilizing and converging to the optimal policy under an excitation assumption which is imposed through the injection of exploratory noise in the iterative policies, the proposed work analytically guarantees stability of the closed-loop switched dynamics.