Leader-Follower Based Online Reinforcement Learning Algorithm in Problem with Hierarchy Decision Makers

Abstract

This paper presents an online integral reinforcement learning (RL) solution for problems with hierarchy decision makers. Specifically, we reformulate this model as a leader-follower game in which control input and deterministic disturbance act as decision makers at different levels of hierarchy: the control input plays the role of the leader while the disturbance plays the role of the follower. The main contributions of this paper can be summarized as follows. First, we introduce online RL to deal with systems that have partially unknown information, meaning that accurate dynamic information is not required. Second, we solve the leader-follower coupled Hamilton-Jacobi (HJ) and Riccati equations approximately online using the derived algorithm. Third, we provide turning laws for cost functions and controllers that ensure closed-loop stability simultaneously.