A Robust Reinforcement Learning System Using Concept of Sliding Mode Control for Unknown Nonlinear Dynamical System

Abstract

In this chapter, a novel control method using a reinforcement learning (RL) (Sutton and Barto (1998)) with concept of sliding mode control (SMC) (Slotine and Li (1991)) for unknown dynamical system is considered. In designing the control system for unknown dynamical system, there are three approaches. The first one is the conventional model-based controller design, such as optimal control and robust control, each of which is mathematically elegant, however both controller design procedures present a major disadvantage posed by the requirement of the knowledge of the system dynamics to identify and model it. In such cases, it is usually difficult to model the unknown system, especially, the nonlinear dynamical complex system, to make matters worse, almost all real systems are such cases. The second one is the way to use only the soft-computing, such as neural networks, fuzzy systems, evolutionary systems with learning and so on. However, in these cases it is well known that modeling and identification procedures for the dynamics of the given uncertain nonlinear system and controller design procedures often become time consuming iterative approaches during parameter identification and model validation at each step of the iteration, and in addition, the control system designed through such troubles does not guarantee the stability of the system. The last one is the way to use the method combining the above the soft-computing method with the model-based control theory, such as optimal control, sliding mode control (SMC), H∞ control and so on. The control systems designed through such above control theories have some advantages, that is, the good nature which its adopted theory has originally, robustness, less required iterative learning number which is useful for fragile system controller design not allowed a lot of iterative procedure. This chapter concerns with the last one, that is, RL system, a kind of soft-computing method, supported with robust control theory, especially SMC for uncertain nonlinear systems. RL has been extensively developed in the computational intelligence and machine learning societies, generally to find optimal control policies for Markovian systems with discrete state and action space. RL-based solutions to the continuous-time optimal control problem have been given in Doya (Doya (2000). The main advantage of using RL for solving optimal