Design of a Reinforcement Learning PID Controller

Abstract

This paper addresses a design scheme of a proportional‐integral‐derivative (PID) controller with a new adaptive updating rule based on reinforcement learning (RL) approach for nonlinear systems. A new design scheme that RL can be used to complement the conventional PID control technology is presented. In the proposed scheme, a single radial basis function (RBF) network is considered to calculate the control policy function of Actor and the value function of Critic simultaneously. Regarding the PID controller structure, the inputs of RBF network are system errors, the difference of output as well as the second‐order difference of output, and they are composed of system states. The temporal difference (TD) error in the proposed scheme involves the reinforcement signal, the current and the previous stored value of the value function. The gradient descent method is adopted based on the TD error performance index, then, the updating rules can be yielded. Therefore, the network weights and the kernel function can be calculated in an adaptive way. Finally, the numerical simulations are conducted in nonlinear systems to illustrate the efficiency and robustness of the proposed scheme. © 2021 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.