The application of continuous action reinforcement learning automata to adaptive PID tuning

Abstract

This paper investigates the application of the continuous action reinforcement learning automata (CARLA) methodology to PID controller parameter tuning. The PLD controller parameters are initially set using the standard Zeigler-Nichols methods (1942). The CARLA then selects parameters stochastically based on a distribution that converges to a Gaussian around the optimal parameter values. The CARLA adaptively tunes the controller parameters online, to minimise a performance criterion such as the sum of time error square. The method has the benefit of producing a controller with improved performance over the Zeigler-Nichols settings that is robust to noise and to the system nonlinearities. Minimal system modelling is required since it can be applied online optimising the parameters for the actual system. The method is demonstrated on various different systems in simulation. It is also demonstrated as a practical example for parameter tuning of a PID controller of an engine idle speed control system for a Ford Zetec 1.8 engine during load change disturbances. Idle speed control is important to prevent engine stall and to help to reduce vehicle emissions.