Deep Reinforcement Learning Based Controller Design for Model of The Vertical Take off and Landing System

Abstract

In this study, the Deep Deterministic Policy Gradient (DDPG) algorithm, which consists of a combination of artificial neural networks and reinforcement learning, was applied to the Vertical Takeoff and Landing (VTOL) system model in order to control the pitch angle. This algorithm was selected because conventional control algorithms such as Proportional Integral Derivative (PID) controllers which cannot always generate a suitable control signal eliminating the disturbance and unwanted environment effects on the considered system. In order to control the system, training was carried out for a sinusoidal reference in the mathematical model of the VTOL system in the Simulink environment, through the DDPG algorithm with continuous action space from deep reinforcement learning methods that can produce control action values that take the structure that can maximize the reward according to a determined reward function for the purpose of control and the generalization ability of artificial neural networks. For sinusoidal reference and a constant reference, tracking error performances obtained for the pitch angle, which is the output for the specified VTOL system, were compared with the conventional PID controller performance in terms of mean square error, integral square error, integral absolute error, percentage overshoot and settling time. The obtained results are presented via the simulations studies.