Vertical Take-Off and Landing (VTOL) System Based On Noise Parameter for Exploring of Continuous Reinforcement Learning Approach

Abstract

Research about air-craft autonomous vehicles has become a trend issue among researchers. Vertical Take-Off and Landing (VTOL) controlling for a hover-craft vehicle is one of benchmarking control problems. VTOL ability of hover-craft is significantly crucial for the vehicle to do surveillance and rescue, military, farming, etc. There are many papers concerned with this VTOL benchmarking control. The purpose of this paper is applying Deep Deterministic Policy Gradient (DDPG) as the brand new continuous deep reinforcement learning approach to solve a complex nonlinear system, such as air-craft VTOL problem. Furthermore, this paper analyses the effect of Ornstein-Uhlenbeck (OU) Noise injection to action as the exploration key issue of DDPG algorithm, in addition of the analysis, this paper also applies the gaussian noise to the plant as the disturbance of system. The test was performed in the VTOL simulation. The evaluation used the episode reward, average reward, total reward from the training process. Further analysis also uses IAE (Integral of Absolute Magnitude of Error), ISE (Integral of the Square Error) and MSE (Mean of the Square Error) as the parameter error evaluation from the system. The comparison results of all conditions this paper show that applying DDPG with the injection of OU noise to VTOL system with the disturbance of Gaussian noise achieves the best total reward of −14,355.37 with the shortest episode training of 243, thus it works best. The VTOL system without the disturbance has higher IAE and VTOL with Gaussian noise as the disturbance has higher ISE and MSE.