Learning to Perform a Perched Landing on the Ground Using Deep Reinforcement Learning

Abstract

A UAV with a variable sweep wing has the potential to perform a perched landing on the ground by achieving high pitch rates to take advantage of dynamic stall. This study focuses on the generation and evaluation of a trajectory to perform a perched landing on the ground using a non-linear constraint optimiser (Interior Point OPTimizer) and a Deep Q-Network (DQN). The trajectory is generated using a numerical model that characterises the dynamics of a UAV with a variable sweep wing which was developed through wind tunnel testing. The trajectories generated by a DQN have been compared with those produced by non-linear constraint optimisation in simulation and flown on the UAV to evaluate performance. The results show that a DQN generates trajectories with a lower cost function and have the potential to generate trajectories from a range of starting conditions (on average generating a trajectory takes 174 milliseconds). The trajectories generated performed a rapid pitch up before the landing site is reached, to reduce the airspeed (on average less than 0.5m/s just above the landing site) without generating an increase in altitude, and then the nose dropped just before hitting the ground to allow the aircraft to be recovered without damaging the tail. The trajectories generated by a DQN produced a final airspeed (when it hit the ground) of 3.25m/s (with a standard deviation of 0.97m/s) in the downward direction, which would allow the aircraft to be safely recovered and significantly less than a traditional landing (∼ 10m/s).