An innovative bio-inspired flight controller for quad-rotor drones: Quad-rotor drone learning to fly using reinforcement learning

Abstract

Animals learn to master their capabilities by trial and error, and with out having any knowledge about their dynamics model and mathematical or physical rules. They use their maximum capabilities in an optimized way. This is the result of millions of years of evolution where the best of different possibilities are kept, and makes us rethink How does the nature perform things?, particularly when natural systems outperform our rigid systems. In this study, inspired by the nature, we developed an innovative algorithm by enhancing an existing reinforcement learning algorithm (proximal policy optimization (PPO)). Our algorithm is capable of learning to control a quad-rotor drone in order to fly. This new algorithm called Bio-inspired Flight Controller (BFC) does not use any conventional controller such as PID or MPC to control the quad-rotor drone. The goal of BFC is to completely replace the conventional controller with a controller that acts in a similar way to the animals where they learn to control their movements. It is capable of stabilizing a quad-copter in a desired point, and following way points. We implemented our algorithm in an AscTec Hummingbird quad-copter simulated in Gazebo, and tested it using different scenarios to fully measure its capabilities.