Self-Learning MAV Under Safety-Guaranteed Flight Test Environment

Abstract

This paper presents a novel methodology for designing the flight control system of a micro aerial vehicle (MAV); it allows the MAV to learn how to fly by itself, without the risk of damage. The proposed methodology uses a magnetic levitation–based safety-guaranteed flight test environment and deep reinforcement learning techniques to avoid the current problems in flight control system design procedures, such as the reality gap issue of simulations, and the safety issues inherent to real flight testing. The safety-guaranteed flight test environment was achieved using a developed magnetic suspension and balance system, which dynamically adjusts magnetic forces interacting with a magnetically levitated MAV. As a result, the MAV can perform in either a free-flight condition for reinforcement learning or can be constrained for safety. This learning environment was developed to permit safe reinforcement learning of MAV, since trial-and-error-based learning typically makes the MAV unstable. In this regard, the MAV can learn to fly by itself based on trial and error, by interacting with the emulated free-flight test environment. Notably, the entire learning process can be conducted without numerical models for both the MAV itself and the flight environment, and the safety of the MAV is guaranteed, even when attempting undesirable actions that might cause the model to become unstable. By avoiding the modeling errors typical of computational simulations and preventing the risk of damage in real flight tests, this approach could enhance the use of reinforcement learning in the development of advanced flight control systems.