Worst-Case Analysis of Complex Nonlinear Flight Control Designs Using Deep Q-Learning

Abstract

With the objective of exposing hidden design deficiencies in complex nonlinear systems, this paper presents the use of reinforcement learning techniques for application in flight control law development and testing. Following the idea of worst-case testing, a deep Q-network agent is trained to identify input sequences that lead to detrimental system behavior. Because the analysis is based directly on the repeated interaction between the agent and the investigated system, no model simplifications are required, making the presented method applicable to highly complex systems. The capability of the learning-based worst-case analysis is demonstrated for the speed protection function of the hover flight control law of an electric vertical takeoff and landing (eVTOL) aircraft. The analysis discovers possible piloted maneuvers that violate the implemented protection algorithm. A root cause analysis of the emerging behavior reveals the neglect of an important flight mechanical coupling term in the design of the protection algorithm and ultimately leads to the revision and improvement of the controller. This demonstrates the benefits of the presented testing method for the design, verification, and validation of complex systems. The application to a high-fidelity system used for control law development of an actual eVTOL prototype currently under construction demonstrates the relevance of the method beyond academia.