Real-time adaptive entry trajectory generation with modular policy and deep reinforcement learning

Abstract

Recently, data-driven entry trajectory generation algorithms of hypersonic vehicles were highlighted for their high accuracy with low computational costs. Learning a unique black box model mapping the state to control variables is expected. However, insufficient adaptability remains a challenging problem when applying the algorithms in practice, especially for multi-missions. In this article, a real-time entry trajectory generation algorithm with modular policy is proposed to achieve adaptability to various missions, such as changing targets and emergency entry. Based on the modular idea, the entry trajectory problem is decomposed into two decision problems, i.e., adjusting the profile of the angle of attack (AOA) to determine the flight capability and planning the bank angle to ensure the satisfaction of task constraints, which correspond to AOA module and bank module respectively. Then, algorithms are developed by utilizing deep reinforcement learning (DRL) to train the two modules with which an intelligent entry trajectory generation algorithm is proposed to achieve real-time trajectory design in a wide range of reachable areas. Moreover, a non-uniform discretization approach with a state-related independent variable is proposed to deal with state misalignment and contradiction in stepsize settings. The simulation results demonstrate that the proposed algorithm can provide a reliable trajectory in a very short time and adapt to various tasks.