Reinforcement learning-based hybrid differential evolution for global optimization of interplanetary trajectory design

Abstract

The global optimization and design of interplanetary trajectories is one of the most important tasks in deep space exploration. Its search space is often characterized by multi-constraints, extreme non-linearity and sensitivity to initial conditions. To cope with these difficulties, an improved reinforcement learning-based hybrid differential evolution (DE) named RL_HDE is proposed in this paper. In RL_HDE, a novel multi-mutation strategy LSHADE based on an adaptive Q-Learning framework is proposed for global exploration. To further balance the global exploration and local exploitation of RL_HDE, a new parameter adaptive strategy based on Q-Learning is designed to control two trigger parameters. The performance of RL_HDE is verified by the well-known Global Trajectory Optimization Problems (GTOP), which are developed by Advanced Concepts Team of European Space Agency (ESA-ACT). A comparison of RL_HDE with three sets of algorithms, including ten state-of-the-art hybrid evolutionary algorithms, four interplanetary trajectory design algorithms, and seven reinforcement learning-based hybridizations. Experimental statistical results demonstrate that RL_HDE outperforms other competitors in terms of convergence efficiency and accuracy. RL_HDE has better performance for solving complex interplanetary trajectory design problems such as Cassini2 and Messenger-full.