Adaptive closed-loop maneuver planning for low-thrust spacecraft using reinforcement learning

Abstract

Autonomy is an increasingly essential component of future space missions, and new technologies are necessary to accommodate off-nominal occurrences onboard that may pose risks to mission success. Identifying a suitable maneuver plan for onboard, low-thrust mission applications in cislunar space remains challenging, particularly in the case of unanticipated events. This investigation addresses this challenge by demonstrating artificial neural networks as promising tools for accurately estimating startup solutions for a conventional targeting-based iterative guidance and control algorithm. This blended approach produces a robust ‘hybrid’ architecture that simultaneously benefits from the computational simplicity of neural networks and the robustness of differential corrections to satisfy mission requirements. In this paradigm, a multiple shooting scheme is incorporated directly into a reinforcement learning process, tasking the resulting neural network controller with convergence basin identification for trajectory recovery. Rapid low-thrust maneuver planning is demonstrated in a ‘runaway’ spacecraft scenario, where deviation over time from a planned near rectilinear halo orbit path renders stationkeeping ineffective and demands an alternative approach to determine an effective recovery plan. The proposed Neural Network-Initialized Targeting (NNIT) algorithm significantly extends the recovery window compared to a crossing-control orbit maintenance approach and exhibits promising results in identifying both the timing and control components for low-thrust maneuver plans.